Currently browsing tag

pyspark

Package and Distribute PySpark with PyInstaller

One of my customer asked how to package PySpark application in one file with PyInstaller, after some research, I got the answer …

Pay attention to union function of pyspark

In SQL the UNION clause combines the results of two SQL queries into a single table of all matching rows. The two queries must result in the same number …

How two read SAS data with PySpark

For some reason, I have to convert sas data to hdfs then analyse with  pyspark. after some research I found spark-sas7bdat is …