Starting with Spark 2.2, it is now super easy to set up pyspark.
-
Download Spark
Download the spark tarball from the Spark website and untar it:
$ tar zxvf spark-2.2.0-bin-hadoop2.7.tgz
-
Install pyspark
If you use
conda
, simply do:$ conda install pyspark
or if you prefer
pip …