Examples for the Learning Spark book. These examples require a number of libraries and as such have long build files. We have also added a stand alone example with minimal dependencies and a small build file in the mini-complete-example directory.
These examples have been updated to run against Spark 1.3 so they may be slightly different than the versions in your copy of "Learning Spark".
- JDK 1.7 or higher
- Scala 2.10.3
- scala-lang.org
- Spark 1.3
- Protobuf compiler
- On debian you can install with sudo apt-get install protobuf-compiler
- R & the CRAN package Imap are required for the ChapterSixExample
- The Python examples require urllib3
From spark just run ./bin/pyspark ./src/python/[example]
You can also create an assembly jar with all of the dependencies for running either the java or scala versions of the code and run the job with the spark-submit script
./sbt/sbt assembly OR mvn package cd $SPARK_HOME; ./bin/spark-submit --class com.oreilly.learningsparkexamples.[lang].[example] ../learning-spark-examples/target/scala-2.10/learning-spark-examples-assembly-0.0.1.jar
