Skip to content

Commit

Permalink
Merge pull request apache#484 from tdas/run-example-fix
Browse files Browse the repository at this point in the history
Made run-example respect SPARK_JAVA_OPTS and SPARK_MEM.

bin/run-example scripts was not passing Java properties set through the SPARK_JAVA_OPTS to the example. This is important for examples like Twitter** as the Twitter authentication information must be set through java properties. Hence added the same JAVA_OPTS code in run-example as it is in bin/spark-class script.

Also added SPARK_MEM, in case someone wants to run the example with different amounts of memory. This can be removed if it is not tune with the intended semantics of the run-example scripts.

@matei Please check this soon I want this to go in 0.9-rc4
  • Loading branch information
pwendell committed Jan 21, 2014
2 parents 6b4eed7 + 65869f8 commit c67d3d8
Showing 1 changed file with 11 additions and 2 deletions.
13 changes: 11 additions & 2 deletions bin/run-example
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,20 @@ else
fi
fi

# Set JAVA_OPTS to be able to load native libraries and to set heap size
JAVA_OPTS="$SPARK_JAVA_OPTS"
JAVA_OPTS="$JAVA_OPTS -Djava.library.path=$SPARK_LIBRARY_PATH"
# Load extra JAVA_OPTS from conf/java-opts, if it exists
if [ -e "$FWDIR/conf/java-opts" ] ; then
JAVA_OPTS="$JAVA_OPTS `cat $FWDIR/conf/java-opts`"
fi
export JAVA_OPTS

if [ "$SPARK_PRINT_LAUNCH_COMMAND" == "1" ]; then
echo -n "Spark Command: "
echo "$RUNNER" -cp "$CLASSPATH" "$@"
echo "$RUNNER" -cp "$CLASSPATH" $JAVA_OPTS "$@"
echo "========================================"
echo
fi

exec "$RUNNER" -cp "$CLASSPATH" "$@"
exec "$RUNNER" -cp "$CLASSPATH" $JAVA_OPTS "$@"

0 comments on commit c67d3d8

Please sign in to comment.