Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Shark 0.9.1 support #45

Merged
merged 2 commits into from
Apr 11, 2014
Merged

Add Shark 0.9.1 support #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3969e47
Add Shark 0.9.1 support
harveyfeng Apr 11, 2014
b35e59d
Add popd's before returning in shark/init.sh
harveyfeng Apr 11, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions shark/init.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ pushd /root

if [ -d "shark" ]; then
echo "Shark seems to be installed. Exiting."
popd
return
fi

Expand Down Expand Up @@ -45,8 +46,16 @@ else
wget http://s3.amazonaws.com/spark-related-packages/shark-0.8.1-bin-cdh4.tgz
fi
;;
0.9.0 | 0.9.1)
if [[ "$HADOOP_MAJOR_VERSION" == "1" ]]; then
wget https://s3.amazonaws.com/spark-related-packages/shark-0.9.1-bin-hadoop1.tgz
else
wget https://s3.amazonaws.com/spark-related-packages/shark-0.9.1-bin-hadoop2.tgz
fi
;;
*)
echo "ERROR: Unknown Shark version"
popd
return
esac

Expand Down
7 changes: 6 additions & 1 deletion templates/root/shark/conf/shark-env.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,12 @@ SPARK_JAVA_OPTS+="-Dspark.kryoserializer.buffer.mb=10 "
#SPARK_JAVA_OPTS+="-verbose:gc -XX:-PrintGCDetails -XX:+PrintGCTimeStamps "
export SPARK_JAVA_OPTS

export HIVE_HOME="/root/hive-0.9.0-bin"
if [ -f "/root/hive-0.9-bin" ]; then
# Point HIVE_HOME to the Hive 0.9 binary manually fetched
# during instance setup. This only applies for Shark v0.8.
export HIVE_HOME="/root/hive-0.9.0-bin"
fi

export HADOOP_HOME=/root/ephemeral-hdfs
export HIVE_CONF_DIR=/root/ephemeral-hdfs/conf

Expand Down