Skip to content

Commit

Permalink
SPARK-938 - Openstack Swift object storage support
Browse files Browse the repository at this point in the history
This is initial documentation describing how to integrate Spark with
Swift. This commit contains documentation for stand alone cluster.
Next patches will contain details how to integrate Swift in other
deployment of Spark.
  • Loading branch information
gilv committed Jun 8, 2014
1 parent b6c37ef commit ce483d7
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/openstack-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ Create core-sites.xml and place it under /spark/conf directory. Configure core-s
<value>true</value>
</property>

We left with `fs.swift.service.<PROVIDER>.tenant`, `fs.swift.service.<PROVIDER>.username`, `fs.swift.service.<PROVIDER>.password`. The best way is to provide them to SparkContext in run time, which seems to be impossible yet.
We left with `fs.swift.service.<PROVIDER>.tenant`, `fs.swift.service.<PROVIDER>.username`, `fs.swift.service.<PROVIDER>.password`. The best way to provide those parameters to SparkContext in run time, which seems to be impossible yet.
Another approach is to change Hadoop Swift FS driver to provide them via system environment variables. For now we provide them via core-sites.xml

<property>
Expand Down

0 comments on commit ce483d7

Please sign in to comment.