Skip to content

Using the Docker Container

Jump to bottom
Brian Burns edited this page Oct 28, 2016 · 8 revisions

The docker container provides you with the following:

  • Jupyter Gateway
  • Apache Toree
  • Apache Spark
  • EclairJS Server JAR
  • EclairJS Example data

EclairJS can be run in a docker container using Docker 1.8+. You can download a Docker image from DockerHub (the docker image on docker hub is eclairjs/minimal-gateway) by running:

docker run -p 8888:8888 eclairjs/minimal-gateway

This will download the pre-built Docker image and start the kernel gateway server. Please note that since this is a pre-built image it may not include everything that is in development on github.

To run the Word Count example against a local docker image:

export JUPYTER_HOST=??.??.??.?? (your docker ip)
export JUPYTER_PORT=8888
git clone https://github.com/EclairJS/eclairjs.git
cd eclairjs/examples
npm install
./run.sh --docker wordcount/wordcount.js

If you wish run something which accesses data other than the eclairjs example data, you need pass the appropriate docker parameters to allow the docker container access to your data.

You can also access the Notebook GUI by:

  • Pointing a browser to {docker ip}:8888
  • Create a new notebook by selected New -> Spark 2.0.0 (EclairJS)

Deploying on IBM Bluemix

You can use the Docker image to deploy Spark/EclairJS to IBM Bluemix. This will provide a single node version of Spark 2.0.0 with the necessary EclairJS addons.

First go to Dashboard and then choose Start Containers. Click on Add your own and you will be asked to set a registry namespace. If you haven't already, download the CLI tools as explained here.

Once setup we login and authenticate via:

cf login

and

cf ic login

Now we can go ahead and deploy the EclairJS Docker image to your private Bluemix namespace. Be sure to substitute your registry name into the following commands:

cf ic cpi eclairjs/minimal-gateway registry.ng.bluemix.net/<your_registry_name_here>/minimal_gateway
cf ic run --name eclairjs  -p 8888:8888 -m 128 registry.ng.bluemix.net/<your_registry_name_here>/minimal_gateway

This will create a Docker container named eclairjs. It may take a while to complete, you can go to the Bluemix Dashboard and click on the eclairjs container to view its status. Once it is in the running state you can go ahead and equest a public IP address:

cf ic ip request

This will result in a output of The IP address "<your.ip.address>" was obtained. and we bind that IP to our Docker image with:

cf ic ip bind <your.ip.address> eclairjs

Now we can run a EclairJS example:

export JUPYTER_HOST=<your.ip.address>
git clone [email protected]:EclairJS/eclairjs.git
cd eclairjs/examples
npm install
node --harmony examples/wordcount/wordcount.js
Clone this wiki locally