This repo includes JupyterLab extensions that have been built for the MAAP project (https://che-k8s.maap.xyz/)
In order to use each of these extensions they must be installed and enabled in your environment. Instructions for each extension can be found in the respective folder. Make sure each extension's dependencies are installed first.
These extensions have been developed for Jupyter 4.4.0
and Jupyter Lab 2.1.4
.
All the MAAP Jupyter extensions share some common dependencies.
conda install conda=4.7.12 jupyterlab=2.1.4 nodejs=10.13.0 gitpython=3.0.2
jupyter labextension install jupyterlab_toastify@2.3.0 --no-build
npm i jupyterlab_toastify@2.3.0
pip install plotly==4.0.0
jupyter labextension install @jupyterlab/plotly-extension@1.0.0
jupyter labextension install @jupyter-widgets/jupyterlab-manager@1.0
There is no strict order to install the extensions in, but since some extensions depend on others, those dependencies should be installed first. For each of the following extensions (in recommended order), the sub-bullet points will indicate if it requires installing another extension listed above it first.
Note: Extensions marked as Che-only don't make sense to install locally, since they only work in the context of the Che UI and/or containerized contexts, and require use of the Che API.
hide_side_panel
(Che-only)ipycmc
pull_projects
(Che-only)show_ssh_info
- ssh features (Che-only)
- s3 features work locally
- requires
s3fs-fuse
util and AWS keys/role,boto3
Python library installed (see extension README)
- requires
edsc_extension
- requires
maap-py
library (see extension README)
- requires
maapsec
submit_jobs
- requires
maapsec
- requires
dps_magic
- requires
maapsec
,submit_jobs
,show_ssh_info
- requires
dps_info
- requires
maapsec
,submit_jobs
- requires
insert_defaults_to_notebook
- requires
maap-py
library (see extension README)
- requires
user_meta_form
To build additional extensions for the project, it is recommended to start from a cookie-cutter or off a previously built extension.
Some Jupyter Extensions/Resources we have found helpful:
- xkcd tutorial Great place to start for extending JupyterLab
- pizzabutton Implements a basic server extension and connects it with UI
- cookie-cutter-ts Base to build UI extensions off of
- jupyterlab_autoversion
- jupyterlab-logout
- jupyterlab_iframe
- jupyterlab_toastify
- jupyter-notify
- jupyterlab A lot of figuring out how to add things where has happened through looking at the source code of jupyter
- Jupyterlab-html
- Jupyterlab-sandbox
In JupyterLab's update to the stable 1.0 version, they have also updated and added lots of documentation on extension development. I recommend taking a look at this.
Our development process involves building and running an extension locally in jupyterlab using a conda env before
installing it on the che server. To enable an extension in Che, it must be included in the base docker image/stack that a
Che workspace is launched with. The dockerfile that extensions are included in is the Dockerfile
and the highest level
in this repo. At the point of adding your extension into the Docker image, some minor changes may have to be made
(mainly path issues). This will be explained in the bullets below.
An instance of this repository lives on the Che server under ~/che/dockerfiles/maap-jupyter-ide
. Once an extension has been tested locally, rebuild the docker
image with your new extensions.
- Add your install to the Dockerfile. For example:
# jlab pull projects into /projects directory COPY pull_projects /pull_projects RUN cd /pull_projects && npm run build RUN cd /pull_projects && jupyter labextension link . RUN cd /pull_projects && pip install -e . RUN cd /pull_projects && jupyter serverextension enable --py pull_projects --sys-prefix
- If your extension includes a server extension you also need to modify
entrypoint.sh
. This is because jupyter server extensions function off of having a standard base url, but in the context of che the url is not what jupyter thinks it is.- Here is some magic that fixes it (add this line and replace with the path to where
your
load_jupyter_server_extension
function is)perl -pi -e "s|web_app.settings\['base_url'\]|'/'|g" /show_ssh_info/show_ssh_info/__init__.py
- Here is some magic that fixes it (add this line and replace with the path to where
your
- Then rebuild the docker image.
microk8s.docker build -t localhost:32000/che-jupyter-lab-ide .
- Push!
microk8s.docker push localhost:32000/che-jupyter-lab-ide
- Now when you build a new workspace with the
localhost:32000/che-jupyter-lab-ide
image it will automatically fetch the new image. (found in the stack'sRecipe
orRaw Configuration
)- you can also specify the image tag to use in your build on the stack if you want to use a previous build
- Any change pushed to
microk8s.docker push localhost:32000/che-jupyter-lab-ide
will affect the default stacks on all user accounts. If you are testing something, you can create your own image and your own stack to play around with.
To make your custom docker image available to users in Che, you need to make a new stack that creates workspaces using your image and make it available to users. Below is an example stack configuration using our locally built dockerized juptyer image with MAAP extensions installed.
Make sure to replace the image name in workspaceConfig.environments.default.recipe.image
with the location of your image.
In order for SSH-ing into the workspace to be possible, the org.eclipse.che.exec
and org.eclipse.che.ssh
installers must be enabled under workspaceConfig.environments.default.machines.ws/jupyter.installers
.
{
"scope": "general",
"description": "Use this one. Stable jupyter. No extra packages",
"creator": "b07e3a58-ed50-4a6e-be17-fcf49ff8b242",
"tags": [
"MAAP",
"JUPYTER",
"STABLE"
],
"workspaceConfig": {
"defaultEnv": "default",
"environments": {
"default": {
"recipe": {
"contentType": "text/x-yaml",
"type": "kubernetes",
"content": "kind: List\nitems:\n - \n apiVersion: v1\n kind: Pod\n metadata:\n name: ws\n labels:\n name: ws\n spec:\n containers:\n - \n name: jupyter\n image: 'localhost:32000/stable-ide:latest'\n resources:\n limits:\n memory: 1024Mi\n securityContext:\n privileged: true\n - \n apiVersion: v1\n kind: Service\n metadata:\n name: ws\n spec:\n type: NodePort\n ports:\n - \n port: 22\n selector:\n name: ws\n \n "
},
"machines": {
"ws/jupyter": {
"env": {
"MACHINE_NAME": "WS_JUPYTER"
},
"servers": {
"jupyter": {
"path": "/",
"attributes": {
"cookiesAuthEnabled": "true",
"type": "ide",
"secure": "true"
},
"protocol": "http",
"port": "3100"
}
},
"installers": [
"org.eclipse.che.exec",
"org.eclipse.che.ssh"
],
"volumes": {
"projects": {
"path": "/projects"
}
},
"attributes": {}
}
}
}
},
"projects": [],
"commands": [],
"name": "default",
"attributes": {},
"links": []
},
"components": [],
"name": "maap-jupyter-ide",
"id": "stacktdo2q0ixhv7cge00"
}
- Cluster Privileges
- in
/var/snap/microk8s/current/args/kubelet
and/var/snap/microk8s/current/args/kube-apiserver
, append--allow-privileged
- restart both services:
sudo systemctl restart snap.microk8s.daemon-apiserver
sudo systemctl restart snap.microk8s.daemon-kubelet
- Che Permissions
- in
che/dockerfiles/init/manifest/che.env
, setCHE_DOCKER_PRIVILEGED=true
under the Privileged Mode section - restart Che
To create a stack, you write a raw configuration with all the che and docker settings your workspace will require, including installers, volumes, docker run tags, docker images, etc. See the example above.
To share a stack, you will need to be the owner (creator) of the stack.
Go to the homepage of where your Che instance is hosted and add /swagger
to the end of the url for an interface with Che's API. Under the permissions
section, make a POST request with the users you want to share with and the id of your stack (shows up at the bottom of the configuration after creation).
POST body:
{
"userId": "*",
"domainId": "stack",
"instanceId": "${STACK_ID}",
"actions": [
"read",
"search"
]
}
reference: https://www.eclipse.org/che/docs/che-6/stacks.html#sharing-stacks-and-system-stacks .