Improvements to developer onboarding documentation

[choldgraf]

Summary

In a recent conversation with @damianavila, he noted that it was a bit confusing to learn a few important things about the pilot hubs:

• The overall architecture and design of the pilot hub infrastructure
• Information about cloud accounts and projects that exist, and how these relate to our deployment infrastructure
• Getting started information to quickly get up-to-speed with development
• Information about hub infrastructure that is not in pilot-hubs and how to access it.

Acceptance criteria

• We've got updated Pilot Hubs documentation that @damianavila thinks would improve upon these issues. (I think Damian's perspective is particularly useful as he is still relatively new to this repository).

Tasks to complete

• Agree upon a subset of additions that we should make to the documentation
• Make it clear where the documentation is for "hub administrators" vs. where it is for 2i2c engineers who might work on the base infrastructure
• ...

[choldgraf]

@damianavila perhaps you could share a few other items (or just update the comment above) with some specific ways that the pilot hubs docs could be improved?

[yuvipanda]

I look forward to this, @damianavila! The current architecture is a bit slapdash, and this will give us an opportunity to keep this more focused.

[yuvipanda]

I've started #352 to refactor some of the things in this repo. I'm hoping that a combination of code review and refactoring will make this repository easier to understand :)

[damianavila]

@damianavila perhaps you could share a few other items (or just update the comment above) with some specific ways that the pilot hubs docs could be improved?

Sure... let me paste here some of my notes about the pilot and pilot-hubs docs.

pilot.2i2c.org (hub admin consumers)

---

https://pilot.2i2c.org/en/latest/about/infrastructure.html

Each 2i2c Hub has two main interactive interfaces: Jupyter interfaces (Notebook and Lab), and RStudio. Each of them is accessible from your session via /tree, /lab, and /rstudio endpoints in your URL.

Should mention VSCode if there is an icon in the image...

To learn about how the pilot-hubs/ repository works, we recommend checking out the pilot-hubs documentation.

Missing link

---

https://pilot.2i2c.org/en/latest/about/projects.html

Pangeo pilot should be mentioned first as in the previous section (https://pilot.2i2c.org/en/latest/about/infrastructure.html)

---

https://pilot.2i2c.org/en/latest/admin/configuration/login.html#

Currently, there are only two supported methods for authorizing regular users

I was expecting also to add users using the helm config file...
If the authz users are manually added, how are you backing up that info?

---

https://pilot.2i2c.org/en/latest/admin/configuration/culling.html

For the server culling, I know there is some way in binder to detect inactivity, should not be using that to prevent killing a long computation?

There is currently no maximum time limit for a user’s notebook.

This is confusing...

---

https://pilot.2i2c.org/en/latest/admin/configuration/default-interface.html

The legacy notebook should have a picture as well...

Do not we supppot VSCode?
Theia?
Why not? License, I guess...

---

https://pilot.2i2c.org/en/latest/admin/howto/support.html

Who is receiving the emails sent to [email protected]?

---

https://pilot.2i2c.org/en/latest/admin/howto/configurator.html

Warning

Is there any validation before submitting the change?

Once you’re at the control panel, you can access the configurator under Services in the top navigation bar. It will ask you to authenticate to the configurator so it knows you are an admin.

So you need to auth twice?

You must click the submit button at the bottom of the form for your changes to take effect!

A green message should help here...

If you are using a custom docker image, you should make sure these interfaces are available in your docker image!

How to prevent that failure or fail graciously...

There’s a ‘interface selector’ in the hub home page, but unfortunately that does not reflect this option. So if your user selects an interface there, they will be put in that interface but just for the first time after they log in.

How do we start with the selected interface?

---

https://pilot.2i2c.org/en/latest/admin/howto/environment.html

Many popular data science user interfaces installed

VScode, Theia? Dupe from previous comments...

Ask for changes to the base image¶

The default image should be as small as possible

If the new image you use breaks user server starts, or is broken in some other way, you can revert back to the old image by specifying that.

that? Really?

While tempting, do not use !pip install --user to install packages.

With conda, you can "save" your temp environment as well... not sure about conflicts

---

https://pilot.2i2c.org/en/latest/admin/howto/nbgitpuller.html

automatically merged sound nice, wondering about issue with the process and if there is any thought about a conflict resolution UI in the future...

---

https://pilot.2i2c.org/en/latest/admin/howto/create-content.html

This section probably needs more love (development), right?

---

https://pilot.2i2c.org/en/latest/admin/howto/manage-users.html

Persistence of manually added users? Dupe question IIRC

---

https://pilot.2i2c.org/en/latest/admin/howto/control-user-server.html

Switch the warnings. Btw, the RTC does not apply here because you are actually impersonating the user...

---

https://pilot.2i2c.org/en/latest/admin/howto/share-datasets.html

Wondering how the shared-readwrite is implemented (there are some docs ahead)

---

https://pilot.2i2c.org/en/latest/admin/howto/replicate.html

Your hub-specific configuratior
Your hub template configuration

Any export tools to get the config you may use in your own deployment?

---

https://pilot.2i2c.org/en/latest/admin/howto/create-billing-account.html

Fix AWS link

Enter the following user emails under ‘New members’: [email protected], [email protected]. These are two of the folks at 2i2c who will help manage your project.

We should use some generic user... or can others be invited by the two of you? (in any case, it is a bottleneck).

---

https://pilot.2i2c.org/en/latest/user/download-data.html

Interesting tool, jupyter-tree-download (is there a JLab counterpart?)

---

https://pilot.2i2c.org/en/latest/user/interface.html

VSCode?

---

I will post my pilot-hub docs notes in the next message on this thread.

[damianavila]

pilot-hubs (2i2c eng docs)

---

https://pilot-hubs.2i2c.org/en/latest/

hubs.yaml should be always links to the repo, IMHO.

You can add a new hub by adding an entry there, and pushing that to github.

Is the CI deploying the new hub somehow??? (I noticed that is the case after looking at the repo...)

---

https://pilot-hubs.2i2c.org/en/latest/structure.html

People visit our website to learn about us

Our site is not linking the Hub users/admin site?

---

https://pilot-hubs.2i2c.org/en/latest/howto/configure/hub-config.html

Each JupyterHub can use its own configuration via the hubs.yaml file.

No link

This should be a valid Zero to JupyterHub configuration.

Any validation?

---

https://pilot-hubs.2i2c.org/en/latest/howto/configure/auth-management.html#

Double "Manage authentication" in ToC, the same for other sections...

---

https://pilot-hubs.2i2c.org/en/latest/howto/configure/add-hub-domains.html

For this to happen we need to add the new domain as an ingress domain for this hub using the hubs.yaml configuration file.

Ingress domain, interested to know more about this one...

---

https://pilot-hubs.2i2c.org/en/latest/howto/configure/update-env.html

Currently there is no automated image building - so you will have to manually build & push it after making a change.

This should be automated!

gcloud tool

CLI? Link missing...

---

https://pilot-hubs.2i2c.org/en/latest/howto/customize/custom-image.html

Only on merge will the hub get the new image.

How does the automatic deployment work? Looked at the repo, I think I have already got it...

---

https://pilot-hubs.2i2c.org/en/latest/howto/customize/docs-service.html

Why jupyterbook is not listed here?

I should try Nikola with this one!

---

https://pilot-hubs.2i2c.org/en/latest/howto/customize/configure-login-page.html

The log-in pages are built with the base template at this repository.

Interesting tool!

---

https://pilot-hubs.2i2c.org/en/latest/howto/operate/index.html#team-process

Endless loop with Add a new hub link ;)

---

https://pilot-hubs.2i2c.org/en/latest/howto/operate/cmd-access.html

Get access with local machine

---

https://pilot-hubs.2i2c.org/en/latest/howto/operate/delete-hub.html

Backup the hub database

How? To where?

Delete our Authentication entry. If the hub hasn’t been recreated anywhere, remove the entry from auth0.com

I guess I need access to auth0.com?

---

https://pilot-hubs.2i2c.org/en/latest/howto/operate/manual-deploy.html

deploy.py deploy

Mmm... name duplication... confusing, IMHO.

Deploys all the hubs in the cluster passed as argument (the hubs listed in hubs.yaml file unde )

Typo

you first need to make sure you are authenticated in all the right places and have the right credentials exported as environmental variables.

How do you know if this is happening?

---

https://pilot-hubs.2i2c.org/en/latest/howto/operate/test-hub.html

If the deploy.py script is used to deploy all pilot hubs at once, then a failed hub health check will halt all further deployments!

Mmm... that worrisome, how do you handle a problematic automated deployment?

---

https://pilot-hubs.2i2c.org/en/latest/howto/operate/grafana.html

For example, the pilot hubs are accessible at grafana.pilot.2i2c.cloud.

Ask user and pass

---

https://pilot-hubs.2i2c.org/en/latest/howto/operate/setup-new-project.html

How does this work with already existing Hubs/orgs?

---

https://pilot-hubs.2i2c.org/en/latest/topic/cluster-config.html

Are all the clusters deployments inside the same Kube cluster? I think that is the case, but I need to go deeper to check it out...

---

https://pilot-hubs.2i2c.org/en/latest/topic/defaults.html

hub/values.yaml is https://github.com/2i2c-org/pilot-hubs/blob/master/hub-templates/base-hub/values.yaml

---

https://pilot-hubs.2i2c.org/en/latest/topic/hub-templates.html

Desn’t let servers run for more than 8h

Typo

Enables outgoing SSH

More info...

---

https://pilot-hubs.2i2c.org/en/latest/topic/storage-layer.html

The NFS share creator job will be created pre-deploy, run, and cleaned up before deployment proceeds. Ideally, this would only happen once per hub setup - but we don’t have a clear way to do that yet.

Where is this happening in the codebase?

This feature comes from the custom KubeSpawner that the pilot hubs use, that allows providing extra configuration for admin users only.

That link is probably wrong...

Is there any backup on the disk

[damianavila]

@choldgraf and @yuvipanda those are notes I have taken last week almost without checking the repos contain. Now that I have started to look at the repos, some things are clearer than before. I still think this is probably good feedback to fix some bugs and improve some sections in the docs.

You may also find some notes not as useful as they should be (because I wrote them for me first)... feel free to disregard those 😉

@yuvipanda, I will look into your PR, and try to provide useful feedback!!

[yuvipanda]

This is very helpful, @damianavila! I'm slowly responding to these one by one, either here or by fixing things :)

automatically merged sound nice, wondering about issue with the process and if there is any thought about a conflict resolution UI in the future...

See https://jupyterhub.github.io/nbgitpuller/topic/automatic-merging.html for how this is handled.

[yuvipanda]

@damianavila I think I've addressed your questions around manual deployment with this PR: #360

[yuvipanda]

@damianavila as part of clearing up the 'hub templates' documentation, I've just removed the 'ephemeral-hubs' hub template in #361.

[damianavila]

This is very helpful, @damianavila! I'm slowly responding to these one by one, either here or by fixing things :)

Thank you!! I will check your PRs ASAP 😉

[choldgraf]

@damianavila what do you think that we should do with this one? I feel like we should either turn this into a short and actionable list, or just close it because it's quite broad and will be hard to act upon. I think that we've made a lot of improvements to the documentation since it was opened, but am not sure what exactly has been addressed here

[damianavila]

I think that we've made a lot of improvements to the documentation since it was opened, but am not sure what exactly has been addressed here

I agree, most likely the stuff here was already addressed or it is no longer relevant, so closing here.