Offer image choices for the JMTE hub

[yuvipanda]

I think overall, we want to reduce the number of images we maintain for our end users. A big part of this to use upstream images directly wherever possible, and allow users to choose. This helps us benefit from upstream fixes as quickly as possible, and reduces the total amount of work done. For example, instead of specifically bumping the version of Julia just for this one
image (pangeo-data/jupyter-earth#166), we could instead do that upstream and benefit
everyone (jupyter/docker-stacks#1917).

Faster startup times is another benefit, as the more specific images are smaller than a big 'all-in-one' image.

There are some features of the all-in-one image that currently don't easily exist upstream:

• Linux desktop
• Nix
• Specific extra packages that maybe installed

We can figure these out over time, but not maintaining the all-in-one
image is a nice goal to shoot for. To this end, the JMTE image is
still the default, but marked as 'deprecated' as I don't want to
continue doing a lot of maintenance on it.

Ref #2201

This is how it looks like.

[yuvipanda]

I will communicate this to the users of the hub.

[yuvipanda]

The following message was communicated to hub users:

Hello hub users! More exciting changes (and features) afoot, thanks to integration with the 2i2c infrastructure! Once #2683 lands, you'll be able to specifically choose from multiple images to launch into your session! To begin with, there are 4 images:

1. The existing, 'all-in-one' big image. No changes here.
2. Upstream Pangeo image for pytorch. (1) is based off an older version of this, so this newer image will bring in newer versions of libraries that may help!
3. Upstream pangeo image for tensorflow. New!
4. Upstream jupyter/docker-stacks' 'datascience notebook' image, which brings in an up to date version of Julia as well as R.

In the long run, I'd like us to stop maintaining (1) and instead allow users to choose from various special purpose images for the following reasons:

1. Faster startup time! The all in one image is about 9G big, while the special purpose images are much smaller. For example, the datascience-notebook image is only 1.8G, compared to 9G for our all-in-one image! This means pod startups should be much faster.
2. Image changes are shared across the whole ecosystem, and hence faster. There's a larger community maintaining these images than just us, so this benefits everyone. The example I have is of bumping version of Julia I did for our image (Bump version of Julia pangeo-data/jupyter-earth#166), vs me doing that upstream (Bump Julia version to 1.9 jupyter/docker-stacks#1917).
3. Allows us to introduce more features without paying a huge complexity penalty. The more combination of things you want in a single image, the more complex it becomes. It's far easier to have a special purpose image that just does one thing. For example, an image focused on showing the Linux Desktop would be far more suited to that than putting that here.

Nothing changes right now, as the old image is still available. However, I ask that y'all try out the other images and let me know what you think! Particularly,

1. Are you using GPUs? Try out the pytorch & tensorflow images and let me know if they are enough, or if further modifications are needed
2. Are you using Julia? Try out the jupyter/datascience-notebook image, and let me know if that is enough!

[yuvipanda]

#2583 is related

[yuvipanda]

A good example of this is the fact that the julia package is installed in the JMTE image for interfacing between python and julia, but it doesn't actually work because that package isn't compatible by default with python installed from conda! I'd rather we solve that upstream than downstream here :)

[GeorgianaElena]

This makes sense to me and agree it's a good pattern to follow! Thank you @yuvipanda

[github-actions]

🎉🎉🎉🎉

Monitor the deployment of the hubs here 👉 https://github.com/2i2c-org/infrastructure/actions/runs/5335693246