Migrate the Pangeo AWS hub to 2i2c infrastructure

[choldgraf]

Summary

The Pangeo Project currently runs a number of JupyterHubs and BinderHubs for their community, but do not have dedicated resources to operate and maintain this infrastructure. We have plans to migrate these hubs to 2i2c infrastructure so that we can provide this service for them.

Now that our AWS deployment infrastructure is in better shape (see #366 for reference), we should consider migrating the Pangeo AWS JupyterHub deployments to be deployed and operated from our pilot-hubs repositories.

Because the Pangeo community is currently without any assistance on their hubs, and they are a key strategic partner of the 2i2c community, I propose that we set this as a high-priority item. Let me know if others believe this is not as important to do quickly :-)

Tasks

• Engage with current AWS Pangeo community to check on their opinions
• Plan to move the AWS hubs (not binders)
• Figure out how we shall do authentication
• Figure out where the infrastructure should run from a credits perspective. Ideally they will continue where they are
• Gain access to current AWS cloud accounts (assumign we will continue running there)
• Create alternate k8s cluster in the same account
• Adapt hub config from https://github.com/pangeo-data/pangeo-cloud-federation/ to this repo.
• Deploy hub from this repo, and adjust until it works as desired. We can re-use current EFS setup to share home directory.
• Move hub domain over to new hub + cluster
• Decomission old cluster

[choldgraf]

cc @rabernat and @yuvipanda with whom I've discussed a bit about this. @yuvipanda could you think up some Tasks that are needed to migrate this hub here?

[yuvipanda]

I updated the issue with what I consider next steps.

[yuvipanda]

Another thing to figure out here is if we commit to using kops based clusters for this too, or use EKS based ones. A round of somewhat formal evaluation criteria might be helpful. I opened #431 to help us make an informed decision.

[yuvipanda]

Reducing priority since there's no active fire right now, and we can pick this up with more intensity once our pangeo hire starts

[choldgraf]

closing as a duplicate of:

• Pangeo JupyterHub on AWS  #1990