Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate hub deployment for all our clusters #507

Closed
4 tasks done
yuvipanda opened this issue Jul 12, 2021 · 8 comments
Closed
4 tasks done

Automate hub deployment for all our clusters #507

yuvipanda opened this issue Jul 12, 2021 · 8 comments
Labels
Enhancement An improvement to something or creating something new.

Comments

@yuvipanda
Copy link
Member

yuvipanda commented Jul 12, 2021

Description

When a kubernetes cluster is present, we can automate the creation of a hub on top of it. We should do this for hubs on all of the clusters that we run.

Benefit

Keeping all the hubs in sync makes sure we don't end up in a situation where some hubs need a lot of work for hub upgrades. It also reduces the amount of manual steps that we need to take in order to deploy a given hub.

Implementation

We should ensure that our post-deploy tests don't leave running servers behind costing money.

Tasks to complete

Google Cloud Hubs

  • Deploy basic GCP hubs

Specific Hubs

AWS Hubs

Azure hubs

@GeorgianaElena GeorgianaElena added the Enhancement An improvement to something or creating something new. label Jul 14, 2021
yuvipanda added a commit to yuvipanda/pilot-hubs that referenced this issue Jul 30, 2021
Ref 2i2c-org#507

I'm worried that incomplete tests might leave notebook
nodes running, but the culler should take care of those? We
must verify this.

The creds for the carbonplan cluster are from EKS,
and I'm not sure how long they are valid for either.
yuvipanda added a commit to yuvipanda/pilot-hubs that referenced this issue Aug 5, 2021
Ref 2i2c-org#507

I'm worried that incomplete tests might leave notebook
nodes running, but the culler should take care of those? We
must verify this.

The creds for the carbonplan cluster are from EKS,
and I'm not sure how long they are valid for either.
@yuvipanda
Copy link
Member Author

With #582 everything except AWS hubs are autodeployed.

@damianavila
Copy link
Contributor

With #582 everything except AWS hubs are autodeployed.

AFAIK #582 is not automating any Azure deployment, right?

@yuvipanda
Copy link
Member Author

@damianavila yeah, but we don't have any Azure deployments merged yet - #287 is unmerged.

@damianavila
Copy link
Contributor

damianavila commented Aug 11, 2021

#287 was actually merged a long time ago, I guess you are referring to #513, right?
In any case, I get what you are saying 😉 .

@sgibson91
Copy link
Member

RE AWS hubs: following code needs to be ported into deployer for authentication https://github.com/yuvipanda/hubploy/blob/dfc00254abb30d8887230472e83b405ff5872933/hubploy/auth.py#L107

@choldgraf
Copy link
Member

Thanks @sgibson91 - I've separated out the top comment into a section for each major cloud provider in case we need to track more provider-specific tasks.

@choldgraf
Copy link
Member

OK I believe that we've got an issue for each major cloud provider listed in the top comment, and each issue is resolved! So perhaps we can close this one?

@damianavila
Copy link
Contributor

Yep, I think we can close this one! (we can always re-open if someone disagrees 😉 ).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement An improvement to something or creating something new.
Projects
None yet
Development

No branches or pull requests

5 participants