Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs on setting up a new project #340

Merged
merged 4 commits into from
Apr 9, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/howto/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,5 @@ customize/configure-login-page.md
## Hub operation
```{toctree}
operate/index.md
operate/setup-new-project.md
```
72 changes: 72 additions & 0 deletions docs/howto/operate/setup-new-project.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Setup a new project for a client organization

Each client organization of 2i2c gets their own [GCP Project](https://cloud.google.com/resource-manager/docs/creating-managing-projects).
This allows us to:

1. Maintain billing easily, since projects are the easiest unit of
billing.
2. Give members of client organizations (such as IT teams) full access to
the cloud project, without granting access to other client organizations'
resources
3. Allow client organizations to take over running of the infrastructure
themselves, with minimal disruption.

Client organizations can give us access to a billing account, or
just to a project. This document describes what 2i2c engineers should
do once client organizations give us access.

## Client organization provides billing account

[Our pilot docs](https://pilot.2i2c.org/en/latest/admin/howto/create-billing-account.html#full-billing-account-access)
has information on how client organizations can provision a billing
account and give us access.

Once the client organization provides us with a billing account,
we provision a project for them inside the 2i2c.org GCP organization.
ALl this work happens in the [org-ops](https://github.com/2i2c-org/org-ops)
damianavila marked this conversation as resolved.
Show resolved Hide resolved
repository.

1. Edit `projects.tfvars`, adding an entry to `fully_managed_projects`.
The key should be the name of the project we want to create, and the
value should be the billing account ID. Add a comment referencing more
information about the client organization, ideally pointing to a GitHub
issue.

2. Run terraform (with instructions in the org-ops repo README). This will give you a
yuvipanda marked this conversation as resolved.
Show resolved Hide resolved
detaile plan on what exactly terraform will do. **Scrutinize this carefully** -
damianavila marked this conversation as resolved.
Show resolved Hide resolved
only a new project should be created, nothing else should change. Particularly,
there should be no **delete** actions.

3. Validate that the project has been created, and you have access to it with
your user account.

4. Commit the change, make a PR to the repo with it, and merge it. This is
preferable to pushing to the repo directly. You can self-merge your PR.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure you want to allow self-merge when the PR has the potential to destroy things? (the second point you wrote above)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The danger is in the terraform apply step, which is why it doesn't run automatically. Merging doesn't actually trigger anything.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, thanks for the clarification, so how do we make sure others can see/review that "state" before being applied?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great question! What kinda workflow do you think would work here? I don't want to run terraform apply in CI, so not sure how to approach review.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not worked with the terraform stuff yet (although I have some experience with AWS Cloudformation), but looking quickly at the terraform docs, it seems you have a terraform plan command. That should surface any problem (or deletion you are trying to prevent), right? Then, you could run that command by the CI and detect any "bad" message. And others can review that plan.
Additionally, it seems you can run that command with a -out parameter and get some file that you can eventually run manually later with terraform apply command (modulo you still do not want to run apply within the CI which could be an event happening on the merge event after a successful CI run and approval from others, but I totally get it is frightening to give the CI such destructive power): https://www.terraform.io/docs/cli/commands/plan.html
Finally, I may be saying stupids things (as I did in the other PR) because I do not know yet the deep stuff, so be patient with me 😉

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little bit confused now as step 3 is "Check the project has been created and you have access to it" and step 4 is "open the PR" which implies terraform apply was run before opening the PR?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, my brain just caught up with me! Yes, I think we should approve/merge the PR before running terraform apply - always good to have a second pair of eyes on code/configs before actioning anything I think.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Someone else approves PR

This is where terrraform plan (with sensitive on to prevent leaking info/secret) plays a role, IMHO.
Otherwise, the person reviewing the PR does not have any info about how the plan looks like.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@damianavila @sgibson91 I updated the instructions based on the conversations here. LMK what you think? I wanna balance code review with appropriate load on our small team.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good!


## Without billing account access

Sometimes, we won't have access to the billing account - the project would
be pre-created for us by the client organization. Our pilot docs have
information on [how client organizations can give us access](https://pilot.2i2c.org/en/latest/admin/howto/create-billing-account.html#project-level-access)

If these projects can *not* be moved into the 2i2c.org GCP organization,
we [can not](https://cloud.google.com/resource-manager/reference/rest/v1/projects/setIamPolicy)
automatically add 2i2c engineers as owners on the project. They will
need to be added manually via the [GCP Console web interface](https://console.cloud.google.com).
They will get an invite in their mail, which they must manually accept.

1. Find current list of 2i2c engineers who should have access to this GCP project,
maintained in `variables.tf` under `project_owners` in the [org-ops](https://github.com/2i2c-org/org-ops),
repository.

2. Use the GCP console to [invite all these users](https://cloud.google.com/iam/docs/granting-changing-revoking-access)
to the project, giving them `Basic -> Owner` permissions. Make sure you
are doing this in the correct GCP project!

3. Ping all those 2i2c engineers to make sure they accept the invite.

In the future, we should support:

1. Moving projects into 2i2c.org GCP organization when possible.
2. Decommission access to GCP projects when 2i2c engineers leave.
3. Make sure new 2i2c engineers are added to all projects we have access to.