Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update runbooks #6556

Merged
merged 2 commits into from
Dec 6, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions runbooks/source/changes-in-cloudplatform.html.md.erb
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
---
title: Change Process in Cloud Platform
weight: 51
last_reviewed_on: 2024-06-06
last_reviewed_on: 2024-12-06
review_in: 6 months
---

# <%= current_page.data.title %>

The Cloud Platform team implements and encourage an Infrastructure as Code (IaC) methodology to manage the platform. It means every single change is done through code (if you spot something different raise it with us please).
The Cloud Platform team implements and encourages an Infrastructure as Code (IaC) methodology to manage the platform. It means every single change is done through code (if you spot something different, raise it with us please).

[cloud-platform-infrastructure repository](https://github.com/ministryofjustice/cloud-platform-infrastructure) is our main repository where infrastructure gets created and all components fit together. From this repository everything gets linked.

Mostly all components are Terraform [modules](https://developer.hashicorp.com/terraform/language/modules), and they are managed in their own git repository; you can list all modules [here](https://github.com/ministryofjustice?q=cloud-platform-terraform&type=&language=).
Mostly all components are Terraform [modules](https://developer.hashicorp.com/terraform/language/modules), and they are managed in their own git repositories; you can list all modules [here](https://github.com/ministryofjustice?q=cloud-platform-terraform&type=&language=).

## Making Changes to cloud-platform-infrastructure

1. Create a PR which includes the change you want to make. Make sure the [plan pipeline](https://concourse.cloud-platform.service.justice.gov.uk/teams/main/pipelines/infrastructure-live/jobs/terraform-plan-infra-live/) is happy with the change. It is good practice to link the PR to the issue that describes the change.
2. Chase the team to get your PR approved, and if it involves downtime ensure you have communications in place.
2. Chase the team to get your PR approved, and if it involves downtime, ensure you have communications in place.
3. For certain changes, pausing the infrastructure apply pipelines and running them individually is favourable. If you are pausing bootstrap and infrastructure pipelines, be sure to communicate this in the team slack channel.
4. Merge and wait until the [apply pipeline](https://concourse.cloud-platform.service.justice.gov.uk/teams/main/pipelines/infrastructure-live/jobs/terraform-apply-infra-live/) applies your change.

Expand Down Expand Up @@ -48,10 +48,10 @@ The Cloud Platform source of truth for team's environments (Kubernetes namespace
1. Create a PR and verify the planned changes in the [plan pipeline](https://concourse.cloud-platform.service.justice.gov.uk/teams/main/pipelines/environments-live). If you are happy with them, ask a team member for approval.
2. Merge PR

Within the environments repository, we have an end to end process fully automated with pipelines. Once the change is merged, it is going to be automatically applied by the apply pipeline ([apply-live](https://concourse.cloud-platform.service.justice.gov.uk/teams/main/pipelines/environments-live/jobs/apply-live) and [apply-namespace-changes-live](https://concourse.cloud-platform.service.justice.gov.uk/teams/main/pipelines/environments-live/jobs/apply-namespace-changes-live/builds/1) and if the change is destroying a namespace it will be detected by [detect-deleted-namespace](https://concourse.cloud-platform.service.justice.gov.uk/teams/main/pipelines/environments-live/jobs/detect-deleted-namespaces/builds/1) pipeline.
Within the environments repository, we have an end to end process fully automated with pipelines. Once the change is merged, it is going to be automatically applied by the apply pipeline ([apply-live](https://concourse.cloud-platform.service.justice.gov.uk/teams/main/pipelines/environments-live/jobs/apply-live) and [apply-namespace-changes-live](https://concourse.cloud-platform.service.justice.gov.uk/teams/main/pipelines/environments-live/jobs/apply-namespace-changes-live/builds/1) and if the change is destroying a namespace, it will be detected by [detect-deleted-namespace](https://concourse.cloud-platform.service.justice.gov.uk/teams/main/pipelines/environments-live/jobs/detect-deleted-namespaces/builds/1) pipeline.

## Communications

If any changes involve downtime we must inform teams about it via Slack (#cloud-platform-update). It is important to include **which services** are going to be down and **for how long**.
If any changes involve downtime, we must inform teams about it via Slack (#cloud-platform-update). It is important to include **which services** are going to be down and **for how long**.

Sometimes teams misinterpret downtimes of Cloud Platform services to involve downtime of **their services**, so it is very important to be precise about the change and it's potential impact.
Sometimes teams misinterpret downtimes of Cloud Platform services to involve downtime of **their services**, so it is very important to be precise about the change and its potential impact.
Loading