Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[deploy] Consolidate deployment documentation #984

Merged
merged 4 commits into from
Jan 9, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions assets/deploy_infrastructure_modules.puml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
@startuml

frame "Infrastructure" #lightblue {
package "terraform-*-dss" as TD #lightblue {
[terraform-*-kubernetes] as TK #lightblue
[terraform-common-dss] as TCD #lightblue
}
}

cloud "Cloud Provider" as C {
package "Kubernetes Cluster" as K #lightblue {
component "Kubernetes Node" as KN #lightblue {
}
component "Kubernetes Node" as KN2 #lightblue {
}
}
}

TD -down-> K #blue: 1. Prepare the Infrastructure
TK -> TCD: harmonized\noutputs
TCD -right-> ( ): 2. Generate Services configuration

@enduml
37 changes: 37 additions & 0 deletions assets/deploy_overview.puml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
@startuml

frame "Infrastructure" #lightblue {
package "terraform-*-dss" as TD #lightblue {
[terraform-*-kubernetes] as TK #lightblue
[terraform-common-dss] as TCD #lightblue
}
}

frame "Services" #lightgreen {
[Helm Charts] as H #lightgreen
}

frame "Operations" as O #lightyellow {
[Scripts] as S #lightyellow
}

cloud "Cloud Provider" as C {
package "Kubernetes Cluster" as K #lightblue {
component "Kubernetes Node" as KN #lightblue {
() pods as P #lightgreen
}
component "Kubernetes Node" as KN2 #lightblue {
() pods as P2 #lightgreen
}
}
}

TD -> K #blue: 1. Prepare the Infrastructure
TK -> TCD: harmonized\noutputs
TCD -right-> H: 2. Generate Services configuration
H -down-> P #green: 3. Deploy the Services
H -down-> P2 #green
S -up-> P #orange
S -> P2 #orange: 4. Operate the Services

@enduml
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/generated/deploy_overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
32 changes: 25 additions & 7 deletions deploy/README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,31 @@
# DSS Deployment

This folder contains the increments toward the new deployment approach as described in [#874](https://github.com/interuss/dss/issues/874).
> This folder contains the increments toward the new deployment approach as described in #874.

The infrastructure folder contains the terraform modules to deploy the DSS to kubernetes clusters of various cloud providers:
An operational DSS requires two different services: the DSS core-service API and the Cockroach database.
This folder contains the tools to prepare the infrastructure in multiple cloud providers, deploy the services and operate it.

- Amazon Web Services: [terraform-aws-dss](./infrastructure/modules/terraform-aws-dss/README.md)
- Google Cloud Engine: [terraform-google-dss](./infrastructure/modules/terraform-google-dss/README.md)
The deployment tools are conceptually broken down in three main categories:

The service folder contains the scripts required to deploy the DSS to a Kubernetes cluster:
- **Infrastructure**: It is responsible to prepare infrastructure on various cloud providers to accept deployment of Services below. It includes the Kubernetes cluster creation, cluster nodes, load balancer and associated fixed IPs, etc. This stage is cloud provider specific.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The infrastructure README isn't directly navigable currently;

Suggested change
- **Infrastructure**: It is responsible to prepare infrastructure on various cloud providers to accept deployment of Services below. It includes the Kubernetes cluster creation, cluster nodes, load balancer and associated fixed IPs, etc. This stage is cloud provider specific.
- [**Infrastructure**](./infrastructure): It is responsible to prepare infrastructure on various cloud providers to accept deployment of Services below. It includes the Kubernetes cluster creation, cluster nodes, load balancer and associated fixed IPs, etc. This stage is cloud provider specific.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the bullet points to sections and added navigation to all headers.


Terraform modules are provided for:
- [Amazon Web Services (EKS)](infrastructure/modules/terraform-aws-dss)
- [Google (GKE)](infrastructure/modules/terraform-google-dss)

- Helm Charts: [services/helm-charts](./services/helm-charts)
- Tanka: [../build/deploy/](../build/deploy)
- **Services**: It is responsible for managing Kubernetes resources and **deploying** the Services required by an operational DSS. The ambition is to be cloud provider agnostic for the Services part. Services can be deployed using those approaches:
- [Helm Charts](services/helm-charts/dss)
- [Tanka](../build/deploy)

- **Operations**: It is responsible to provide diagnostic capabilities and utilities to **operate** the Services, such as certificates management may be simplified using the deployment manager CLI tools. It also contains the Infrastructure and Services configurations [used by the CI](../.github/workflows/dss-deploy.yml).

The following diagram represents the various modules per category and their impact on the various resources.
![Deploy Overview](../assets/generated/deploy_overview.png)

## Getting started

If you wish to deploy a DSS from scratch, "Getting Started" instructions can be found in the terraform modules and covers all steps to get a running DSS:
- [Amazon Web Services (EKS)](infrastructure/modules/terraform-aws-dss/README.md#Getting-started)
- [Google (GKE)](infrastructure/modules/terraform-google-dss/README.md#Getting-started)

For a real use case, you can look into the configurations of the [CI job](../.github/workflows/dss-deploy.yml) in operations: [ci](operations/ci)
19 changes: 17 additions & 2 deletions deploy/infrastructure/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,21 @@
# DSS Deployment
# Infrastructure

The `modules` directory contains the terraform modules required to deploy the DSS using cloud providers.
This folder contains the terraform modules required to prepare the infrastructure to host a DSS deployment.
See [Services](../services) to deploy the DSS once the infrastructure is ready.
barroco marked this conversation as resolved.
Show resolved Hide resolved

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The services folder doesn't have a README any more, so dropping a docs reader there probably isn't great without some additional context. I think we could keep and improve the services README or:

Suggested change
See [Services](../services) to deploy the DSS once the infrastructure is ready.
See [Services](../README.md#services) to deploy the DSS once the infrastructure is ready.

...and change **Services** in that README to ## Services.

The reason for linking there is that that is currently the only place we mention that there are two approaches to deploying services (along with the introduction).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the README from the services folder because it was only repeating the content of the parent README.
It is possible that we bring it back when we add content to it. I wanted to limit the duplication of the information. For the meantime, I changed the PR to direct the user to the README.md section Services as you proposed.

## Modules
The [modules](modules) directory contains the terraform public modules required to prepare the infrastructure on various cloud providers.

- [terraform-aws-dss](./modules/terraform-aws-dss/README.md): Amazon Web Services deployment
- [terraform-google-dss](./modules/terraform-google-dss/README.md): Google Cloud Engine deployment

## Dependencies
The [dependencies](dependencies) directory contains submodules used by the public modules described above. They are not expected to be
used directly by users. Those submodules are the combination of the cloud specific dependencies `terraform-*-kubernetes`
and `terraform-common-dss`. `terraform-common-dss` module aggregates and outputs the infrastructure configuration
which can be used as input to the `Services` deployment as shown in the diagram below.

![Infrastructure Modules](../../assets/generated/deploy_infrastructure_modules.png)

## Utils
This [utils](utils) folder contains scripts to help manage the terraform modules and dependencies. See [utils/README.md](utils/README.md) for details.
5 changes: 0 additions & 5 deletions deploy/services/README.md

This file was deleted.

Loading