Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to GitOps Config File #29

Merged
merged 21 commits into from
Dec 13, 2023
53 changes: 37 additions & 16 deletions .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,28 @@ jobs:
setup:
runs-on: ubuntu-latest
steps:
- name: Plan Atmos Component
uses: cloudposse/github-action-atmos-terraform-plan@main
with:
component: "foobar"
stack: "plat-ue2-sandbox"
component-path: "components/terraform/foobar"
terraform-plan-role: ${{ secrets.TERRAFORM_APPLY_ROLE }}
terraform-state-role: ${{ secrets.TERRAFORM_STATE_ROLE }}
- shell: bash
run: |
mkdir -p ${{ runner.temp }}
cat > ${{ runner.temp }}/atmos-gitops.yaml <<EOL
atmos-version: 1.45.3
atmos-config-path: ./tests/
terraform-state-bucket: ${{ secrets.TERRAFORM_STATE_BUCKET }}
terraform-state-table: ${{ secrets.TERRAFORM_STATE_TABLE }}
terraform-version: "1.5.2"
terraform-state-role: ${{ secrets.TERRAFORM_STATE_ROLE }}
terraform-plan-role: ${{ secrets.TERRAFORM_APPLY_ROLE }}
terraform-version: 1.5.2
aws-region: ${{ env.AWS_REGION }}
sort-by: .stack_slug
group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-")
EOL

- name: Plan Atmos Component
uses: cloudposse/github-action-atmos-terraform-plan@incapsulate-configs
with:
component: "foobar"
stack: "plat-ue2-sandbox"
atmos-gitops-config-path: "${{ runner.temp }}/atmos-gitops.yaml"
test:
runs-on: ubuntu-latest
needs: setup
Expand All @@ -39,15 +48,27 @@ jobs:
with:
ref: ${{ github.event.pull_request.head.ref }}

- shell: bash
run: |
mkdir -p ${{ runner.temp }}
cat > ${{ runner.temp }}/atmos-gitops.yaml <<EOL
atmos-version: 1.45.3
atmos-config-path: ./tests/
terraform-state-bucket: ${{ secrets.TERRAFORM_STATE_BUCKET }}
terraform-state-table: ${{ secrets.TERRAFORM_STATE_TABLE }}
terraform-state-role: ${{ secrets.TERRAFORM_STATE_ROLE }}
terraform-plan-role: ${{ secrets.TERRAFORM_APPLY_ROLE }}
terraform-apply-role: ${{ secrets.TERRAFORM_APPLY_ROLE }}
terraform-version: 1.5.2
aws-region: ${{ env.AWS_REGION }}
sort-by: .stack_slug
group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-")
EOL

- name: Apply Atmos Component
uses: ./
with:
component: "foobar"
stack: "plat-ue2-sandbox"
component-path: "components/terraform/foobar"
terraform-apply-role: ${{ secrets.TERRAFORM_APPLY_ROLE }}
terraform-state-role: ${{ secrets.TERRAFORM_STATE_ROLE }}
terraform-state-bucket: ${{ secrets.TERRAFORM_STATE_BUCKET }}
terraform-state-table: ${{ secrets.TERRAFORM_STATE_TABLE }}
terraform-version: "1.5.2"
aws-region: ${{ env.AWS_REGION }}
atmos-gitops-config-path: "${{ runner.temp }}/atmos-gitops.yaml"
debug: true
89 changes: 70 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,26 @@ For Cloud Posse documentation on setting up GitHub OIDC, see our [`github-oidc-p
In order to retrieve Terraform State, we configure an S3 Bucket to store plan files and a DynamoDB table to track plan metadata. Both will need to be deployed before running
this action. For more on setting up those components, see the `gitops` component (__documentation pending__). This action will then use the [github-action-terraform-plan-storage](https://github.com/cloudposse/github-action-terraform-plan-storage) action to update these resources.

### Config

The action expects the atmos gitops configuration file to be present in the repository in `./.github/config/atmos-gitops.yaml`.
The config should have the following structure:

```yaml
atmos-version: 1.45.3
atmos-config-path: ./rootfs/usr/local/etc/atmos/
terraform-state-bucket: cptest-core-ue2-auto-gitops
terraform-state-table: cptest-core-ue2-auto-gitops
terraform-state-role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha
terraform-plan-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops
terraform-apply-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops
terraform-version: 1.5.2
aws-region: us-east-2
enable-infracost: false
sort-by: .stack_slug
group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-")
```

### Workflow example

```yaml
Expand All @@ -105,18 +125,58 @@ this action. For more on setting up those components, see the `gitops` component
plan:
runs-on: ubuntu-latest
steps:
- name: Plan Atmos Component
uses: cloudposse/github-action-atmos-terraform-apply@v1
- name: Terraform Apply
uses: cloudposse/github-action-atmos-terraform-apply@v2
with:
component: "foobar"
stack: "plat-ue2-sandbox"
component-path: "components/terraform/s3-bucket"
terraform-apply-role: "arn:aws:iam::111111111111:role/acme-core-gbl-identity-gitops"
terraform-state-bucket: "acme-core-ue2-auto-gitops"
terraform-state-role: "arn:aws:iam::999999999999:role/acme-core-ue2-auto-gitops-gha"
terraform-state-table: "acme-core-ue2-auto-gitops"
aws-region: "us-east-2"
```

### Migrating from `v1` to `v2`

`v2` drop `component-path` variable. Now it fetches from `atmos.yaml` file automatically.

`v2` moved variables from `inputs` to atmos gitops config path `./.github/config/atmos-gitops.yaml`

| name |
|--------------------------|
| `atmos-version` |
| `atmos-config-path` |
| `terraform-state-bucket` |
| `terraform-state-table` |
| `terraform-state-role` |
| `terraform-plan-role` |
| `terraform-apply-role` |
| `terraform-version` |
| `aws-region` |
| `enable-infracost` |


If you want the same behavior in `v2` as in`v1` you should create config `./.github/config/atmos-gitops.yaml` with the same variables as in `v1` inputs.

```yaml
- name: Terraform apply
uses: cloudposse/github-action-atmos-terraform-apply@v2
with:
atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml
component: "foobar"
stack: "plat-ue2-sandbox"
```

same behaviour as

```yaml
- name: Terraform apply
uses: cloudposse/github-action-atmos-terraform-apply@v1
with:
component: "foobar"
stack: "plat-ue2-sandbox"
component-path: "components/terraform/s3-bucket"
terraform-apply-role: "arn:aws:iam::111111111111:role/acme-core-gbl-identity-gitops"
terraform-state-bucket: "acme-core-ue2-auto-gitops"
terraform-state-role: "arn:aws:iam::999999999999:role/acme-core-ue2-auto-gitops-gha"
terraform-state-table: "acme-core-ue2-auto-gitops"
aws-region: "us-east-2"
```


Expand All @@ -130,23 +190,14 @@ this action. For more on setting up those components, see the `gitops` component

| Name | Description | Default | Required |
|------|-------------|---------|----------|
| atmos-config-path | The path to the atmos.yaml file | atmos.yaml | false |
| atmos-version | Atmos version to use for vendoring. Default 'latest' | latest | false |
| aws-region | AWS region for assuming identity. | us-east-1 | false |
| atmos-gitops-config-path | The path to the atmos-gitops.yaml file | ./.github/config/atmos-gitops.yaml | false |
| branding-logo-image | Branding logo image url | https://cloudposse.com/logo-300x69.svg | false |
| branding-logo-url | Branding logo url | https://cloudposse.com/ | false |
| commit-sha | Commit SHA to apply. Default: github.sha | ${{ github.sha }} | true |
| component | The name of the component to apply. | N/A | true |
| component-path | The path to the base component. Atmos defines this value as component\_path. | N/A | true |
| debug | Enable action debug mode. Default: 'false' | false | false |
| enable-infracost | Whether to enable infracost summary. Requires secret `infracost-api-key` to be specified. Default: 'false | false | false |
| infracost-api-key | Infracost API key | N/A | false |
| sha | Commit SHA to apply. Default: github.sha | ${{ github.event.pull\_request.head.sha }} | true |
| stack | The stack name for the given component. | N/A | true |
| terraform-apply-role | The AWS role to be used to apply Terraform. | N/A | true |
| terraform-state-bucket | The S3 Bucket where the planfiles are stored. | N/A | true |
| terraform-state-role | The AWS role to be used to retrieve the planfile from AWS. | N/A | true |
| terraform-state-table | The DynamoDB table where planfile metadata is stored. | N/A | true |
| terraform-version | The version of Terraform CLI to install. Instead of full version string you can also specify constraint string starting with "<" (for example `<1.13.0`) to install the latest version satisfying the constraint. A value of `latest` will install the latest version of Terraform CLI. Defaults to `latest`. | latest | false |
| token | Used to pull node distributions for Atmos from Cloud Posse's GitHub repository. Since there's a default, this is typically not supplied by the user. When running this action on github.com, the default value is sufficient. When running on GHES, you can pass a personal access token for github.com if you are experiencing rate limiting. | ${{ github.server\_url == 'https://github.com' && github.token \|\| '' }} | false |


Expand Down
77 changes: 69 additions & 8 deletions README.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,26 @@ usage: |-
In order to retrieve Terraform State, we configure an S3 Bucket to store plan files and a DynamoDB table to track plan metadata. Both will need to be deployed before running
goruha marked this conversation as resolved.
Show resolved Hide resolved
this action. For more on setting up those components, see the `gitops` component (__documentation pending__). This action will then use the [github-action-terraform-plan-storage](https://github.com/cloudposse/github-action-terraform-plan-storage) action to update these resources.
goruha marked this conversation as resolved.
Show resolved Hide resolved

### Config

The action expects the atmos gitops configuration file to be present in the repository in `./.github/config/atmos-gitops.yaml`.
The config should have the following structure:

```yaml
atmos-version: 1.45.3
atmos-config-path: ./rootfs/usr/local/etc/atmos/
terraform-state-bucket: cptest-core-ue2-auto-gitops
terraform-state-table: cptest-core-ue2-auto-gitops
Comment on lines +64 to +65
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to rename these. These are not for the Terraform State Bucket. These are for the Terraform Plan File.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add this task to the backlog

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

terraform-state-role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add this task to the backlog

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

terraform-plan-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops
terraform-apply-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops
terraform-version: 1.5.2
aws-region: us-east-2
enable-infracost: false
sort-by: .stack_slug
group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-")
```

### Workflow example

```yaml
Expand All @@ -75,20 +95,61 @@ usage: |-
plan:
runs-on: ubuntu-latest
steps:
- name: Plan Atmos Component
uses: cloudposse/github-action-atmos-terraform-apply@v1
- name: Terraform Apply
uses: cloudposse/github-action-atmos-terraform-apply@v2
with:
component: "foobar"
stack: "plat-ue2-sandbox"
component-path: "components/terraform/s3-bucket"
terraform-apply-role: "arn:aws:iam::111111111111:role/acme-core-gbl-identity-gitops"
terraform-state-bucket: "acme-core-ue2-auto-gitops"
terraform-state-role: "arn:aws:iam::999999999999:role/acme-core-ue2-auto-gitops-gha"
terraform-state-table: "acme-core-ue2-auto-gitops"
aws-region: "us-east-2"
```

### Migrating from `v1` to `v2`

`v2` drop `component-path` variable. Now it fetches from `atmos.yaml` file automatically.

`v2` moved variables from `inputs` to atmos gitops config path `./.github/config/atmos-gitops.yaml`

goruha marked this conversation as resolved.
Show resolved Hide resolved
| name |
|--------------------------|
| `atmos-version` |
| `atmos-config-path` |
| `terraform-state-bucket` |
| `terraform-state-table` |
| `terraform-state-role` |
| `terraform-plan-role` |
| `terraform-apply-role` |
| `terraform-version` |
| `aws-region` |
| `enable-infracost` |


If you want the same behavior in `v2` as in`v1` you should create config `./.github/config/atmos-gitops.yaml` with the same variables as in `v1` inputs.
goruha marked this conversation as resolved.
Show resolved Hide resolved

```yaml
- name: Terraform apply
uses: cloudposse/github-action-atmos-terraform-apply@v2
with:
atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml
component: "foobar"
stack: "plat-ue2-sandbox"
```

same behaviour as
goruha marked this conversation as resolved.
Show resolved Hide resolved

```yaml
- name: Terraform apply
uses: cloudposse/github-action-atmos-terraform-apply@v1
with:
component: "foobar"
stack: "plat-ue2-sandbox"
component-path: "components/terraform/s3-bucket"
terraform-apply-role: "arn:aws:iam::111111111111:role/acme-core-gbl-identity-gitops"
terraform-state-bucket: "acme-core-ue2-auto-gitops"
terraform-state-role: "arn:aws:iam::999999999999:role/acme-core-ue2-auto-gitops-gha"
terraform-state-table: "acme-core-ue2-auto-gitops"
aws-region: "us-east-2"
```


include:
- "docs/github-action.md"

Expand Down
Loading
Loading