feat: ECS Partial Task Definitions & GitHub OIDC setup

docs/layers/github-actions/github-oidc-with-aws.mdx

@@ -88,10 +88,18 @@ sequenceDiagram
   ### <StepNumber/> Deploy GitHub OIDC Provider Component
 
   After deploying the [GitHub OIDC Provider component](/components/library/aws/github-oidc-provider/) into an account, you should see the Identity Provider in IAM in the AWS Web Console.
+
+  Deploy this component in each account where GitHub Actions need to assume a role.
+
+  <Steps>
+    - Import `catalog/github-oidc-provider` in the `gbl` stack for the given account
+    - Deploy the `github-oidc-provider` component: `atmos terraform apply github-oidc-provider -s plat-gbl-dev`
+  </Steps>
+
   </Step>
 
   <Step>
-  ### <StepNumber/> Configure GitHub OIDC Mixin Role and Policy
+  ### <StepNumber/> Option 1: Configure GitHub OIDC Mixin Role and Policy
 
   Use the mixin to grant GitHub the ability to assume a role for a specific component.
 
@@ -100,7 +108,7 @@ sequenceDiagram
   </Step>
 
   <Step>
-  ### <StepNumber/> Deploy GitHub OIDC Role Component
+  ### <StepNumber/> Option 2: Deploy GitHub OIDC Role Component
 
   Deploy the [GitHub OIDC Role component](/components/library/aws/github-oidc-role/) to create a generalized role for GitHub to access several resources in AWS.
   </Step>

docs/layers/software-delivery/ecs-ecspresso/ecs-partial-task-definitions.mdx

@@ -0,0 +1,221 @@
+---
+title: "ECS Partial Task Definitions"
+sidebar_label: "Partial Task Definitions"
+sidebar_position: 20
+---
+
+import Intro from '@site/src/components/Intro';
+import Steps from '@site/src/components/Steps';
+
+<Intro>
+  This document describes what partial task definitions are and how we can use them to set up ECS services using Terraform and GitHub Actions.
+</Intro>
+
+## The Problem
+
+Managing ECS Services is challenging. Ideally, we want our services to be managed by Terraform so everything is living
+in code. However, we also want to update the task definition via GitOps as through the GitHub release lifecycle. This is
+challenging because Terraform can create the task definition, but if updated by the application repository, the task
+definition will be out of sync with the Terraform state.
+
+Managing it entirely through Terraform means we cannot easily update the newly built image by the application repository
+unless we directly commit to the infrastructure repository, which is not ideal.
+
+Managing it entirely through the application repository means we cannot codify the infrastructure and have to hardcode
+ARNs, secrets, and other infrastructure-specific configurations.
+
+## Introduction
+
+ECS Partial task definitions is the idea of breaking the task definition into smaller parts. This allows for easier
+management of the task definition and makes it easier to update the task definition.
+
+We do this by setting up Terraform to manage a portion of the task definition, and the application repository to manage
+another portion.
+
+The Terraform (infrastructure) portion is created first. It will create an ECS Service in ECS, and then upload the task
+definition JSON to S3 as `task-template.json`.The application repository will have a `task-definition.json` git
+controlled, during the development lifecycle, the application repository will download the task definition from S3,
+merge the task definitions, then update the ECS Service with the new task definition. Finally, GitHub actions will
+update the S3 bucket with the deployed task definition under `task-definition.json`. If Terraform is planned again, it
+will use the new task definition as the base for the next deployment, thus not resetting the image or application
+configuration.
+
+<img src="/assets/ecs-partial-task-definitions.png" /><br/>
+
+### Pros
+
+The **benefit** to using this approach is that we can manage the task definition portion in Terraform with the
+infrastructure, meaning secrets, volumes, and other ARNs can be managed in Terraform. If a filesystem ID updates we can
+re-apply Terraform to update the task definition with the new filesystem ID. The application repository can manage the
+container definitions, environment variables, and other application-specific configurations. This allows developers who
+are closer to the application to quickly update the environment variables or other configuration.
+
+### Cons
+
+The drawback to this approach is that it is more complex than managing the task definition entirely in Terraform or the
+application repository. It requires more setup and more moving parts. It can be confusing for a developer who is not
+familiar with the setup to understand how the task definition is being managed and deployed.
+
+This also means that when something goes wrong, it becomes harder to troubleshoot as there are more moving parts.
+
+### Getting Setup
+
+#### Pre-requisites
+
+- Application Repository - [Cloud Posse Example ECS Application](https://github.com/cloudposse-examples/app-on-ecs)
+- Infrastructure Repository
+- ECS Cluster - [Cloud Posse Docs](https://docs.cloudposse.com/components/library/aws/ecs/) -
+  [Component](https://github.com/cloudposse/Terraform-aws-components/tree/main/modules/ecs).
+- `ecs-service` - [Cloud Posse Docs](https://docs.cloudposse.com/components/library/aws/ecs-service/) -
+  [Component](https://github.com/cloudposse/Terraform-aws-components/tree/main/modules/ecs-service).
+  - **Must** use the Cloud Posse Component.
+  - [`v1.416.0`](https://github.com/cloudposse/Terraform-aws-components/releases/tag/1.416.0) or later.
+- S3 Bucket - [Cloud Posse Docs](https://docs.cloudposse.com/components/library/aws/s3-bucket/) -
+  [Component](https://github.com/cloudposse/Terraform-aws-components/tree/main/modules/s3-bucket).
+
+#### Steps
+
+<Steps>
+
+1. Set up the S3 Bucket that will store the task definition.
+
+   <br/>This bucket should be in the same account as the ECS Cluster.
+
+   <br/>
+   <details>
+   <summary>S3 Bucket Default Definition</summary>
+
+   ```yaml
+   components:
+     terraform:
+       s3-bucket/defaults:
+         metadata:
+           type: abstract
+         vars:
+           enabled: true
+           account_map_tenant_name: core
+           # Suggested configuration for all buckets
+           user_enabled: false
+           acl: "private"
+           grants: null
+           force_destroy: false
+           versioning_enabled: false
+           allow_encrypted_uploads_only: true
+           block_public_acls: true
+           block_public_policy: true
+           ignore_public_acls: true
+           restrict_public_buckets: true
+           allow_ssl_requests_only: true
+           lifecycle_configuration_rules:
+             - id: default
+               enabled: true
+               abort_incomplete_multipart_upload_days: 90
+               filter_and:
+                 prefix: ""
+                 tags: {}
+               # Move to Glacier after 2 years
+               transition:
+                 - storage_class: GLACIER
+                   days: 730
+               # Never expire
+               expiration: {}
+               # Versioning isnt enabled, but these default values are still required
+               noncurrent_version_transition:
+                 - storage_class: GLACIER
+                   days: 90
+               noncurrent_version_expiration: {}
+   ```
+
+   </details>
+
+   ```yaml
+   import:
+     - catalog/s3-bucket/defaults
+
+   components:
+     terraform:
+       s3-bucket/ecs-tasks-mirror: #NOTE this is the component instance name.
+         metadata:
+           component: s3-bucket
+           inherits:
+             - s3-bucket/defaults
+         vars:
+           enabled: true
+           name: ecs-tasks-mirror
+   ```
+
+2. Create an ECS Service in Terraform
+
+   <br/>Set up the ECS Service in Terraform using the
+   [`ecs-service` component](https://github.com/cloudposse/Terraform-aws-components/tree/main/modules/ecs-service). This
+   will create the ECS Service and upload the task definition to the S3 bucket.
+
+   <br/>To enable Partial Task Definitions, set the variable `s3_mirror_name` to be the component instance name of the
+   bucket to mirror to. For example `s3-bucket/ecs-tasks-mirror`
+
+   ```yaml
+   components:
+     terraform:
+       ecs-services/defaults:
+         metadata:
+           component: ecs-service
+           type: abstract
+         vars:
+           enabled: true
+           ecs_cluster_name: "ecs/cluster"
+           s3_mirror_name: s3-bucket/ecs-tasks-mirror
+   ```
+
+3. Set up an Application repository with GitHub workflows.
+
+   An example application repository can be found [here](https://github.com/cloudposse-examples/app-on-ecs).
+
+   <br/> Two things need to be pulled from this repository:
+
+   - The `task-definition.json` file under `deploy/task-definition.json`
+   - The GitHub Workflows.
+
+   An important note about the GitHub Workflows, in the example repository they all live under `.github/workflows`. This
+   is done so development of workflows can be fast, however we recommend moving the shared workflows to a separate
+   repository and calling them from the application repository. The application repository should only contain the
+   workflows `main-branch.yaml`, `release.yaml` and `feature-branch.yml`.
+
+   <br/>To enable Partial Task Definitions in the workflows, the call to
+   [`cloudposse/github-action-run-ecspresso` (link)](https://github.com/cloudposse-examples/app-on-ecs/blob/main/.github/workflows/workflow-cd-ecspresso.yml#L133-L147)
+   should have the input `mirror_to_s3_bucket` set to the S3 bucket name. the variable `use_partial_taskdefinition`
+   should be set to `'true'`
+
+   <details>
+   <summary> Example GitHub Action Step </summary>
+
+   ```yaml
+   - name: Deploy
+     uses: cloudposse/[email protected]
+     continue-on-error: true
+     if: ${{ steps.db_migrate.outcome != 'failure' }}
+     id: deploy
+     with:
+       image: ${{ steps.image.outputs.out }}
+       image-tag: ${{ inputs.tag }}
+       region: ${{ steps.environment.outputs.region }}
+       operation: deploy
+       debug: false
+       cluster: ${{ steps.environment.outputs.cluster }}
+       application: ${{ steps.environment.outputs.name }}
+       taskdef-path: ${{ inputs.path }}
+       mirror_to_s3_bucket: ${{ steps.environment.outputs.s3-bucket }}
+       use_partial_taskdefinition: "true"
+       timeout: 10m
+   ```
+
+   </details>
+
+</Steps>
+
+## Operation
+
+Changes through Terraform will not immediately be reflected in the ECS Service. This is because the task template has
+been updated, but whatever was in the `task-definition.json` file in the S3 bucket will be used for deployment.
+
+To update the ECS Service after updating the Terraform for it, you must deploy through GitHub Actions. This will then
+download the new template and create a new updated `task-defintion.json` to store in s3.

docs/layers/software-delivery/ecs-ecspresso/setup.mdx

@@ -23,7 +23,8 @@ import AtmosWorkflow from '@site/src/components/AtmosWorkflow';
 | 3. Validate the environment configuration            | Click Ops                                                                         |
 | 4. Create a GitHub PAT                               | Click Ops                                                                         |
 | 5. Set all Example App repository secrets            | Click Ops                                                                         |
-| 6. Deploy the example ECS services                   | `atmos workflow deploy/app-on-ecs -f app-on-ecs`                                  |
+| 6. Deploy the shared ECS Task Definition S3 Bucket   | `atmos apply s3-bucket/ecs-tasks-mirror -s < YOUR STACK >`                        |
+| 7. Deploy the example ECS services                   | `atmos workflow deploy/app-on-ecs -f app-on-ecs`                                  |
 
 <Note title="Note">
 
@@ -306,6 +307,12 @@ We do not recommend keeping all shared workflows in the same repository as in th
     </dl>
   </Step>
 
+  <Step>
+    ### <StepNumber/> Configure the S3 Mirror Bucket, if not already configured
+
+    If you haven't already configured the S3 mirror bucket, deploy and configure the shared S3 bucket for ECS tasks definitions now. Follow the [ECS Partial Task Definitions guide](/layers/software-delivery/ecs-ecspresso/ecs-partial-task-definitions/#steps)
+  </Step>
+
   <Step>
     ### <StepNumber/> Deploy the Example App ECS Service
 
@@ -368,7 +375,7 @@ We do not recommend keeping all shared workflows in the same repository as in th
 
     </details>
 
-    Apply this component with the following:
+    Finally, apply the `ecs-services/example-app-on-ecs` component to deploy the Example App ECS service.
 
     <AtmosWorkflow workflow="deploy/app-on-ecs" fileName="app-on-ecs" />
   </Step>