Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(layer): publish SAR v2 via Github actions #1585

Merged
merged 45 commits into from
Oct 13, 2022
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
567939b
feat(sar): add support for publishing SAR via Github Actions
rubenfonseca Oct 11, 2022
447f51a
chore: generate hashes for all deps
rubenfonseca Oct 11, 2022
558e333
chore: use git version to build layer
rubenfonseca Oct 11, 2022
320f47f
chore: fix action
rubenfonseca Oct 11, 2022
958e61c
chore: rely on RELEASE_TAG_VERSION
rubenfonseca Oct 11, 2022
1ddedf2
fix: typo
rubenfonseca Oct 11, 2022
2132216
fix: require version input
rubenfonseca Oct 11, 2022
470a097
chore: fix
rubenfonseca Oct 11, 2022
75a7a7c
fix: rename stack names so it doesn't clash with v1
rubenfonseca Oct 11, 2022
c9f64b9
fix: change stack names
rubenfonseca Oct 11, 2022
6c9c066
fix: use new SAR role
rubenfonseca Oct 12, 2022
17b6f3d
fix: assume role after assuming OIDC role
rubenfonseca Oct 12, 2022
09b984d
fix: typo
rubenfonseca Oct 12, 2022
498c6c3
fix: assume sar
rubenfonseca Oct 12, 2022
7f1fe80
fix: role duration session
rubenfonseca Oct 12, 2022
a3eecbf
fix: env
rubenfonseca Oct 12, 2022
91bc1e1
fix: sed escape
rubenfonseca Oct 12, 2022
d50d531
fix: sam version
rubenfonseca Oct 12, 2022
c467ee0
fix: remove cdk
rubenfonseca Oct 12, 2022
ede9983
fix: debug
rubenfonseca Oct 12, 2022
99b88a3
fix: readme and license path
rubenfonseca Oct 12, 2022
ae235bd
fix: path
rubenfonseca Oct 12, 2022
477545d
fix: change SAR region
rubenfonseca Oct 12, 2022
45e8c27
fix: force version
rubenfonseca Oct 12, 2022
66ca24a
chore: now remove the test SAR
rubenfonseca Oct 12, 2022
46435de
fix: artifact path
rubenfonseca Oct 12, 2022
90c175b
feat: add SAR canary
rubenfonseca Oct 12, 2022
25dd302
fix: typo
rubenfonseca Oct 12, 2022
4a66642
fix: aws output
rubenfonseca Oct 13, 2022
e0774a7
fix: debug
rubenfonseca Oct 13, 2022
46425ea
fix: it will work now
rubenfonseca Oct 13, 2022
e7fa0ae
fix: typo
rubenfonseca Oct 13, 2022
226242f
chore: add production
rubenfonseca Oct 13, 2022
9d6177c
chore: parallelize SAR
rubenfonseca Oct 13, 2022
b8798b2
fix: cleanup
rubenfonseca Oct 13, 2022
136d0e3
fix: comment out deploy production layer for the time being
rubenfonseca Oct 13, 2022
b98d539
fix: cleanup
rubenfonseca Oct 13, 2022
987b524
chore: apply suggestions from code review
rubenfonseca Oct 13, 2022
0f62b6e
chore: add comments to the SAR workflow
rubenfonseca Oct 13, 2022
f4cc412
fix: stop using deprecated set-output
rubenfonseca Oct 13, 2022
9f10f5e
fix: remove unecessary ENV
rubenfonseca Oct 13, 2022
9abec5a
chore: line editing
heitorlessa Oct 13, 2022
578448a
chore: line editing
heitorlessa Oct 13, 2022
57b40ab
chore: line editing part2
heitorlessa Oct 13, 2022
54f79f8
chore: typo from line-editing
heitorlessa Oct 13, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 27 additions & 2 deletions .github/workflows/publish_v2_layer.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ on:
workflow_dispatch:
inputs:
latest_published_version:
description: "Latest PyPi published version to rebuild latest docs for, e.g. v1.22.0"
default: "v2.0.0"
description: "Latest PyPi published version to rebuild latest docs for, e.g. v2.0.0"
required: true
# workflow_run:
# workflows: ["Publish to PyPi"]
Expand All @@ -23,6 +22,8 @@ jobs:
defaults:
run:
working-directory: ./layer
outputs:
heitorlessa marked this conversation as resolved.
Show resolved Hide resolved
release-tag-version: ${{ steps.release-notes-tag.outputs.release-tag-version }}
steps:
- name: checkout
uses: actions/checkout@v3
Expand All @@ -46,10 +47,12 @@ jobs:
poetry export --format requirements.txt --output requirements.txt
pip install -r requirements.txt
- name: Set release notes tag
id: release-notes-tag
run: |
RELEASE_INPUT=${{ inputs.latest_published_version }}
LATEST_TAG=$(git describe --tag --abbrev=0)
RELEASE_TAG_VERSION=${RELEASE_INPUT:-$LATEST_TAG}
echo "::set-output name=release-tag-version::$RELEASE_TAG_VERSION"
heitorlessa marked this conversation as resolved.
Show resolved Hide resolved
echo RELEASE_TAG_VERSION="${RELEASE_TAG_VERSION:1}" >> "$GITHUB_ENV"
- name: Set up QEMU
uses: docker/setup-qemu-action@8b122486cedac8393e77aa9734c3528886e4a1a8 # v2.0.0
Expand Down Expand Up @@ -90,3 +93,25 @@ jobs:
# stage: "PROD"
# artefact-name: "cdk-layer-artefact"
# environment: "layer-prod"

deploy-sar-beta:
needs:
- build-layer
rubenfonseca marked this conversation as resolved.
Show resolved Hide resolved
uses: ./.github/workflows/reusable_deploy_v2_sar.yml
secrets: inherit
with:
stage: "BETA"
artefact-name: "cdk-layer-artefact"
environment: "layer-beta"
package-version: ${{ needs.build-layer.outputs.release-tag-version }}

deploy-sar-prod:
needs:
- deploy-sar-beta
rubenfonseca marked this conversation as resolved.
Show resolved Hide resolved
uses: ./.github/workflows/reusable_deploy_v2_sar.yml
secrets: inherit
with:
stage: "PROD"
artefact-name: "cdk-layer-artefact"
environment: "layer-prod"
package-version: ${{ needs.build-layer.outputs.release-tag-version }}
4 changes: 2 additions & 2 deletions .github/workflows/reusable_deploy_v2_layer_stack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,6 @@ jobs:
- name: unzip artefact
run: unzip cdk.out.zip
- name: CDK Deploy Layer
run: cdk deploy --app cdk.out --context region=${{ matrix.region }} 'LayerStack' --require-approval never --verbose
run: cdk deploy --app cdk.out --context region=${{ matrix.region }} 'LayerV2Stack' --require-approval never --verbose
- name: CDK Deploy Canary
run: cdk deploy --app cdk.out --context region=${{ matrix.region}} --parameters DeployStage="${{ inputs.stage }}" 'CanaryStack' --require-approval never --verbose
run: cdk deploy --app cdk.out --context region=${{ matrix.region}} --parameters DeployStage="${{ inputs.stage }}" 'CanaryV2Stack' --require-approval never --verbose
124 changes: 124 additions & 0 deletions .github/workflows/reusable_deploy_v2_sar.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
name: Deploy V2 SAR

permissions:
id-token: write
contents: read

env:
SAM_VERSION: 2.44.0
NODE_VERSION: 16.12
AWS_REGION: eu-west-1
SAR_NAME: aws-lambda-powertools-python-layer-v2
heitorlessa marked this conversation as resolved.
Show resolved Hide resolved

on:
workflow_call:
inputs:
stage:
description: "Deployment stage (BETA, PROD)"
required: true
type: string
artefact-name:
description: "CDK Layer Artefact name to download"
required: true
type: string
package-version:
description: "The version of the package to deploy"
required: true
type: string
environment:
description: "GitHub Environment to use for encrypted secrets"
required: true
type: string

jobs:
deploy-sar-app:
runs-on: ubuntu-latest
environment: ${{ inputs.environment }}
strategy:
matrix:
architecture: ["x86_64", "arm64"]
steps:
- name: Checkout
uses: actions/checkout@v3
- name: AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-region: ${{ env.AWS_REGION }}
role-to-assume: ${{ secrets.AWS_LAYERS_ROLE_ARN }}
- name: AWS credentials SAR role
uses: aws-actions/configure-aws-credentials@v1
id: aws-credentials-sar-role
with:
aws-access-key-id: ${{ env.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ env.AWS_SECRET_ACCESS_KEY }}
aws-session-token: ${{ env.AWS_SESSION_TOKEN }}
role-duration-seconds: 1200
aws-region: ${{ env.AWS_REGION }}
role-to-assume: ${{ secrets.AWS_SAR_V2_ROLE_ARN }}
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_VERSION }}
- name: Download artifact
uses: actions/download-artifact@v3
with:
name: ${{ inputs.artefact-name }}
- name: Unzip artefact
run: unzip cdk.out.zip
- name: Configure SAR name
run: |
NAME=${{ env.SAR_NAME }}
if [[ "${{ inputs.stage }}" == "BETA" ]]; then
NAME="test-$NAME"
rubenfonseca marked this conversation as resolved.
Show resolved Hide resolved
fi
echo SAR_NAME="${NAME}" >> "$GITHUB_ENV"
- name: Adds arm64 suffix to SAR name
if: ${{ matrix.architecture == 'arm64' }}
run: echo SAR_NAME="${SAR_NAME}-arm64" >> "$GITHUB_ENV"
- name: Deploy SAR
run: |
rubenfonseca marked this conversation as resolved.
Show resolved Hide resolved
asset=$(jq -jc '.Resources[] | select(.Properties.CompatibleArchitectures == ["${{ matrix.architecture }}"]) | .Metadata."aws:asset:path"' cdk.out/LayerV2Stack.template.json)
sed -e "s|<VERSION>|${{ inputs.package-version }}|g" -e "s/<SAR_APP_NAME>/${{ env.SAR_NAME }}/g" -e "s|<LAYER_CONTENT_PATH>|./cdk.out/$asset|g" layer/sar/template.txt > template.yml
cp README.md LICENSE "./cdk.out/$asset/"
sam package --template-file template.yml --output-template-file packaged.yml --s3-bucket ${{ secrets.AWS_SAR_S3_BUCKET }}
sam publish --template packaged.yml --region "$AWS_REGION"
- name: Deploy BETA canary
if: ${{ inputs.stage == 'BETA' }}
run: |
STACK_NAME=serverlessrepo-v2-powertools-layer-test-stack
rubenfonseca marked this conversation as resolved.
Show resolved Hide resolved

if [[ "${{ matrix.architecture }}" == "arm64" ]]; then
STACK_NAME="${STACK_NAME}-arm64"
fi

echo "Check if stack does not exist"
stack_exists=$(aws cloudformation list-stacks --query "StackSummaries[?(StackName == '$STACK_NAME' && StackStatus == 'CREATE_COMPLETE')].{StackId:StackId, StackName:StackName, CreationTime:CreationTime, StackStatus:StackStatus}" --output text)
if [[ -n "$stack_exists" ]] ; then
rubenfonseca marked this conversation as resolved.
Show resolved Hide resolved
echo "Found test deployment stack, removing..."
aws cloudformation delete-stack --stack-name "$STACK_NAME"
aws cloudformation wait stack-delete-complete --stack-name "$STACK_NAME"
rubenfonseca marked this conversation as resolved.
Show resolved Hide resolved
fi

echo "Creating canary stack"
echo "Stack name: $STACK_NAME"
aws serverlessrepo create-cloud-formation-change-set --application-id arn:aws:serverlessrepo:${{ env.AWS_REGION }}:${{ steps.aws-credentials-sar-role.outputs.aws-account-id }}:applications/${{ env.SAR_NAME }} --stack-name "${STACK_NAME/serverlessrepo-/}" --capabilities CAPABILITY_NAMED_IAM
CHANGE_SET_ID=$(aws cloudformation list-change-sets --stack-name "$STACK_NAME" --query 'Summaries[*].ChangeSetId' --output text)
aws cloudformation wait change-set-create-complete --change-set-name "$CHANGE_SET_ID"
aws cloudformation execute-change-set --change-set-name "$CHANGE_SET_ID"
aws cloudformation wait stack-create-complete --stack-name "$STACK_NAME"
echo "Waiting until stack deployment completes..."

echo "Exit with error if stack is not in CREATE_COMPLETE"
stack_exists=$(aws cloudformation list-stacks --query "StackSummaries[?(StackName == '$STACK_NAME' && StackStatus == 'CREATE_COMPLETE')].{StackId:StackId, StackName:StackName, CreationTime:CreationTime, StackStatus:StackStatus}")
if [[ -z "$stack_exists" ]] ; then
echo "Could find successful deployment, exit error..."
exit 1
fi
echo "Deployment successful"
heitorlessa marked this conversation as resolved.
Show resolved Hide resolved
- name: Publish SAR
if: ${{ inputs.stage == 'PROD' }}
run: |
# wait until SAR registers the app, otherwise it fails to make it public
sleep 15
heitorlessa marked this conversation as resolved.
Show resolved Hide resolved
echo "Make SAR app public"
aws serverlessrepo put-application-policy --application-id arn:aws:serverlessrepo:${{ env.AWS_REGION }}:${{ steps.aws-credentials-sar-role.outputs.aws-account-id }}:applications/${{ env.SAR_NAME }} --statements Principals='*',Actions=Deploy
4 changes: 2 additions & 2 deletions layer/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,15 @@

LayerStack(
app,
"LayerStack",
"LayerV2Stack",
powertools_version=POWERTOOLS_VERSION,
ssm_paramter_layer_arn=SSM_PARAM_LAYER_ARN,
ssm_parameter_layer_arm64_arn=SSM_PARAM_LAYER_ARM64_ARN,
)

CanaryStack(
app,
"CanaryStack",
"CanaryV2Stack",
powertools_version=POWERTOOLS_VERSION,
ssm_paramter_layer_arn=SSM_PARAM_LAYER_ARN,
ssm_parameter_layer_arm64_arn=SSM_PARAM_LAYER_ARM64_ARN,
Expand Down
Loading