diff --git a/README.md b/README.md index 7bc79b9..d418fe6 100644 --- a/README.md +++ b/README.md @@ -5,6 +5,28 @@ This repository contains GitHub Actions that are common to drivers. +## Setup + +There is a common setup action that is meant to be run before all +other actions. It handles fetching secrets from AWS Secrets Manager, +signing into Artifactory, setting up Garasign credentials, and +setting up environment variables used in other actions. +The action requires `id-token: write` permissions. + +```yaml +- name: setup + uses: mongodb/drivers-github-tools/setup@v2 + with: + aws_role_arn: ${{ secrets.AWS_ROLE_ARN }} + aws_region_name: ${{ vars.AWS_REGION_NAME }} + aws_secret_id: ${{ secrets.AWS_SECRET_ID }} +``` + +> [!Note] +> You *must* use the `actions/checkout` action prior to calling the `setup` action, +> Since the `setup` action sets up git config that would be overridden by the +> `actions/checkout action` + ## Signing tools The actions in the `garasign` folder are used to sign artifacts using the team's @@ -15,60 +37,49 @@ GPG key. Use this action to create signed git artifacts: ```yaml -- name: "Create signed commit" - uses: mongodb/drivers-github-tools/garasign/git-sign@main +- name: Setup + uses: mongodb/drivers-github-tools/setup@v2 with: - command: "git commit -m 'Commit' -s --gpg-sign=${{ vars.GPG_KEY_ID }}" - garasign_username: ${{ secrets.GRS_CONFIG_USER1_USERNAME }} - garasign_password: ${{ secrets.GRS_CONFIG_USER1_PASSWORD }} - artifactory_username: ${{ secrets.ARTIFACTORY_USER }} - artifactory_password: ${{ secrets.ARTIFACTORY_PASSWORD }} - -- name: "Create signed tag" - uses: mongodb/drivers-github-tools/garasign/git-sign@main - with: - command: "git tag -m 'Tag' -s --local-user=${{ vars.GPG_KEY_ID }} " - garasign_username: ${{ secrets.GRS_CONFIG_USER1_USERNAME }} - garasign_password: ${{ secrets.GRS_CONFIG_USER1_PASSWORD }} - artifactory_username: ${{ secrets.ARTIFACTORY_USER }} - artifactory_password: ${{ secrets.ARTIFACTORY_PASSWORD }} - skip_setup: true -``` + ... -If the action is used multiple times within the same job, the `skip_setup` -option can be set to a truthy value to avoid unnecessary logins to artifactory. +- name: Create signed commit + uses: mongodb/drivers-github-tools/git-sign@v2 + +- name: Create signed tag + uses: mongodb/drivers-github-tools/git-sign@v2 +``` ### gpg-sign This action is used to create detached signatures for files: ```yaml -- name: "Create detached signature" - uses: mongodb/drivers-github-tools/garasign/gpg-sign@main +- name: Setup + uses: mongodb/drivers-github-tools/setup@v2 + with: + ... + +- name: Create detached signature + uses: mongodb/drivers-github-tools/gpg-sign@v2 with: filenames: somefile.ext - garasign_username: ${{ secrets.GRS_CONFIG_USER1_USERNAME }} - garasign_password: ${{ secrets.GRS_CONFIG_USER1_PASSWORD }} - artifactory_username: ${{ secrets.ARTIFACTORY_USER }} - artifactory_password: ${{ secrets.ARTIFACTORY_PASSWORD }} ``` The action will create a signature file `somefile.ext.sig` in the working directory. -If the action is used multiple times within the same job, the `skip_setup` -option can be set to a truthy value to avoid unnecessary logins to artifactory. -You can also supply multiple space-separated filenames to sign a list of files: +You can also supply a glob pattern to sign a group of files: ```yaml -- name: "Create detached signature" - uses: mongodb/drivers-github-tools/garasign/gpg-sign@main +- name: Setup + uses: mongodb/drivers-github-tools/setup@v2 + with: + ... + +- name: Create detached signature + uses: mongodb/drivers-github-tools/garasign/gpg-sign@v1 with: filenames: dist/* - garasign_username: ${{ secrets.GRS_CONFIG_USER1_USERNAME }} - garasign_password: ${{ secrets.GRS_CONFIG_USER1_PASSWORD }} - artifactory_username: ${{ secrets.ARTIFACTORY_USER }} - artifactory_password: ${{ secrets.ARTIFACTORY_PASSWORD }} ``` ## Reporting tools @@ -76,17 +87,69 @@ You can also supply multiple space-separated filenames to sign a list of files: The following tools are meant to aid in generating Software Security Development Lifecycle reports associated with a product release. -### Papertrail +### Authorized Publication This action will create a record of authorized publication on distribution channels. -By default it will create a "papertrail.txt" file in the current directory. +It will create the file `$S3_ASSETS/authorized_publication.txt` ```yaml -- name: "Create papertrail report" - uses: mongodb/drivers-github-tools/papertrail@main +- name: Setup + uses: mongodb/drivers-github-tools/setup@v2 + with: + ... + +- name: Create Authorized Publication Report + uses: mongodb/drivers-github-tools/authorized-pub@v2 with: product_name: Mongo Python Driver release_version: ${{ github.ref_name }} filenames: dist/* token: ${{ github.token }} ``` + +## Python Helper Scripts + +These scripts are opinionated helper scripts for Python releases. + +### Bump and Tag + +Bump the version and create a new tag. Verify the tag. +Push the commit and tag to the source branch unless `dry_run` is set. + +```yaml +- name: Setup + uses: mongodb/drivers-github-tools/setup@v2 + with: + ... + +- uses: mongodb/drivers-github-tools/python/bump-and-tag@v2 + with: + version: ${{ inputs.version }} + version_bump_script: ./.github/scripts/bump-version.sh + dry_run: ${{ inputs.dry_run }} +``` + +### Publish + +Handles tasks related to publishing Python packages, including +signing `dist` file and publishing the `dist` files to PyPI. +It will also push the following (dev) version to the source branch. +It will create a draft GitHub release and attach the signature files. +Finally, it will publish a report to the appropriate S3 bucket. +If `dry_run` is set, nothing will be published or pushed. + +```yaml +- name: Setup + uses: mongodb/drivers-github-tools/setup@v2 + with: + ... + +- uses: mongodb-labs/drivers-github-tools/python/publish@v2 + with: + version: ${{ inputs.version }} + following_version: ${{ inputs.following_version }} + version_bump_script: ./.github/scripts/bump-version.sh + product_name: winkerberos + token: ${{ github.token }} + dry_run: ${{ inputs.dry_run }} +``` \ No newline at end of file diff --git a/authorized-pub/action.yml b/authorized-pub/action.yml new file mode 100644 index 0000000..7bcf677 --- /dev/null +++ b/authorized-pub/action.yml @@ -0,0 +1,35 @@ +name: Authorized Publication +description: Generate report for authorized publication on distribution channels +inputs: + product_name: + description: Name of product + required: true + release_version: + description: The release version + required: true + filenames: + description: Artifact filename(s) to include in the report, can be a glob pattern + required: true + token: + description: The GitHub token for the action + required: true + +runs: + using: composite + steps: + - name: Prepare report + shell: bash + run: | + export GH_TOKEN=${{ inputs.token }} + NAME=$(gh api users/${{ github.actor }} --jq '.name') + export REPORT=$S3_ASSETS/authorized_publication.txt + echo "Product: ${{ inputs.product_name }}" > $REPORT + echo "Version: ${{ inputs.release_version }}" >> $REPORT + echo "Releaser: $NAME" >> $REPORT + echo "Build Source: GitHub Actions" + echo "Build Number: ${{ github.run_id }}" + for filename in ${{ inputs.filenames }}; do + SHA=$(shasum -a 256 $filename | awk '{print $1;}') + echo "Filename: $filename" >> $REPORT + echo "Shasum: $SHA" >> $REPORT + done diff --git a/garasign/git-sign/action.yml b/garasign/git-sign/action.yml deleted file mode 100644 index adfd484..0000000 --- a/garasign/git-sign/action.yml +++ /dev/null @@ -1,58 +0,0 @@ -name: "Run git actions in a signing container" -description: "Allows running arbitrary git actions in a container with GPG keys loaded" -inputs: - command: - description: "Command to run inside the container" - required: true - garasign_username: - description: "Garasign username" - required: true - garasign_password: - description: "Garasign password" - required: true - artifactory_username: - description: "Artifactory user" - required: true - artifactory_password: - description: "Artifactory password" - required: true - artifactory_image: - description: "Image to use for artifactory" - default: release-tools-container-registry-local/garasign-git - artifactory_registry: - description: "Artifactory registry to be used" - default: artifactory.corp.mongodb.com - skip_setup: - description: "Whether to skip setup" - default: "false" - -runs: - using: composite - steps: - - name: Create the envfile - if: ${{ inputs.skip_setup == 'false' }} - run: | - cat << EOF > envfile - GRS_CONFIG_USER1_USERNAME=${{ inputs.garasign_username }} - GRS_CONFIG_USER1_PASSWORD=${{ inputs.garasign_password }} - EOF - shell: bash - - - name: Log in to artifactory - if: ${{ inputs.skip_setup == 'false' }} - uses: redhat-actions/podman-login@v1 - with: - username: ${{ inputs.artifactory_username }} - password: ${{ inputs.artifactory_password }} - registry: ${{ inputs.artifactory_registry }} - - - name: "Run git command" - run: | - podman run \ - --env-file=envfile \ - --rm \ - -v $(pwd):$(pwd) \ - -w $(pwd) \ - ${{ inputs.artifactory_registry }}/${{ inputs.artifactory_image }} \ - /bin/bash -c "gpgloader && ${{ inputs.command }}" - shell: bash diff --git a/garasign/gpg-sign/action.yml b/garasign/gpg-sign/action.yml deleted file mode 100644 index d9440e3..0000000 --- a/garasign/gpg-sign/action.yml +++ /dev/null @@ -1,58 +0,0 @@ -name: "Sign artifact using garasign" -description: "Signs a release artifact" -inputs: - filenames: - description: "File name(s) to sign, can be a glob pattern" - required: true - garasign_username: - description: "Garasign username" - required: true - garasign_password: - description: "Garasign password" - required: true - artifactory_username: - description: "Artifactory user" - required: true - artifactory_password: - description: "Artifactory password" - required: true - artifactory_image: - description: "Image to use for artifactory" - default: release-tools-container-registry-local/garasign-gpg - artifactory_registry: - description: "Artifactory registry to be used" - default: artifactory.corp.mongodb.com - skip_setup: - description: "Whether to skip setup" - default: "false" - -runs: - using: composite - steps: - - name: Create the envfile - if: ${{ inputs.skip_setup == 'false' }} - run: | - cat << EOF > envfile - GRS_CONFIG_USER1_USERNAME=${{ inputs.garasign_username }} - GRS_CONFIG_USER1_PASSWORD=${{ inputs.garasign_password }} - EOF - shell: bash - - - name: Log in to artifactory - if: ${{ inputs.skip_setup == 'false' }} - uses: redhat-actions/podman-login@v1 - with: - username: ${{ inputs.artifactory_username }} - password: ${{ inputs.artifactory_password }} - registry: ${{ inputs.artifactory_registry }} - - - name: "Create detached signature for file" - run: | - podman run \ - --env-file=envfile \ - --rm \ - -v $(pwd):$(pwd) \ - -w $(pwd) \ - ${{ inputs.artifactory_registry }}/${{ inputs.artifactory_image }} \ - /bin/bash -c 'gpgloader && for filename in ${{ inputs.filenames }}; do gpg --detach-sign --armor --output ${filename}.sig ${filename}; done' - shell: bash diff --git a/git-sign/action.yml b/git-sign/action.yml new file mode 100644 index 0000000..5ca4b03 --- /dev/null +++ b/git-sign/action.yml @@ -0,0 +1,23 @@ +name: "Run git actions in a signing container" +description: "Allows running arbitrary git actions in a container with GPG keys loaded" +inputs: + command: + description: "Command to run inside the container" + required: true + artifactory_image: + description: "Image to use for artifactory" + default: release-tools-container-registry-local/garasign-git + +runs: + using: composite + steps: + - name: "Run git command" + run: | + podman run \ + --env-file=$GARASIGN_ENVFILE \ + --rm \ + -v $(pwd):$(pwd) \ + -w $(pwd) \ + ${ARTIFACTORY_REGISTRY}/${{ inputs.artifactory_image }} \ + /bin/bash -c "gpgloader && ${{ inputs.command }}" + shell: bash diff --git a/gpg-sign/action.yml b/gpg-sign/action.yml new file mode 100644 index 0000000..fb6ac01 --- /dev/null +++ b/gpg-sign/action.yml @@ -0,0 +1,28 @@ +name: "Sign artifact(s) using garasign" +description: "Signs release artifact(s)" +inputs: + filenames: + description: "File name(s) to sign, can be a glob pattern" + required: true + artifactory_image: + description: "Image to use for artifactory" + default: release-tools-container-registry-local/garasign-gpg + +runs: + using: composite + steps: + - name: "Create detached signature for file" + shell: bash + run: | + podman run \ + --env-file=$GARASIGN_ENVFILE \ + --rm \ + -v $(pwd):$(pwd) \ + -w $(pwd) \ + ${ARTIFACTORY_REGISTRY}/${{ inputs.artifactory_image }} \ + /bin/bash -c 'gpgloader && for filename in ${{ inputs.filenames }}; do gpg --detach-sign --armor --output ${filename}.sig ${filename}; done' + + - name: "Move the signature files to the release directory" + shell: bash + run: | + mv ${{inputs.filenames}}.sig $RELEASE_ASSETS diff --git a/papertrail/action.yml b/papertrail/action.yml deleted file mode 100644 index 9df1888..0000000 --- a/papertrail/action.yml +++ /dev/null @@ -1,38 +0,0 @@ -name: "Papertrail Report" -description: "Generate report for authorized publication on distribution channels" -inputs: - product_name: - description: "Name of product" - required: true - release_version: - description: "The release version" - required: true - filenames: - description: "Artifact filename(s) to include in the report, can be a glob pattern" - required: true - token: - description: "The GitHub token for the action" - required: true - output: - description: "The output filename" - default: "papertrail.txt" - -runs: - using: composite - steps: - - name: "Prepare report" - shell: bash - run: | - export GH_TOKEN=${{ inputs.token }} - NAME=$(gh api users/${{ github.actor }} --jq '.name') - export PAPERTRAIL="${{ inputs.output }}" - echo "Product: ${{ inputs.product_name }}" > $PAPERTRAIL - echo "Version: ${{ inputs.release_version }}" >> $PAPERTRAIL - echo "Releaser: $NAME" >> $PAPERTRAIL - echo "Build Source: GitHub Actions" - echo "Build Number: ${{ github.run_id }}" - for filename in ${{ inputs.filenames }}; do - SHA=$(shasum -a 256 $filename | awk '{print $1;}') - echo "Filename: $filename" >> $PAPERTRAIL - echo "Shasum: $SHA" >> $PAPERTRAIL - done \ No newline at end of file diff --git a/python/bump-and-tag/action.yml b/python/bump-and-tag/action.yml new file mode 100644 index 0000000..477045c --- /dev/null +++ b/python/bump-and-tag/action.yml @@ -0,0 +1,50 @@ +name: Python Bump and Tag +description: Perform bump and tag operations for Python Libraries +inputs: + version: + description: "The published version" + required: true + post_version: + description: "The post version" + required: true + version_bump_script: + description: "The version bump script" + required: true + dry_run: + description: "Whether this is a dry run" + required: true + +runs: + using: composite + steps: + - uses: actions/setup-python@v5 + with: + python-version: '3.11' + - name: Set new version + shell: bash -eux {0} + run: | + bash ${{ inputs.version_bump_script }} ${{ inputs.version }} + - name: Commit the version bump + uses: mongodb-labs/drivers-github-tools/git-sign@v2 + with: + command: git commit -a -m \"BUMP ${{ inputs.version }}\" -s --gpg-sign=${{ env.GPG_KEY_ID }} + - name: Tag the version + uses: mongodb-labs/drivers-github-tools/git-sign@v2 + with: + command: git tag -a \"${{ inputs.version }}\" -m \"BUMP ${{ inputs.version }}\" -s --local-user=${{ env.GPG_KEY_ID }} + - name: Verify the tag + shell: bash -eux {0} + run: | + curl $GPG_PUBLIC_URL --output /tmp/signature.pub + gpg --import /tmp/signature.pub + git verify-tag ${{inputs.version}} + - name: Push the commit and tag to the source branch + shell: bash -eux {0} + run: | + if [ ${{ inputs.dry_run }} != "true" ]; then + git push origin + git push origin --tags + echo "### Created tag: ${{inputs.version}}" >> $GITHUB_STEP_SUMMARY + else + echo "### Dry run for version: ${{inputs.version}}" >> $GITHUB_STEP_SUMMARY + fi diff --git a/python/publish/action.yml b/python/publish/action.yml new file mode 100644 index 0000000..ea8d0cc --- /dev/null +++ b/python/publish/action.yml @@ -0,0 +1,73 @@ + +name: Publish Python +description: "Publish Assets and Report" +inputs: + version: + description: "The published version" + required: true + following_version: + description: "The following (dev) version" + required: true + version_bump_script: + description: "The version bump script" + required: true + product_name: + description: "The name of the product" + required: true + token: + description: "The GitHub access token" + required: true + dry_run: + description: "Whether this is a dry run" + required: true + +runs: + using: composite + steps: + - uses: actions/setup-python@v5 + with: + python-version: '3.11' + - name: Download all the dists + uses: actions/download-artifact@v4 + with: + name: all-dist-${{ github.run_id }} + path: dist/ + - name: Create detached signature for dist files + uses: mongodb-labs/drivers-github-tools/gpg-sign@v2 + with: + filenames: dist/* + - uses: mongodb-labs/drivers-github-tools/authorized-pub@v2 + with: + product_name: ${{ inputs.product_name }} + release_version: ${{ inputs.version }} + filenames: dist/* + token: ${{ inputs.token }} + - name: Run publish script + shell: bash + run: ${{github.action_path}}/publish.sh + env: + GH_TOKEN: ${{ inputs.token }} + VERSION: ${{ inputs.version }} + PRODUCT_NAME: ${{ inputs.product_name }} + DRY_RUN: ${{ inputs.dry_run }} + # https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/#publishing-the-distribution-to-pypi + - name: Publish distribution 📦 to PyPI + if: inputs.dry_run == 'false' + uses: pypa/gh-action-pypi-publish@release/v1 + - name: Set following version + shell: bash -eux {0} + run: | + git clean -dffx + bash ${{ inputs.version_bump_script }} ${{ inputs.version }} + - name: Commit the version bump + uses: mongodb-labs/drivers-github-tools/git-sign@v2 + with: + command: git commit -a -m \"BUMP ${{ inputs.following_version }}\" -s --gpg-sign=${{ env.GPG_KEY_ID }} + - name: Push the commit to the source branch + shell: bash -eux {0} + run: | + if [ ${{ inputs.dry_run }} != "true" ]; then + git push origin --tags + else + echo "Not pushing the following_version tag since it is a dry run" + fi diff --git a/python/publish/publish.sh b/python/publish/publish.sh new file mode 100755 index 0000000..c25e0bf --- /dev/null +++ b/python/publish/publish.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash + +set -eux + +if [ "$DRY_RUN" == "false" ]; then + echo "Uploading Release Reports" + TARGET=s3://${AWS_BUCKET}/${PRODUCT_NAME}/${VERSION} + aws s3 cp $S3_ASSETS $TARGET --recursive + + echo "Creating draft release with attached files" + gh release create ${VERSION} --draft --verify-tag --title ${VERSION} --notes "" + gh release upload ${VERSION} $RELEASE_ASSETS/*.* + gh release view ${VERSION} >> $GITHUB_STEP_SUMMARY +else + echo "Dry run, not uploading to S3 or creating GitHub Release" + ls -ltr $RELEASE_ASSETS + ls -ltr $S3_ASSETS +fi diff --git a/setup/action.yml b/setup/action.yml new file mode 100644 index 0000000..cd812b5 --- /dev/null +++ b/setup/action.yml @@ -0,0 +1,39 @@ +name: Setup +description: "Set up the Release Environment" +inputs: + aws_role_arn: + description: "The aws role to acquire" + required: true + aws_region_name: + description: "The aws region to use" + required: true + aws_secret_id: + description: "The name of the aws secret to use" + required: true + artifactory_registry: + description: "Artifactory registry to be used" + default: artifactory.corp.mongodb.com + +runs: + using: composite + steps: + - name: configure aws credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + role-to-assume: ${{ inputs.aws_role_arn }} + role-session-name: release-session + aws-region: ${{ inputs.aws_region_name }} + - name: Read secrets from AWS Secrets Manager into environment variables + uses: aws-actions/aws-secretsmanager-get-secrets@v2 + with: + secret-ids: | + ${{ inputs.aws_secret_id }} + parse-json-secrets: true + - name: Set up + shell: bash + id: setup + run: ${{ github.action_path }}/setup.sh + env: + ARTIFACTORY_REGISTRY: ${{ inputs.artifactory_registry }} + ARTIFACTORY_IMAGE: ${{ inputs.artifactory_image }} + AWS_SECRET_ID: ${{ inputs.aws_secret_id }} diff --git a/setup/setup.sh b/setup/setup.sh new file mode 100755 index 0000000..75cabdc --- /dev/null +++ b/setup/setup.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +set -eu + +echo "Normalize secrets variable names" +prefix=$(echo $AWS_SECRET_ID | tr '[:lower:]' '[:upper:]' | sed -r 's/[-/]+/_/g') +prefix=${prefix}_ +vars=$(compgen -A variable | grep $prefix | tr '\n' ' ') +for var in $vars; do + new_key=$(echo $var | sed "s/$prefix//g") + declare $new_key=${!var} +done + +echo "::group::Set up artifactory" +echo $ARTIFACTORY_PASSWORD | podman login -u $ARTIFACTORY_USERNAME --password-stdin $ARTIFACTORY_REGISTRY +echo "::endgroup::" + +echo "Set up envfile for artifactory image" +GARASIGN_ENVFILE=/tmp/envfile +cat << EOF > $GARASIGN_ENVFILE +GRS_CONFIG_USER1_USERNAME=$GARASIGN_USERNAME +GRS_CONFIG_USER1_PASSWORD=$GARASIGN_PASSWORD +EOF + +echo "Set up output directories" +export RELEASE_ASSETS=/tmp/release-assets +mkdir $RELEASE_ASSETS +echo "$GITHUB_RUN_ID" > $RELEASE_ASSETS/release_run_id.txt +export S3_ASSETS=/tmp/s3-assets +mkdir $S3_ASSETS + +echo "Set up global variables" +cat <> $GITHUB_ENV +AWS_BUCKET=$RELEASE_ASSETS_BUCKET +GPG_KEY_ID=$GPG_KEY_ID +GPG_PUBLIC_URL=$GPG_PUBLIC_URL +GARASIGN_ENVFILE=$GARASIGN_ENVFILE +ARTIFACTORY_REGISTRY=$ARTIFACTORY_REGISTRY +RELEASE_ASSETS=$RELEASE_ASSETS +S3_ASSETS=$S3_ASSETS +EOF + +echo "Set up git credentials" +git config user.email "167856002+mongodb-dbx-release-bot[bot]@users.noreply.github.com" +git config user.name "mongodb-dbx-release-bot[bot]" \ No newline at end of file