Merge branch 'master' into CLOUDP-241732_gha_migration_guide

* master: fix: Removes default comment when creating resource `mongodbatlas_privatelink_endpoint_serverless` and adds update support to `mongodbatlas_privatelink_endpoint_service_serverless` (#2133) doc: Documents testing best practices (#2132) chore: Uses correct format when setting remote (#2147) chore: Configures git correctly for automatic commit (#2146) chore: Uses PAT of bot to commit changelog updates (#2144) doc: Adjust RELEASING.md documentation with new process (#2142) fix: Fixes nil pointer dereference if `advanced_configuration` update fails in `mongodbatlas_cluster` (#2139) chore: Adds a new step in release process that marks Jira version as released (#2136) chore: Sends Slack notification when changelog update fails (#2140) chore: Adds new step in release process for updating header of changelog (#2134)
mongodb · Apr 15, 2024 · 44ddd47 · 44ddd47
2 parents b51d658 + 2c63f39
commit 44ddd47
Show file tree

Hide file tree

Showing 21 changed files with 583 additions and 87 deletions.
diff --git a/.changelog/2133.txt b/.changelog/2133.txt
@@ -0,0 +1,6 @@
+```release-note:bug
+resource/mongodbatlas_privatelink_endpoint_serverless: Removes setting default comment during create.
+```
+```release-note:enhancement
+resource/mongodbatlas_privatelink_endpoint_service_serverless: Adds support for updating `comment` attribute in-place.
+```
diff --git a/.changelog/2139.txt b/.changelog/2139.txt
@@ -0,0 +1,3 @@
+```release-note:bug
+resource/mongodbatlas_cluster: Fixes nil pointer dereference if `advanced_configuration` update fails
+```
diff --git a/.github/workflows/acceptance-tests-runner.yml b/.github/workflows/acceptance-tests-runner.yml
@@ -419,6 +419,8 @@ jobs:
           terraform_wrapper: false    
       - name: Acceptance Tests
         env:
+          AWS_ACCESS_KEY_ID: ${{ secrets.aws_access_key_id }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.aws_secret_access_key }}
           ACCTEST_PACKAGES: |
             ./internal/service/serverlessinstance
             ./internal/service/privatelinkendpointserverless

diff --git a/.github/workflows/generate-changelog.yml b/.github/workflows/generate-changelog.yml
@@ -14,6 +14,7 @@ jobs:
       - uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
         with:
           fetch-depth: 0
+          persist-credentials: false
       - uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491
         with:
           go-version-file: 'go.mod'
@@ -27,7 +28,48 @@ jobs:
             fi
             git config --local user.email [email protected]
             git config --local user.name changelogbot
+            git remote set-url origin https://svc-apix-bot:${{ secrets.APIX_BOT_PAT }}@github.com/${{ github.repository }}
             git add CHANGELOG.md
             git commit -m "$MSG"
             git push
           fi
+
+  slack-notification:
+    needs: [generate-changelog]
+    if: ${{ !cancelled() && needs.generate-changelog.result == 'failure' }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Send Slack message
+        id: slack
+        uses: slackapi/slack-github-action@6c661ce58804a1a20f6dc5fbee7f0381b469e001
+        with:
+          payload: |
+            {
+              "text": "Automatic Changelog update failed",
+              "blocks": [
+                {
+                  "type": "section",
+                  "text": {
+                    "type": "mrkdwn",
+                    "text": "*Automatic Changelog update failed* ${{ secrets.SLACK_ONCALL_TAG }}"
+                  }
+                },
+                {
+                  "type": "actions",
+                  "elements": [
+                    {
+                        "type": "button",
+                        "text": {
+                            "type": "plain_text",
+                            "text": ":github: Failed action"
+                        },
+                        "url": "${{github.server_url}}/${{github.repository}}/actions/runs/${{github.run_id}}"
+                    }
+                ]
+                }
+              ]
+            }
+        env:
+          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
+          SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK
+
diff --git a/.github/workflows/migration-tests.yml b/.github/workflows/migration-tests.yml
@@ -462,6 +462,8 @@ jobs:
           terraform_wrapper: false    
       - name: Migration Tests
         env:
+          AWS_ACCESS_KEY_ID: ${{ secrets.aws_access_key_id }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.aws_secret_access_key }}
           MONGODB_ATLAS_LAST_VERSION: ${{ needs.get-provider-version.outputs.provider_version }}
           ACCTEST_PACKAGES: |
             ./internal/service/serverlessinstance

diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -22,31 +22,66 @@ jobs:
         run: |
           echo "${{ inputs.version_number }}" | grep -P '^v\d+\.\d+\.\d+(-pre[A-Za-z0-9-]*)?$'  
 
-    # QA acceptance tests are skipped when explicit input parameter is used
   run-qa-acceptance-tests:
     needs: [ validate-version-input ]
+    # QA acceptance tests are skipped when explicit input parameter is used
+    # As this job may be skipped following jobs require using 'always()' to make sure they are still run
     if: needs.validate-version-input.result == 'success' && inputs.skip_tests != 'true'
     secrets: inherit
     uses: ./.github/workflows/acceptance-tests.yml
     with:
       atlas_cloud_env: "qa"
       ref: ${{ inputs.use_existing_tag == 'true' && inputs.version_number  || github.ref }}
 
-  release:
+  update-changelog-header:
     runs-on: ubuntu-latest
     needs: [ validate-version-input, run-qa-acceptance-tests ]
-    if: ${{ always() && needs.validate-version-input.result == 'success' && (needs.run-qa-acceptance-tests.result == 'skipped' || needs.run-qa-acceptance-tests.result == 'success') }}
+    # Skipped if use_existing_tag is defined, is a pre-release, or previous jobs failed.
+    if: >-
+      always() 
+      && inputs.use_existing_tag != 'true' 
+      && !contains(inputs.version_number, 'pre') 
+      && needs.validate-version-input.result == 'success' 
+      && (needs.run-qa-acceptance-tests.result == 'skipped' || needs.run-qa-acceptance-tests.result == 'success')
+    steps:
+      - uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
+        with:
+          fetch-depth: 0
+      - run: ./scripts/update-changelog-header-for-release.sh ${{inputs.version_number}}
+      - run: |
+          if [[ $(git status --porcelain) ]]; then
+            MSG="Update CHANGELOG.md header for ${{inputs.version_number}} release"
+            git config --local user.email [email protected]
+            git config --local user.name changelogbot
+            git add CHANGELOG.md
+            git commit -m "$MSG"
+            git push
+          fi
+    
+  release:
+    runs-on: ubuntu-latest
+    needs: [ validate-version-input, run-qa-acceptance-tests, update-changelog-header ]
+    # Release is skipped if there are failures in previous steps
+    if: >- 
+      always() 
+      && needs.validate-version-input.result == 'success' 
+      && (needs.run-qa-acceptance-tests.result == 'skipped' || needs.run-qa-acceptance-tests.result == 'success') 
+      && (needs.update-changelog-header.result == 'skipped' || needs.update-changelog-header.result == 'success')
     steps:
       - name: Checkout
         uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
         with:
-          ref: ${{ inputs.use_existing_tag == 'true' && inputs.version_number || github.ref }}
+          ref: ${{ inputs.use_existing_tag == 'true' && inputs.version_number || 'master' }}
       - name: Unshallow
         run: git fetch --prune --unshallow
+      - name: Get the latest commit SHA
+        id: get-sha
+        run: echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
       - name: Create release tag
         uses: rickstaa/action-create-tag@a1c7777fcb2fee4f19b0f283ba888afa11678b72
         with:
           tag: ${{ inputs.version_number }}
+          commit_sha: ${{ steps.get-sha.outputs.sha }}
           gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }}
           gpg_passphrase: ${{ secrets.PASSPHRASE }}
           tag_exists_error: ${{ inputs.use_existing_tag != 'true' }}
@@ -68,3 +103,17 @@ jobs:
         env:
           GPG_FINGERPRINT: ${{ steps.import_gpg.outputs.fingerprint }}
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+  jira-release-version:
+    if: ${{ !contains(inputs.version_number, 'pre') }}
+    needs: [ release ]
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
+    - uses: actions/setup-go@0c52d547c9bc32b1aa3301fd7a9cb496313a4491
+      with:
+        go-version-file: 'go.mod'
+    - run: make jira-release-version
+      env:
+        JIRA_API_TOKEN: ${{ secrets.JIRA_API_TOKEN }}
+        VERSION_NUMBER: ${{ inputs.version_number }}
diff --git a/.github_changelog_generator b/.github_changelog_generator
diff --git a/GNUmakefile b/GNUmakefile
@@ -146,3 +146,7 @@ generate-changelog-entry:
 .PHONY: check-changelog-entry-file
 check-changelog-entry-file:
 	go run ./tools/check-changelog-entry-file/*.go
+
+.PHONY: jira-release-version
+jira-release-version:
+	go run ./tools/jira-release-version/*.go
diff --git a/RELEASING.md b/RELEASING.md
@@ -1,54 +1,30 @@
 # Releasing
 
-## Prerequisites
-
-- [github-changelog-generator](https://github.com/github-changelog-generator/github-changelog-generator)
-
 ## Steps
 
-### Make sure that the acceptance tests are successful
-While QA acceptance tests are run in the release process automatically, it is advised to check [workflows/test-suite.yml](https://github.com/mongodb/terraform-provider-mongodbatlas/actions/workflows/test-suite.yml) and see if the latest run of the Test Suite action is successful (it runs every day at midnight UTC time). This can help detect failures before proceeding with the next steps.
-
-### Pre-release the provider 
-We pre-release the provider to make for testing purpose. **A Pre-release is not published to the Hashicorp Terraform Registry**.
-
-- Using our [Release GitHub Action](https://github.com/mongodb/terraform-provider-mongodbatlas/actions/workflows/release.yml) run a new workflow using `master` and the following inputs:
-  - Version number: vX.Y.Z-pre
-  - Skip QA acceptance tests: Should be left empty. Only used in case failing tests have been encountered in QA and the team agrees the release can still de done, or successful run of QA acceptance tests has already been done with the most recent changes.
-  - Using an existing tag: Should be left empty (default `false` creates a new tag from `master`). This should be set to `true` only if you want to re-use an existing tag for the release and the tag name should adhere to our version number format.
-
-- You will see the release in the [GitHub Release page](https://github.com/mongodb/terraform-provider-mongodbatlas/releases) once the [release action](.github/workflows/release.yml) has completed.
+### Revise jira release
+Before triggering a release, view the corresponding [unreleased jira page](https://jira.mongodb.org/projects/CLOUDP?selectedItem=com.atlassian.jira.jira-projects-plugin:release-page&status=unreleased&contains=terraform) to ensure there are no pending tickets. In case there are pending tickets, verify with the team if the expectation is to have them included within the current release. After release workflow is successful the version will be marked as released automatically.
 
-**Note**: If a failure is encountered during the go releaser step you must manually delete the created tag and then retry running the action.
-
-### Create PR updating Changelog and Upgrade Guide
+### Make sure that the acceptance tests are successful
+While QA acceptance tests are run in the release process automatically, we check [workflows/test-suite.yml](https://github.com/mongodb/terraform-provider-mongodbatlas/actions/workflows/test-suite.yml) and see if the latest run of the Test Suite action is successful (it runs every day at midnight UTC time). This can help detect failures before proceeding with the next steps.
 
-- Create a JIRA ticket and open a PR against the **master** branch. Make any manual adjustments if needed taking into account date format and format parameter names and resources/data source names if they begin with `mongodbatlas`.
-- Include the PM as a PR reviewer
-- Contact Documentation team in Slack to review the PR
-- Example: [#1478](https://github.com/mongodb/terraform-provider-mongodbatlas/pull/1478).
+### Verify upgrade guide is defined
 
-The PR includes the following changes:
+**Note**: Only applies if the right most version digit is 0 (considered a major or minor version in [semantic versioning](https://semver.org/)).
 
-#### Generate the CHANGELOG.md 
-We use a tool called [github changelog generator](https://github.com/github-changelog-generator/github-changelog-generator) to automatically update our changelog. It provides options for downloading a CLI or using a docker image with interactive mode to update the CHANGELOG.md file locally.
+- A doc in /website/docs/guides/X.Y.0-upgrade-guide.html must be defined containing a summary of the most significant features, breaking changes, and additional information that can be helpful. The expectation is that this file is created during relevant pull requests and not during the release process.
 
-- Update `since_tag` and `future-release` in [.github_changelog_generator](https://github.com/mongodb/terraform-provider-mongodbatlas/blob/master/.github_changelog_generator)
-- **There is a bug with `github_changelog_generator` ([#971](https://github.com/github-changelog-generator/github-changelog-generator/issues/971))**: Make sure to update the `future-tag` with the pre-release tag. Once you generate the changelog, update `future-tag` with the final release tag in [.github_changelog_generator](https://github.com/mongodb/terraform-provider-mongodbatlas/blob/master/.github_changelog_generator). Then, manually update the generated changelog to remove references to the pre-release tag
-- Run the following command: 
-    ```bash 
-    docker run -it --rm -v "$(pwd)":/usr/local/src/your-app githubchangeloggenerator/github-changelog-generator -u mongodb -p terraform-provider-mongodbatlas -t <GH_TOKEN> --breaking-labels "breaking-change" --enhancement-label "**Enhancements**" --bugs-label "**Bug Fixes**"  --issues-label "**Closed Issues**" --pr-label "**Internal Improvements**" --max-issues 1000
-    ```
-    To obtain your github personal access token you can use the following guide: [Authorizing a personal access token for use with SAML single sign-on](https://docs.github.com/en/enterprise-cloud@latest/authentication/authenticating-with-saml-single-sign-on/authorizing-a-personal-access-token-for-use-with-saml-single-sign-on)
-
-#### Define the Upgrade Guide
+### Trigger release workflow
 
-**Note**: Only applies if the right most version digit is 0 (considered a major or minor version in [semantic versioning](https://semver.org/)).
+- Using our [Release GitHub Action](https://github.com/mongodb/terraform-provider-mongodbatlas/actions/workflows/release.yml) run a new workflow using `master` and the following inputs:
+  - Version number: `vX.Y.Z`
+  - Skip QA acceptance tests: Should be left empty. Only used in case failing tests have been encountered in QA and the team agrees the release can still de done, or successful run of QA acceptance tests has already been done with the most recent changes.
+  - Using an existing tag: Should be left empty (default `false` creates a new tag from `master`). This should be set to `true` only if you want to re-use an existing tag for the release. This can be helpful for rerunning a failed release process in which the tag has already been created.
 
-- Create a new doc in /website/docs/guides/X.Y.0-upgrade-guide.html. This will contain a summary of the most significant features and breaking changes. Additional information that can be helpful to users can be defined here.
+#### How to create a pre-release (not part of regular process)
+Pre-releases are not needed for a regular release process, but they can be generated for exceptional cases (e.g. sharing with external teams for additional testing). The process is the same as a regular release except for the format of the version number which must be `vX.Y.Z-pre` with additional numbers at the end if needed. When a pre release is triggered, steps related to updating the changelog header or updating the jira release version are skipped.
 
-### Release the provider
-- Follow the same steps in the pre-release but provide the final release tag (example `v1.9.0`). Harshicorp has a process in place that will retrieve the latest release from the GitHub repository and add the binaries to the Hashicorp Terraform Registry.
+### Post-trigger checks
+- You will see the release in the [GitHub Release page](https://github.com/mongodb/terraform-provider-mongodbatlas/releases) once the [release action](.github/workflows/release.yml) has completed. HashiCorp has a process in place that will retrieve the latest release from the GitHub repository and add the binaries to the HashiCorp Terraform Registry (more details [here](https://developer.hashicorp.com/terraform/registry/providers/publishing#webhooks)).
 - **CDKTF Update - Only for major release, i.e. the left most version digit increment (see this [comment](https://github.com/cdktf/cdktf-repository-manager/pull/202#issuecomment-1602562201))**: Once the provider has been released, we need to update the provider version in our CDKTF. Raise a PR against [cdktf/cdktf-repository-manager](https://github.com/cdktf/cdktf-repository-manager).
   - Example PR: [#183](https://github.com/cdktf/cdktf-repository-manager/pull/183)
-
diff --git a/contributing/README.md b/contributing/README.md
@@ -3,7 +3,8 @@
 Thanks for your interest in contributing to MongoDB Atlas Terraform Provider, the following documents define guidelines necessary to participate in the community.
 
 - [Development Setup](development-setup.md)
-- [Code and Test Best Practices](development-best-practices.md)
+- [Development Best Practices](development-best-practices.md)
+- [Testing Best Practices](testing-best-practices.md)
 - [Documentation](documentation.md)
 - [Changelog process](changelog-process.md)
 - [Atlas SDK](atlas-sdk.md)
diff --git a/contributing/development-best-practices.md b/contributing/development-best-practices.md
@@ -1,5 +1,5 @@
 
-## Code and Test Best Practices
+## Development Best Practices
 
 ## Table of Contents
 - [Creating New Resource and Data Sources](#creating-new-resources-and-data-sources)
@@ -8,13 +8,6 @@
 
 - Each resource (and associated data sources) is in a package in `internal/service`.
 - There can be multiple helper files and they can also be used from other resources, e.g. `common_advanced_cluster.go` defines functions that are also used from other resources using `advancedcluster.FunctionName`.
-- Unit and Acceptances tests are in the same `_test.go` file. They are not in the same package as the code tests, e.g. `advancedcluster` tests are in `advancedcluster_test` package so coupling is minimized.
-- Migration tests are in `_migration_test.go` files.
-- Helper methods must have their own tests, e.g. `common_advanced_cluster_test.go` has tests for `common_advanced_cluster.go`.
-- `internal/testutils/acc` contains helper test methods for Acceptance and Migration tests.
-- Tests that need the provider binary like End-to-End tests don’t belong to the source code packages and go in `test/e2e`.
-- [Testify Mock](https://pkg.go.dev/github.com/stretchr/testify/mock) and [Mockery](https://github.com/vektra/mockery) are used for test doubles in Atlas Go SDK unit tests.
-
 
 ### Creating New Resource and Data Sources