Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Updates migration tests to run separately and use last released version of provider for plan checks #1565

Merged
merged 14 commits into from
Oct 31, 2023
12 changes: 11 additions & 1 deletion .github/workflows/code-health.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ jobs:
- uses: actions/checkout@v4
- name: Run ShellCheck
uses: bewuethr/shellcheck-action@v2
call-accettance-tests-workflow:
call-acceptance-tests-workflow:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

finally, thanks! :-)

needs: [build, lint, shellcheck, unit-test, website-lint]
secrets: inherit
permissions:
Expand All @@ -98,4 +98,14 @@ jobs:
uses: ./.github/workflows/acceptance-tests.yml
with:
parent-event-name: ${{ github.event_name }}
call-migration-tests-workflow:
needs: [build, lint, shellcheck, unit-test, website-lint]
secrets: inherit
permissions:
contents: write
pull-requests: read
repository-projects: read
uses: ./.github/workflows/migration-tests.yml
with:
parent-event-name: ${{ github.event_name }}

111 changes: 111 additions & 0 deletions .github/workflows/migration-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
name: 'Migration Tests'

on:
workflow_dispatch: {} # workflow can be run manually
workflow_call: # workflow runs after code-health
inputs:
parent-event-name:
required: true
type: string
pull_request: # you can run a specic job in your PR using GitHub labels
maastha marked this conversation as resolved.
Show resolved Hide resolved
types: [ labeled ]

jobs:
change-detection:
runs-on: ubuntu-latest
permissions:
pull-requests: read
repository-projects: read
outputs:
project: ${{ steps.filter.outputs.project }}
config: ${{ steps.filter.outputs.config }}
steps:
- uses: actions/checkout@v4
if: ${{ github.event_name == 'push' || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule' || inputs.parent-event-name == 'release' }}
- uses: dorny/paths-filter@v2
id: filter
with:
filters: |
project:
- 'mongodbatlas/fw_data_source_mongodbatlas_project_ip_access_list*.go'
- 'mongodbatlas/fw_resource_mongodbatlas_project_ip_access_list*.go'
config:
- 'mongodbatlas/fw_data_source_mongodbatlas_alert_configuration*.go'
- 'mongodbatlas/fw_data_source_mongodbatlas_database_user*.go'
- 'mongodbatlas/fw_data_source_mongodbatlas_atlas_user*.go'
- 'mongodbatlas/fw_resource_mongodbatlas_alert_configuration*.go'
- 'mongodbatlas/fw_resource_mongodbatlas_database_user*.go'

project:
needs: [ change-detection ]
if: ${{ needs.change-detection.outputs.project == 'true' || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule' || github.event.label.name == 'run-testacc-migration' || github.event.label.name == 'run-testacc-project-migration' || inputs.parent-event-name == 'release' }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i know we have this in other gh actions but I wonder if we could make this easier or have some functions or something

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps we could start fixing this file and then a follow-up to change the others?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lantoli I couldn't find a way to make this reusable or use a function, is there any example or reference you could share?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about the best way, but for instance what about creating a variable in the workflow and use the variable? something like:

on: ...

env:
hasToTrigger: ${{ needs.change-detection.outputs.config == 'true' || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule' || github.event.label.name == 'run-testacc-migration' || github.event.label.name == 'run-testacc-config-migration' || inputs.parent-event-name == 'release' }}

job...

and use:

if: ${{ env.hasToTrigger }}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or use it for the common part and then:
if: ${{ env.hasToTrigger }} || specific conditions here...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this doesn't hold a static value, rather holds a condition, i don't believe this is possible to do via a variable as well

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i suppose it's possible while you use variables that are know before starting execution

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed, see comment #1565 (comment)

runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Get Last Release
id: get_last_release
run: |
LAST_RELEASE=$(curl -sSfL -X GET https://api.github.com/repos/mongodb/terraform-provider-mongodbatlas/releases/latest | jq -r '.tag_name | ltrimstr("v")')
echo "Last release: $LAST_RELEASE"
echo "MONGODB_ATLAS_LAST_VERSION=$LAST_RELEASE" >> $GITHUB_ENV
shell: bash
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version-file: 'go.mod'
- name: Migration Tests
env:
MONGODB_ATLAS_PUBLIC_KEY: ${{ secrets.MONGODB_ATLAS_PUBLIC_KEY_CLOUD_DEV }}
MONGODB_ATLAS_PRIVATE_KEY: ${{ secrets.MONGODB_ATLAS_PRIVATE_KEY_CLOUD_DEV }}
MONGODB_ATLAS_ORG_ID: ${{ vars.MONGODB_ATLAS_ORG_ID_CLOUD_DEV }}
MONGODB_ATLAS_BASE_URL: ${{ vars.MONGODB_ATLAS_BASE_URL }}
MONGODB_ATLAS_PROJECT_OWNER_ID: ${{ vars.MONGODB_ATLAS_PROJECT_OWNER_ID }}
MONGODB_ATLAS_API_KEYS_IDS: ${{ vars.MONGODB_ATLAS_API_KEYS_IDS }}
MONGODB_ATLAS_TEAMS_IDS: ${{ vars.MONGODB_ATLAS_TEAMS_IDS }}
SKIP_TEST_EXTERNAL_CREDENTIALS: ${{ vars.SKIP_TEST_EXTERNAL_CREDENTIALS }}
ACCTEST_TIMEOUT: ${{ vars.ACCTEST_TIMEOUT }}
TF_LOG: ${{ vars.LOG_LEVEL }}
TF_ACC: 1
PARALLEL_GO_TEST: 20
CI: true
TEST_REGEX: "^TestAccMigrationProject"
run: make testacc

config:
needs: [ change-detection ]
if: ${{ needs.change-detection.outputs.config == 'true' || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule' || github.event.label.name == 'run-testacc-migration' || github.event.label.name == 'run-testacc-config-migration' || inputs.parent-event-name == 'release' }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Get Last Release
id: get_last_release
run: |
LAST_RELEASE=$(curl -sSfL -X GET https://api.github.com/repos/mongodb/terraform-provider-mongodbatlas/releases/latest | jq -r '.tag_name | ltrimstr("v")')
echo "Last release: $LAST_RELEASE"
echo "MONGODB_ATLAS_LAST_VERSION=$LAST_RELEASE" >> $GITHUB_ENV
shell: bash
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version-file: 'go.mod'
- name: Migration Tests
env:
MONGODB_ATLAS_PUBLIC_KEY: ${{ secrets.MONGODB_ATLAS_PUBLIC_KEY_CLOUD_DEV_NETWORK }}
MONGODB_ATLAS_PRIVATE_KEY: ${{ secrets.MONGODB_ATLAS_PRIVATE_KEY_CLOUD_DEV_NETWORK }}
MONGODB_ATLAS_ORG_ID: ${{ vars.MONGODB_ATLAS_ORG_ID_CLOUD_DEV_NETWORK }}
MONGODB_ATLAS_BASE_URL: ${{ vars.MONGODB_ATLAS_BASE_URL }}
MONGODB_ATLAS_PROJECT_OWNER_ID: ${{ vars.MONGODB_ATLAS_PROJECT_OWNER_ID }}
SKIP_TEST_EXTERNAL_CREDENTIALS: ${{ vars.SKIP_TEST_EXTERNAL_CREDENTIALS }}
MONGODB_ATLAS_USERNAME_CLOUD_DEV: ${{ vars.MONGODB_ATLAS_USERNAME_CLOUD_DEV }}
AZURE_ATLAS_APP_ID: ${{vars.AZURE_ATLAS_APP_ID}}
AZURE_SERVICE_PRINCIPAL_ID: ${{vars.AZURE_SERVICE_PRINCIPAL_ID}}
AZURE_TENANT_ID: ${{vars.AZURE_TENANT_ID}}
ACCTEST_TIMEOUT: ${{ vars.ACCTEST_TIMEOUT }}
TF_LOG: ${{ vars.LOG_LEVEL }}
TF_ACC: 1
PARALLEL_GO_TEST: 20
CI: true
TEST_REGEX: "^TestAccMigrationConfig"
run: make testacc
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,21 @@ import (
"github.com/hashicorp/terraform-plugin-testing/plancheck"
)

func TestAccProjectDSProjectIPAccessList_Migration_SettingIPAddress(t *testing.T) {
func TestAccMigrationProjectDSProjectIPAccessList_SettingIPAddress(t *testing.T) {
dataSourceName := "data.mongodbatlas_project_ip_access_list.test"
orgID := os.Getenv("MONGODB_ATLAS_ORG_ID")
projectName := acctest.RandomWithPrefix("test-acc")
ipAddress := fmt.Sprintf("179.154.226.%d", acctest.RandIntRange(0, 255))
comment := fmt.Sprintf("TestAcc for ipAddress (%s)", ipAddress)
lastVersionConstraint := os.Getenv("MONGODB_ATLAS_LAST_VERSION")

resource.Test(t, resource.TestCase{
PreCheck: func() { testAccPreCheckBasic(t) },
Steps: []resource.TestStep{
{
ExternalProviders: map[string]resource.ExternalProvider{
"mongodbatlas": {
VersionConstraint: "1.11.0",
VersionConstraint: lastVersionConstraint,
Source: "mongodb/mongodbatlas",
},
},
Expand All @@ -50,20 +51,21 @@ func TestAccProjectDSProjectIPAccessList_Migration_SettingIPAddress(t *testing.T
})
}

func TestAccProjectDSProjectIPAccessList_Migration_SettingCIDRBlock(t *testing.T) {
func TestAccMigrationProjectDSProjectIPAccessList_SettingCIDRBlock(t *testing.T) {
dataSourceName := "data.mongodbatlas_project_ip_access_list.test"
orgID := os.Getenv("MONGODB_ATLAS_ORG_ID")
projectName := acctest.RandomWithPrefix("test-acc")
cidrBlock := fmt.Sprintf("179.154.226.%d/32", acctest.RandIntRange(0, 255))
comment := fmt.Sprintf("TestAcc for cidrBlock (%s)", cidrBlock)
lastVersionConstraint := os.Getenv("MONGODB_ATLAS_LAST_VERSION")

resource.Test(t, resource.TestCase{
PreCheck: func() { testAccPreCheckBasic(t) },
Steps: []resource.TestStep{
{
ExternalProviders: map[string]resource.ExternalProvider{
"mongodbatlas": {
VersionConstraint: "1.11.0",
VersionConstraint: lastVersionConstraint,
Source: "mongodb/mongodbatlas",
},
},
Expand All @@ -90,7 +92,7 @@ func TestAccProjectDSProjectIPAccessList_Migration_SettingCIDRBlock(t *testing.T
})
}

func TestAccProjectDSProjectIPAccessList_Migration_SettingAWSSecurityGroup(t *testing.T) {
func TestAccMigrationProjectDSProjectIPAccessList_SettingAWSSecurityGroup(t *testing.T) {
SkipTestExtCred(t)
dataSourceName := "data.mongodbatlas_project_ip_access_list.test"
vpcID := os.Getenv("AWS_VPC_ID")
Expand All @@ -102,14 +104,15 @@ func TestAccProjectDSProjectIPAccessList_Migration_SettingAWSSecurityGroup(t *te
projectID := os.Getenv("MONGODB_ATLAS_PROJECT_ID")
awsSGroup := os.Getenv("AWS_SECURITY_GROUP_ID")
comment := fmt.Sprintf("TestAcc for awsSecurityGroup (%s)", awsSGroup)
lastVersionConstraint := os.Getenv("MONGODB_ATLAS_LAST_VERSION")

resource.Test(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
Steps: []resource.TestStep{
{
ExternalProviders: map[string]resource.ExternalProvider{
"mongodbatlas": {
VersionConstraint: "1.11.0",
VersionConstraint: lastVersionConstraint,
Source: "mongodb/mongodbatlas",
},
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,21 @@ import (
"os"
"testing"

matlas "go.mongodb.org/atlas/mongodbatlas"

"github.com/hashicorp/terraform-plugin-testing/helper/acctest"
"github.com/hashicorp/terraform-plugin-testing/helper/resource"
"github.com/hashicorp/terraform-plugin-testing/plancheck"
matlas "go.mongodb.org/atlas/mongodbatlas"
)

func TestAccConfigRSAlertConfiguration_Migration_NotificationsWithMetricThreshold(t *testing.T) {
func TestAccMigrationConfigRSAlertConfiguration_NotificationsWithMetricThreshold(t *testing.T) {
var (
resourceName = "mongodbatlas_alert_configuration.test"
orgID = os.Getenv("MONGODB_ATLAS_ORG_ID")
projectName = acctest.RandomWithPrefix("test-acc")
alert = &matlas.AlertConfiguration{}
config = testAccMongoDBAtlasAlertConfigurationConfig(orgID, projectName, true)
resourceName = "mongodbatlas_alert_configuration.test"
orgID = os.Getenv("MONGODB_ATLAS_ORG_ID")
projectName = acctest.RandomWithPrefix("test-acc")
alert = &matlas.AlertConfiguration{}
config = testAccMongoDBAtlasAlertConfigurationConfig(orgID, projectName, true)
lastVersionConstraint = os.Getenv("MONGODB_ATLAS_LAST_VERSION")
)

resource.ParallelTest(t, resource.TestCase{
Expand All @@ -27,7 +29,7 @@ func TestAccConfigRSAlertConfiguration_Migration_NotificationsWithMetricThreshol
{
ExternalProviders: map[string]resource.ExternalProvider{
"mongodbatlas": {
VersionConstraint: "1.11.0",
VersionConstraint: lastVersionConstraint,
Source: "mongodb/mongodbatlas",
},
},
Expand All @@ -52,13 +54,14 @@ func TestAccConfigRSAlertConfiguration_Migration_NotificationsWithMetricThreshol
})
}

func TestAccConfigRSAlertConfiguration_Migration_WithThreshold(t *testing.T) {
func TestAccMigrationConfigRSAlertConfiguration_WithThreshold(t *testing.T) {
var (
resourceName = "mongodbatlas_alert_configuration.test"
orgID = os.Getenv("MONGODB_ATLAS_ORG_ID")
projectName = acctest.RandomWithPrefix("test-acc")
alert = &matlas.AlertConfiguration{}
config = testAccMongoDBAtlasAlertConfigurationConfigWithThresholdUpdated(orgID, projectName, true, 1)
resourceName = "mongodbatlas_alert_configuration.test"
orgID = os.Getenv("MONGODB_ATLAS_ORG_ID")
projectName = acctest.RandomWithPrefix("test-acc")
alert = &matlas.AlertConfiguration{}
config = testAccMongoDBAtlasAlertConfigurationConfigWithThresholdUpdated(orgID, projectName, true, 1)
lastVersionConstraint = os.Getenv("MONGODB_ATLAS_LAST_VERSION")
)

resource.ParallelTest(t, resource.TestCase{
Expand All @@ -68,7 +71,7 @@ func TestAccConfigRSAlertConfiguration_Migration_WithThreshold(t *testing.T) {
{
ExternalProviders: map[string]resource.ExternalProvider{
"mongodbatlas": {
VersionConstraint: "1.11.0",
VersionConstraint: lastVersionConstraint,
Source: "mongodb/mongodbatlas",
},
},
Expand All @@ -95,13 +98,14 @@ func TestAccConfigRSAlertConfiguration_Migration_WithThreshold(t *testing.T) {
})
}

func TestAccConfigRSAlertConfiguration_Migration_EmptyOptionalBlocks(t *testing.T) {
func TestAccMigrationConfigRSAlertConfiguration_EmptyOptionalBlocks(t *testing.T) {
var (
resourceName = "mongodbatlas_alert_configuration.test"
orgID = os.Getenv("MONGODB_ATLAS_ORG_ID")
projectName = acctest.RandomWithPrefix("test-acc")
alert = &matlas.AlertConfiguration{}
config = testAccMongoDBAtlasAlertConfigurationConfigEmptyOptionalBlocks(orgID, projectName)
resourceName = "mongodbatlas_alert_configuration.test"
orgID = os.Getenv("MONGODB_ATLAS_ORG_ID")
projectName = acctest.RandomWithPrefix("test-acc")
alert = &matlas.AlertConfiguration{}
config = testAccMongoDBAtlasAlertConfigurationConfigEmptyOptionalBlocks(orgID, projectName)
lastVersionConstraint = os.Getenv("MONGODB_ATLAS_LAST_VERSION")
)

resource.ParallelTest(t, resource.TestCase{
Expand All @@ -111,7 +115,7 @@ func TestAccConfigRSAlertConfiguration_Migration_EmptyOptionalBlocks(t *testing.
{
ExternalProviders: map[string]resource.ExternalProvider{
"mongodbatlas": {
VersionConstraint: "1.11.0",
VersionConstraint: lastVersionConstraint,
Source: "mongodb/mongodbatlas",
},
},
Expand Down Expand Up @@ -139,7 +143,7 @@ func TestAccConfigRSAlertConfiguration_Migration_EmptyOptionalBlocks(t *testing.
})
}

func TestAccConfigRSAlertConfiguration_Migration_MultipleMatchers(t *testing.T) {
func TestAccMigrationConfigRSAlertConfiguration_MultipleMatchers(t *testing.T) {
var (
resourceName = "mongodbatlas_alert_configuration.test"
orgID = os.Getenv("MONGODB_ATLAS_ORG_ID")
Expand All @@ -156,6 +160,7 @@ func TestAccConfigRSAlertConfiguration_Migration_MultipleMatchers(t *testing.T)
Operator: "CONTAINS",
Value: "MONGOS",
})
lastVersionConstraint = os.Getenv("MONGODB_ATLAS_LAST_VERSION")
)

resource.ParallelTest(t, resource.TestCase{
Expand All @@ -165,7 +170,7 @@ func TestAccConfigRSAlertConfiguration_Migration_MultipleMatchers(t *testing.T)
{
ExternalProviders: map[string]resource.ExternalProvider{
"mongodbatlas": {
VersionConstraint: "1.11.0",
VersionConstraint: lastVersionConstraint,
Source: "mongodb/mongodbatlas",
},
},
Expand All @@ -190,13 +195,14 @@ func TestAccConfigRSAlertConfiguration_Migration_MultipleMatchers(t *testing.T)
})
}

func TestAccConfigRSAlertConfiguration_Migration_EmptyOptionalAttributes(t *testing.T) {
func TestAccMigrationConfigRSAlertConfiguration_EmptyOptionalAttributes(t *testing.T) {
var (
resourceName = "mongodbatlas_alert_configuration.test"
orgID = os.Getenv("MONGODB_ATLAS_ORG_ID")
projectName = acctest.RandomWithPrefix("test-acc")
alert = &matlas.AlertConfiguration{}
config = testAccMongoDBAtlasAlertConfigurationConfigWithEmptyOptionalAttributes(orgID, projectName)
resourceName = "mongodbatlas_alert_configuration.test"
orgID = os.Getenv("MONGODB_ATLAS_ORG_ID")
projectName = acctest.RandomWithPrefix("test-acc")
alert = &matlas.AlertConfiguration{}
config = testAccMongoDBAtlasAlertConfigurationConfigWithEmptyOptionalAttributes(orgID, projectName)
lastVersionConstraint = os.Getenv("MONGODB_ATLAS_LAST_VERSION")
)

resource.ParallelTest(t, resource.TestCase{
Expand All @@ -206,7 +212,7 @@ func TestAccConfigRSAlertConfiguration_Migration_EmptyOptionalAttributes(t *test
{
ExternalProviders: map[string]resource.ExternalProvider{
"mongodbatlas": {
VersionConstraint: "1.11.0",
VersionConstraint: lastVersionConstraint,
Source: "mongodb/mongodbatlas",
},
},
Expand Down
Loading
Loading