Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tfversion: Add SkipIfNotAlpha version check #388

Merged
merged 3 commits into from
Nov 15, 2024
Merged

tfversion: Add SkipIfNotAlpha version check #388

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e7c7ad9
tfversion: Add `SkipIfNotAlpha` version check
austinvalle Nov 8, 2024
09127ed
changelog
austinvalle Nov 8, 2024
f0de8a0
missing var declaration
austinvalle Nov 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changes/unreleased/FEATURES-20241108-160634.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: FEATURES
body: 'tfversion: Added `SkipIfNotAlpha` version check for testing experimental features
of alpha Terraform builds.'
time: 2024-11-08T16:06:34.662822-05:00
custom:
Issue: "388"
10 changes: 10 additions & 0 deletions ...r/resource/testdata/TestTest_ConfigDirectory_TestNameDirectory_MultipleFiles_Vars/vars.tf
View file
Open in desktop
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After updating our CI to Terraform 1.10.0-rc1, I found an invalid test setup due to a new error from the introduction of ephemeral variables in the apply command, which rejects variables not defined in configuration during apply.

Example (names are different because I was testing it locally)

│ Error: Can't set variable when applying a saved plan
│ 
│   on test.auto.tfvars.json line 3:
│    3:     "var_two": false
│ 
│ The variable var_two cannot be set using the -var and -var-file options when applying a saved plan file, because it is neither ephemeral nor has it been declared during the plan operation.
│ To declare an ephemeral variable which is not saved in the plan file, use ephemeral = true.

I don't think this is a bug in Terraform core necessarily, since it already warns users about this misconfiguration, for example, during plan:

│ Error: Can't set variable when applying a saved plan
│ 
│   on test.auto.tfvars.json line 3:
│    3:     "var_two": false
│ 
│ The variable var_two cannot be set using the -var and -var-file options when applying a saved plan file, because it is neither ephemeral nor has it been declared during the plan operation.
│ To declare an ephemeral variable which is not saved in the plan file, use ephemeral = true.

I think in our case here, the test was simply not setup properly, and the new version of Terraform is confirming that. So the fix here is just correctly declaring the variables in the configuration 👍🏻

Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Copyright (c) HashiCorp, Inc.
# SPDX-License-Identifier: MPL-2.0

variable "length" {
type = number
}

variable "numeric" {
type = bool
}
6 changes: 3 additions & 3 deletions plancheck/expect_deferred_change_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ func Test_ExpectDeferredChange_Reason_Match(t *testing.T) {
r.Test(t, r.TestCase{
TerraformVersionChecks: []tfversion.TerraformVersionCheck{
tfversion.SkipBelow(tfversion.Version1_9_0),
tfversion.SkipIfNotPrerelease(),
tfversion.SkipIfNotAlpha(),
},
AdditionalCLIOptions: &r.AdditionalCLIOptions{
Plan: r.PlanOptions{AllowDeferral: true},
Expand Down Expand Up @@ -75,7 +75,7 @@ func Test_ExpectDeferredChange_Reason_NoMatch(t *testing.T) {
r.Test(t, r.TestCase{
TerraformVersionChecks: []tfversion.TerraformVersionCheck{
tfversion.SkipBelow(tfversion.Version1_9_0),
tfversion.SkipIfNotPrerelease(),
tfversion.SkipIfNotAlpha(),
},
AdditionalCLIOptions: &r.AdditionalCLIOptions{
Plan: r.PlanOptions{AllowDeferral: true},
Expand Down Expand Up @@ -127,7 +127,7 @@ func Test_ExpectDeferredChange_NoDeferredChanges(t *testing.T) {
r.Test(t, r.TestCase{
TerraformVersionChecks: []tfversion.TerraformVersionCheck{
tfversion.SkipBelow(tfversion.Version1_9_0),
tfversion.SkipIfNotPrerelease(),
tfversion.SkipIfNotAlpha(),
},
AdditionalCLIOptions: &r.AdditionalCLIOptions{
Plan: r.PlanOptions{AllowDeferral: true},
Expand Down
6 changes: 3 additions & 3 deletions plancheck/expect_no_deferred_changes_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ func Test_ExpectNoDeferredChange(t *testing.T) {
r.Test(t, r.TestCase{
TerraformVersionChecks: []tfversion.TerraformVersionCheck{
tfversion.SkipBelow(tfversion.Version1_9_0),
tfversion.SkipIfNotPrerelease(),
tfversion.SkipIfNotAlpha(),
},
AdditionalCLIOptions: &r.AdditionalCLIOptions{
Plan: r.PlanOptions{AllowDeferral: true},
Expand Down Expand Up @@ -72,7 +72,7 @@ func Test_ExpectNoDeferredChange_OneDeferral(t *testing.T) {
r.Test(t, r.TestCase{
TerraformVersionChecks: []tfversion.TerraformVersionCheck{
tfversion.SkipBelow(tfversion.Version1_9_0),
tfversion.SkipIfNotPrerelease(),
tfversion.SkipIfNotAlpha(),
},
AdditionalCLIOptions: &r.AdditionalCLIOptions{
Plan: r.PlanOptions{AllowDeferral: true},
Expand Down Expand Up @@ -124,7 +124,7 @@ func Test_ExpectNoDeferredChange_MultipleDeferrals(t *testing.T) {
r.Test(t, r.TestCase{
TerraformVersionChecks: []tfversion.TerraformVersionCheck{
tfversion.SkipBelow(tfversion.Version1_9_0),
tfversion.SkipIfNotPrerelease(),
tfversion.SkipIfNotAlpha(),
},
AdditionalCLIOptions: &r.AdditionalCLIOptions{
Plan: r.PlanOptions{AllowDeferral: true},
Expand Down
31 changes: 31 additions & 0 deletions tfversion/skip_if_not_alpha.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0

package tfversion

import (
"context"
"fmt"
"strings"
)

// SkipIfNotAlpha will skip (pass) the test if the Terraform CLI
// version is not an alpha prerelease (for example, 1.10.0-alpha20241023).
//
// Alpha builds of Terraform include experimental features, so this version check
// can be used for acceptance testing of experimental features, such as deferred actions.
func SkipIfNotAlpha() TerraformVersionCheck {
return skipIfNotAlphaCheck{}
}

// skipIfNotAlphaCheck implements the TerraformVersionCheck interface
type skipIfNotAlphaCheck struct{}

// CheckTerraformVersion satisfies the TerraformVersionCheck interface.
func (s skipIfNotAlphaCheck) CheckTerraformVersion(ctx context.Context, req CheckTerraformVersionRequest, resp *CheckTerraformVersionResponse) {
if strings.Contains(req.TerraformVersion.Prerelease(), "alpha") {
return
}

resp.Skip = fmt.Sprintf("Terraform CLI version %s is not an alpha build: skipping test.", req.TerraformVersion)
}
94 changes: 94 additions & 0 deletions tfversion/skip_if_not_alpha_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0

package tfversion_test

import (
"testing"

"github.com/hashicorp/terraform-plugin-go/tfprotov6"

r "github.com/hashicorp/terraform-plugin-testing/helper/resource"
"github.com/hashicorp/terraform-plugin-testing/internal/testing/testprovider"
"github.com/hashicorp/terraform-plugin-testing/internal/testing/testsdk/providerserver"
"github.com/hashicorp/terraform-plugin-testing/tfversion"

testinginterface "github.com/mitchellh/go-testing-interface"
)

func Test_SkipIfNotAlpha_SkipTest_Stable(t *testing.T) { //nolint:paralleltest
t.Setenv("TF_ACC_TERRAFORM_PATH", "")
t.Setenv("TF_ACC_TERRAFORM_VERSION", "1.8.0")

r.UnitTest(t, r.TestCase{
ProtoV6ProviderFactories: map[string]func() (tfprotov6.ProviderServer, error){
"test": func() (tfprotov6.ProviderServer, error) { //nolint:unparam // required signature
return nil, nil
},
},
TerraformVersionChecks: []tfversion.TerraformVersionCheck{
tfversion.SkipIfNotAlpha(),
},
Steps: []r.TestStep{
{
Config: `//non-empty config`,
},
},
})
}

func Test_SkipIfNotAlpha_SkipTest_Beta1(t *testing.T) { //nolint:paralleltest
t.Setenv("TF_ACC_TERRAFORM_PATH", "")
t.Setenv("TF_ACC_TERRAFORM_VERSION", "1.8.0-beta1")

r.UnitTest(t, r.TestCase{
ProtoV6ProviderFactories: map[string]func() (tfprotov6.ProviderServer, error){
"test": providerserver.NewProviderServer(testprovider.Provider{}),
},
TerraformVersionChecks: []tfversion.TerraformVersionCheck{
tfversion.SkipIfNotAlpha(),
},
Steps: []r.TestStep{
{
Config: `//non-empty config`,
},
},
})
}
func Test_SkipIfNotAlpha_SkipTest_RC(t *testing.T) { //nolint:paralleltest
t.Setenv("TF_ACC_TERRAFORM_PATH", "")
t.Setenv("TF_ACC_TERRAFORM_VERSION", "1.8.0-rc2")

r.UnitTest(t, r.TestCase{
ProtoV6ProviderFactories: map[string]func() (tfprotov6.ProviderServer, error){
"test": providerserver.NewProviderServer(testprovider.Provider{}),
},
TerraformVersionChecks: []tfversion.TerraformVersionCheck{
tfversion.SkipIfNotAlpha(),
},
Steps: []r.TestStep{
{
Config: `//non-empty config`,
},
},
})
}

func Test_SkipIfNotAlpha_RunTest_Alpha(t *testing.T) { //nolint:paralleltest
t.Setenv("TF_ACC_TERRAFORM_PATH", "")
t.Setenv("TF_ACC_TERRAFORM_VERSION", "1.9.0-alpha20240501")

r.UnitTest(&testinginterface.RuntimeT{}, r.TestCase{
ProtoV6ProviderFactories: map[string]func() (tfprotov6.ProviderServer, error){
"test": providerserver.NewProviderServer(testprovider.Provider{}),
},
TerraformVersionChecks: []tfversion.TerraformVersionCheck{
tfversion.SkipIfNotAlpha(),
},
Steps: []r.TestStep{
{
Config: `//non-empty config`,
},
},
})
}
Loading