diff --git a/website/docs/plugin/testing/acceptance-tests/index.mdx b/website/docs/plugin/testing/acceptance-tests/index.mdx index adb9c6c5..4703a1f5 100644 --- a/website/docs/plugin/testing/acceptance-tests/index.mdx +++ b/website/docs/plugin/testing/acceptance-tests/index.mdx @@ -17,11 +17,11 @@ or more configuration files, allowing multiple scenarios to be tested. Terraform acceptance tests use real Terraform configurations to exercise the code in real plan, apply, refresh, and destroy life cycles. When run from the -root of a Terraform Provider codebase, Terraform’s testing framework compiles +root of a Terraform Provider codebase, Terraform's testing framework compiles the current provider in-memory and executes the provided configuration in developer defined steps, creating infrastructure along the way. At the conclusion of all the steps, Terraform automatically destroys the -infrastructure. It’s important to note that during development, it’s possible +infrastructure. It's important to note that during development, it's possible for Terraform to leave orphaned or “dangling” resources behind, depending on the correctness of the code in development. The testing framework provides means to validate all resources are destroyed, alerting developers if any fail to @@ -42,7 +42,7 @@ While the test framework provides a reasonable simulation of real-world usage, t Terraform follows many of the Go programming language conventions with regards to testing, with both acceptance tests and unit tests being placed in a file -that matches the file under test, with an added `_test.go` suffix. Here’s an +that matches the file under test, with an added `_test.go` suffix. Here's an example file structure: ``` @@ -302,6 +302,6 @@ This error indicates that the provider server could not connect to Terraform Cor Terraform relies heavily on acceptance tests to ensure we keep our promise of helping users safely and predictably create, change, and improve infrastructure. In our next section we detail how to create “Test Cases”, -individual acceptance tests using Terraform’s testing framework, in order to +individual acceptance tests using Terraform's testing framework, in order to build and verify real infrastructure. [Proceed to Test Cases](/terraform/plugin/testing/acceptance-tests/testcase) diff --git a/website/docs/plugin/testing/acceptance-tests/testcase.mdx b/website/docs/plugin/testing/acceptance-tests/testcase.mdx index 98aa9586..57ddd888 100644 --- a/website/docs/plugin/testing/acceptance-tests/testcase.mdx +++ b/website/docs/plugin/testing/acceptance-tests/testcase.mdx @@ -9,13 +9,13 @@ description: |- Acceptance tests are expressed in terms of **Test Cases**, each using one or more Terraform configurations designed to create a set of resources under test, -and then verify the actual infrastructure created. Terraform’s `resource` +and then verify the actual infrastructure created. Terraform's `resource` package offers a method `Test()`, accepting two parameters and acting as the -entry point to Terraform’s acceptance test framework. The first parameter is the -standard [\*testing.T struct from Golang’s Testing package][3], and the second is +entry point to Terraform's acceptance test framework. The first parameter is the +standard [\*testing.T struct from Golang's Testing package][3], and the second is [TestCase][1], a Go struct that developers use to setup the acceptance tests. -Here’s an example acceptance test. Here the Provider is named `Example`, and the +Here's an example acceptance test. Here the Provider is named `Example`, and the Resource under test is `Widget`. The parts of this test are explained below the example. @@ -34,15 +34,15 @@ func TestAccExampleWidget_basic(t *testing.T) { Steps: []resource.TestStep{ { Config: testAccExampleResource(rName), - Check: resource.ComposeTestCheckFunc( - testAccCheckExampleResourceExists("example_widget.foo", &widgetBefore), - ), + ConfigStateChecks: []statecheck.StateCheck{ + stateCheckExampleResourceExists("example_widget.foo", &widgetBefore), + }, }, { Config: testAccExampleResource_removedPolicy(rName), - Check: resource.ComposeTestCheckFunc( - testAccCheckExampleResourceExists("example_widget.foo", &widgetAfter), - ), + ConfigStateChecks: []statecheck.StateCheck{ + stateCheckExampleResourceExists("example_widget.foo", &widgetAfter), + }, }, }, }) @@ -81,7 +81,7 @@ func TestAccExampleWidget_basic(t *testing.T) { The majority of acceptance tests will only invoke `resource.Test()` and exit. If at any point this method encounters an error, either in executing the provided Terraform configurations or subsequent developer defined checks, `Test()` will -invoke the `t.Error` method of Go’s standard testing framework and the test will +invoke the `t.Error` method of Go's standard testing framework and the test will fail. A failed test will not halt or otherwise interrupt any other tests currently running. @@ -190,7 +190,7 @@ a configuration file for testing must be represented in this map or the test will fail during initialization. This map is most commonly constructed once in a common `init()` method of the -Provider’s main test file, and includes an object of the current Provider type. +Provider's main test file, and includes an object of the current Provider type. **Example usage:** (note the different files `widget_test.go` and `provider_test.go`) @@ -326,15 +326,15 @@ func TestAccExampleWidget_basic(t *testing.T) { Steps: []resource.TestStep{ { Config: testAccExampleResource(rName), - Check: resource.ComposeTestCheckFunc( - testAccCheckExampleResourceExists("example_widget.foo", &widgetBefore), - ), + ConfigStateChecks: []statecheck.StateCheck{ + stateCheckExampleResourceExists("example_widget.foo", &widgetBefore), + }, }, { Config: testAccExampleResource_removedPolicy(rName), - Check: resource.ComposeTestCheckFunc( - testAccCheckExampleResourceExists("example_widget.foo", &widgetAfter), - ), + ConfigStateChecks: []statecheck.StateCheck{ + stateCheckExampleResourceExists("example_widget.foo", &widgetAfter), + }, }, }, }) @@ -346,7 +346,7 @@ func TestAccExampleWidget_basic(t *testing.T) { `TestCases` are used to verify the features of a given part of a plugin. Each case should represent a scenario of normal usage of the plugin, from simple creation to creating, adding, and removing specific properties. In the next -Section [`TestSteps`][2], we’ll detail `Steps` portion of `TestCase` and see how +Section [`TestSteps`][2], we'll detail `Steps` portion of `TestCase` and see how to create these scenarios by iterating on Terraform configurations. [1]: https://pkg.go.dev/github.com/hashicorp/terraform-plugin-testing/helper/resource#TestCase diff --git a/website/docs/plugin/testing/acceptance-tests/teststep.mdx b/website/docs/plugin/testing/acceptance-tests/teststep.mdx index 5f820fa0..01e79199 100644 --- a/website/docs/plugin/testing/acceptance-tests/teststep.mdx +++ b/website/docs/plugin/testing/acceptance-tests/teststep.mdx @@ -64,22 +64,22 @@ func TestAccExampleWidget_basic(t *testing.T) { Steps: []resource.TestStep{ { Config: testAccExampleResource(rName), - Check: resource.ComposeTestCheckFunc( - testAccCheckExampleResourceExists("example_widget.foo", &widgetBefore), - ), + ConfigStateChecks: []statecheck.StateCheck{ + stateCheckExampleResourceExists("example_widget.foo", &widgetBefore), + }, }, { Config: testAccExampleResource_removedPolicy(rName), - Check: resource.ComposeTestCheckFunc( - testAccCheckExampleResourceExists("example_widget.foo", &widgetAfter), - ), + ConfigStateChecks: []statecheck.StateCheck{ + stateCheckExampleResourceExists("example_widget.foo", &widgetAfter), + }, }, }, }) } ``` -In the above example each `TestCase` invokes a function to retrieve it’s desired +In the above example each `TestCase` invokes a function to retrieve it's desired configuration, based on a randomized name provided, however an in-line string or constant string would work as well, so long as they contain valid Terraform configuration for the plugin or resource under test. This pattern of first @@ -87,19 +87,36 @@ applying and checking a basic configuration, followed by applying a modified configuration with updated or additional checks is a common pattern used to test update functionality. -## State Check Functions +## Plan Checks +Before and after the configuration for a `TestStep` is applied, Terraform's testing framework provides developers an opportunity to make test assertions against `terraform plan` results via the plan file. This is provided via [Plan Checks](/terraform/plugin/testing/acceptance-tests/plan-checks), which provide both built-in plan checks and an interface to implement custom plan checks. + +## State Checks + +After the configuration for a `TestStep` is applied, Terraform's testing +framework provides developers an opportunity to check the results by providing one +or more [state check implementations](/terraform/plugin/testing/acceptance-tests/state-checks). +While possible to only supply a single state check, it is recommended you use multiple state checks +to validate specific information about the results of the `terraform apply` ran in each `TestStep`. -After the configuration for a `TestStep` is applied, Terraform’s testing -framework provides developers an opportunity to check the results by providing a -“Check” function. While possible to only supply a single function, it is -recommended you use multiple functions to validate specific information about -the results of the `terraform apply` ran in each `TestStep`. The `Check` +See the [State Checks](/terraform/plugin/testing/acceptance-tests/state-checks) section for more information about the built-in state checks for resources, data sources, +output values, and how to write custom state checks. + +### Legacy Check function + + + +Use the new `ConfigStateChecks` attribute and [State Check implementations](/terraform/plugin/testing/acceptance-tests/state-checks) +instead of the `Check` function. + + + +The `Check` function is used to check results of a Terraform operation. The `Check` attribute of `TestStep` is singular, so in order to include multiple checks developers should use either `ComposeTestCheckFunc` or `ComposeAggregateTestCheckFunc` (defined below) to group multiple check functions, defined below: -### ComposeTestCheckFunc +#### ComposeTestCheckFunc ComposeTestCheckFunc lets you compose multiple TestCheckFunc functions into a single check. As a user testing their provider, this lets you decompose your @@ -124,10 +141,10 @@ Steps: []resource.TestStep{ }, ``` -### ComposeAggregateTestCheckFunc +#### ComposeAggregateTestCheckFunc ComposeAggregateTestCheckFunc lets you compose multiple TestCheckFunc functions -into a single check. It’s purpose and usage is identical to +into a single check. It's purpose and usage is identical to ComposeTestCheckFunc, however each check is ran in order even if a previous check failed, collecting the errors returned from any checks and returning a single aggregate error. The entire `TestCase` is still stopped, and Terraform @@ -149,7 +166,7 @@ Steps: []resource.TestStep{ }, ``` -## Builtin check functions +#### Built-in check functions Terraform has several TestCheckFunc functions built in for developers to use for common checks, such as verifying the status and value of a specific attribute in @@ -204,7 +221,7 @@ All of these functions also accept the below syntax in attribute keys to enable | `.#` | Number of elements in list or set | `TestCheckResourceAttr("example_widget.foo", "some_list.#", "2")` | | `.%` | Number of keys in map | `TestCheckResourceAttr("example_widget.foo", "some_map.%", "2")` | -## Custom check functions +### Custom check functions The `Check` field of `TestStep` accepts any function of type [TestCheckFunc](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-testing/helper/resource#TestCheckFunc). @@ -299,9 +316,6 @@ func testAccCheckExampleWidgetExists(resourceName string, widget *example.Widget } ``` -## Plan Checks -Before and after the configuration for a `TestStep` is applied, Terraform's testing framework provides developers an opportunity to make test assertions against `terraform plan` results via the plan file. This is provided via [Plan Checks](/terraform/plugin/testing/acceptance-tests/plan-checks), which provide both built-in plan checks and an interface to implement custom plan checks. - ## Sweepers -Acceptance Testing is an essential approach to validating the implementation of a Terraform Provider. Using actual APIs to provision resources for testing can leave behind real infrastructure that costs money between tests. The reasons for these leaks can vary, regardless Terraform provides a mechanism known as [Sweepers](/terraform/plugin/testing/acceptance-tests/sweepers) to help keep the testing account clean. +Acceptance Testing is an essential approach to validating the implementation of a Terraform Provider. Using actual APIs to provision resources for testing can leave behind real infrastructure that costs money between tests. The reasons for these leaks can vary, regardless Terraform provides a mechanism known as [Sweepers](/terraform/plugin/testing/acceptance-tests/sweepers) to help keep the testing account clean. \ No newline at end of file diff --git a/website/docs/plugin/testing/index.mdx b/website/docs/plugin/testing/index.mdx index 34824c81..2ed9cb36 100644 --- a/website/docs/plugin/testing/index.mdx +++ b/website/docs/plugin/testing/index.mdx @@ -26,7 +26,7 @@ verified. Terraform includes a framework for constructing acceptance tests that imitate the execution of one or more steps of applying one or more configuration files, allowing multiple scenarios to be tested. -It’s important to reiterate that acceptance tests in resources _create actual +It's important to reiterate that acceptance tests in resources _create actual cloud infrastructure_, with possible expenses incurred, and are the responsibility of the user running the tests. Creating real infrastructure in tests verifies the described behavior of Terraform Plugins in real world use diff --git a/website/docs/plugin/testing/testing-patterns.mdx b/website/docs/plugin/testing/testing-patterns.mdx index 985a4228..662e51d4 100644 --- a/website/docs/plugin/testing/testing-patterns.mdx +++ b/website/docs/plugin/testing/testing-patterns.mdx @@ -7,9 +7,9 @@ description: |- # Testing Patterns -In [Testing Terraform Plugins][1] we introduce Terraform’s Testing Framework, +In [Testing Terraform Plugins][1] we introduce Terraform's Testing Framework, providing reference for its functionality and introducing the basic parts of -writing acceptance tests. In this section we’ll cover some test patterns that +writing acceptance tests. In this section we'll cover some test patterns that are common and considered a best practice to have when developing and verifying your Terraform plugins. At time of writing these guides are particular to Terraform Resources, but other testing best practices may be added later. @@ -25,7 +25,7 @@ Terraform Resources, but other testing best practices may be added later. ## Built-in Patterns Acceptance tests use [TestCases][2] to construct scenarios that can be evaluated -with Terraform’s lifecycle of plan, apply, refresh, and destroy. The test +with Terraform's lifecycle of plan, apply, refresh, and destroy. The test framework has some behaviors built in that provide very basic workflow assurance tests, such as verifying configurations apply with no diff generated by the next plan. @@ -68,21 +68,21 @@ establish the following: The first and last item are provided by the test framework as described above in **Built-in Patterns**. The middle items are implemented by composing a series of -Check Functions as described in [Acceptance Tests: TestSteps][8]. +State Check implementations as described in [Acceptance Tests: State Checks][8]. To verify attributes are saved to the state file correctly, use a combination of -the built-in check functions provided by the testing framework. See [Built-in -Check Functions][9] to see available functions. +the built-in [`statecheck`](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-testing/statecheck) +implementations provided by the testing framework. See [Resource State Checks][9] to see available +state checks for managed resource and data source attributes. Checking the values in a remote API generally consists of two parts: a function -to verify the corresponding object exists remotely, and a separate function to +to verify the corresponding object exists remotely, and a state check implementation to verify the values of the object. By separating the check used to verify the object exists into its own function, developers are free to re-use it for all -TestCases as a means of retrieving it’s values, and can provide custom check -functions per TestCase to verify different attributes or scenarios specific to -that TestCase. +TestSteps as a means of retrieving it's values, and can provide [custom state checks][10] +functions per TestStep to verify remote values or scenarios specific to that TestStep. -Here’s an example test, with in-line comments to demonstrate the key parts of a +Here's an example test, with in-line comments to demonstrate the key parts of a basic test. ```go @@ -107,62 +107,20 @@ func TestAccExampleWidget_basic(t *testing.T) { // use a dynamic configuration with the random name from above Config: testAccExampleResource(rName), // compose a basic test, checking both remote and local values - Check: resource.ComposeTestCheckFunc( - // query the API to retrieve the widget object - testAccCheckExampleResourceExists("example_widget.foo", &widget), - // verify remote values - testAccCheckExampleWidgetValues(widget, rName), - // verify local values - resource.TestCheckResourceAttr("example_widget.foo", "active", "true"), - resource.TestCheckResourceAttr("example_widget.foo", "name", rName), - ), + ConfigStateChecks: []statecheck.StateCheck{ + // custom state check - query the API to retrieve the widget object + stateCheckExampleResourceExists("example_widget.foo", &widget), + // custom state check - verify remote values + stateCheckExampleWidgetValues(widget, rName), + // built-in state checks - verify local (state) values + statecheck.ExpectKnownValue("example_widget.foo", tfjsonpath.New("active"), knownvalue.Bool(true)), + statecheck.ExpectKnownValue("example_widget.foo", tfjsonpath.New("name"), knownvalue.StringExact(rName)), + }, }, }, }) } -func testAccCheckExampleWidgetValues(widget *example.Widget, name string) resource.TestCheckFunc { - return func(s *terraform.State) error { - if *widget.Active != true { - return fmt.Errorf("bad active state, expected \"true\", got: %#v", *widget.Active) - } - if *widget.Name != name { - return fmt.Errorf("bad name, expected \"%s\", got: %#v", name, *widget.Name) - } - return nil - } -} - -// testAccCheckExampleResourceExists queries the API and retrieves the matching Widget. -func testAccCheckExampleResourceExists(n string, widget *example.Widget) resource.TestCheckFunc { - return func(s *terraform.State) error { - // find the corresponding state object - rs, ok := s.RootModule().Resources[n] - if !ok { - return fmt.Errorf("Not found: %s", n) - } - - // retrieve the configured client from the test setup - conn := testAccProvider.Meta().(*ExampleClient) - resp, err := conn.DescribeWidget(&example.DescribeWidgetsInput{ - WidgetIdentifier: rs.Primary.ID, - }) - - if err != nil { - return err - } - - if resp.Widget == nil { - return fmt.Errorf("Widget (%s) not found", rs.Primary.ID) - } - - // assign the response Widget attribute to the widget pointer - *widget = *resp.Widget - - return nil - } -} - // testAccExampleResource returns an configuration for an Example Widget with the provided name func testAccExampleResource(name string) string { return fmt.Sprintf(` @@ -189,8 +147,8 @@ configuration. Below is an example test, copied and modified from the basic test. Here we preserve the `TestStep` from the basic test, but we add an additional `TestStep`, changing the configuration and rechecking the values, with a -different configuration function `testAccExampleResourceUpdated` and check -function `testAccCheckExampleWidgetValuesUpdated` for verifying the values. +different configuration function `testAccExampleResourceUpdated` and state check +implementation `stateCheckExampleWidgetValuesUpdated` for verifying the values. ```go func TestAccExampleWidget_update(t *testing.T) { @@ -205,39 +163,27 @@ func TestAccExampleWidget_update(t *testing.T) { { // use a dynamic configuration with the random name from above Config: testAccExampleResource(rName), - Check: resource.ComposeTestCheckFunc( - testAccCheckExampleResourceExists("example_widget.foo", &widget), - testAccCheckExampleWidgetValues(widget, rName), - resource.TestCheckResourceAttr("example_widget.foo", "active", "true"), - resource.TestCheckResourceAttr("example_widget.foo", "name", rName), - ), + ConfigStateChecks: []statecheck.StateCheck{ + stateCheckExampleResourceExists("example_widget.foo", &widget), + stateCheckExampleWidgetValues(widget, rName), + statecheck.ExpectKnownValue("example_widget.foo", tfjsonpath.New("active"), knownvalue.Bool(true)), + statecheck.ExpectKnownValue("example_widget.foo", tfjsonpath.New("name"), knownvalue.StringExact(rName)), + }, }, { // use a dynamic configuration with the random name from above Config: testAccExampleResourceUpdated(rName), - Check: resource.ComposeTestCheckFunc( - testAccCheckExampleResourceExists("example_widget.foo", &widget), - testAccCheckExampleWidgetValuesUpdated(widget, rName), - resource.TestCheckResourceAttr("example_widget.foo", "active", "false"), - resource.TestCheckResourceAttr("example_widget.foo", "name", rName), - ), + ConfigStateChecks: []statecheck.StateCheck{ + stateCheckExampleResourceExists("example_widget.foo", &widget), + stateCheckExampleWidgetValuesUpdated(widget, rName), + statecheck.ExpectKnownValue("example_widget.foo", tfjsonpath.New("active"), knownvalue.Bool(false)), + statecheck.ExpectKnownValue("example_widget.foo", tfjsonpath.New("name"), knownvalue.StringExact(rName)), + }, }, }, }) } -func testAccCheckExampleWidgetValuesUpdated(widget *example.Widget, name string) resource.TestCheckFunc { - return func(s *terraform.State) error { - if *widget.Active != false { - return fmt.Errorf("bad active state, expected \"false\", got: %#v", *widget.Active) - } - if *widget.Name != name { - return fmt.Errorf("bad name, expected \"%s\", got: %#v", name, *widget.Name) - } - return nil - } -} - // testAccExampleResource returns an configuration for an Example Widget with the provided name func testAccExampleResourceUpdated(name string) string { return fmt.Sprintf(` @@ -248,7 +194,7 @@ resource "example_widget" "foo" { } ``` -It’s common for resources to just have the above update test, as it is a +It's common for resources to just have the above update test, as it is a superset of the basic test. So long as the basics are covered, combining the two tests is sufficient as opposed to having two separate tests. @@ -287,17 +233,17 @@ func TestAccExampleWidget_expectPlan(t *testing.T) { // use an incomplete configuration that we expect // to result in a non-empty plan after apply Config: testAccExampleResourceIncomplete(rName), - Check: resource.ComposeTestCheckFunc( - resource.TestCheckResourceAttr("example_widget.foo", "name", rName), - ), + ConfigStateChecks: []statecheck.StateCheck{ + statecheck.ExpectKnownValue("example_widget.foo", tfjsonpath.New("name"), knownvalue.StringExact(rName)), + }, ExpectNonEmptyPlan: true, }, { // apply the complete configuration Config: testAccExampleResourceComplete(rName), - Check: resource.ComposeTestCheckFunc( - resource.TestCheckResourceAttr("example_widget.foo", "name", rName), - ), + ConfigStateChecks: []statecheck.StateCheck{ + statecheck.ExpectKnownValue("example_widget.foo", tfjsonpath.New("name"), knownvalue.StringExact(rName)), + }, }, }, }) @@ -362,7 +308,7 @@ then advancing the revisions to evaluate the fix. # Conclusion -Terraform’s [Testing Framework][1] allows for powerful, iterative acceptance +Terraform's [Testing Framework][1] allows for powerful, iterative acceptance tests that enable developers to fully test the behavior of Terraform plugins. By following the above best practices, developers can ensure their plugin behaves correctly across the most common use cases and everyday operations users will @@ -383,6 +329,8 @@ for safely managing infrastructure. [7]: https://pkg.go.dev/github.com/hashicorp/terraform-plugin-testing/helper/resource#TestStep.ExpectError -[8]: /terraform/plugin/testing/acceptance-tests/teststep#check-functions +[8]: /terraform/plugin/testing/acceptance-tests/state-checks + +[9]: /terraform/plugin/testing/acceptance-tests/state-checks/resource -[9]: /terraform/plugin/testing/acceptance-tests/teststep#builtin-check-functions \ No newline at end of file +[10]: /terraform/plugin/testing/acceptance-tests/state-checks/custom \ No newline at end of file