Consider Plan Check for Output Value Changes

[bflad]

terraform-plugin-testing version

v1.5.1

Use cases

There are scenarios where developers may want to use output values for checking plan differences rather than needing a "heavier" configuration setup of a managed resource to store the state and check against that managed resource for plan differences. Checking whole data source contents between provider versions is one such example.

For that example, the configuration looks like:

data "examplecloud_thing" "test" {}

resource "terraform_data" "test" {
  input = data.examplecloud_thing.test
}

If using the legacy null_resource managed resource, there's additional testing configuration necessary such as properly setting up the TestStep.ExternalProviders field.

Developers may not be contextually expecting that a managed resource is the correct way to store a value of a provider concept which implicitly does not store state across Terraform runs, instead reaching for an output, which can accomplish the same thing:

data "examplecloud_thing" "test" {}

output "test" {
  value = data.examplecloud_thing.test
}

When testing this way via TestStep.PlanOnly though, output value changes do not cause the test to fail. Instead a plan check must be used, which there is no built-in plan check available today.

Proposal

At some point, we should likely consider deprecating PlanOnly since its a confusing abstraction for developers and instead recommend plan checks. This will be separately proposed though.

This Go module could introduce a new built-in plan check for this, e.g. as a code sketch

// naming debatable of course, this is just a working code sketch
func ExpectOutputNoChanges(output string) plancheck.PlanCheck {
	return expectOutputNoChangesCheck{
		output: output,
	}
}

type expectOutputNoChangesCheck struct {
	output string
}

func (c expectOutputNoChangesCheck) CheckPlan(ctx context.Context, req plancheck.CheckPlanRequest, resp *plancheck.CheckPlanResponse) {
	if len(req.Plan.OutputChanges) == 0 {
		return
	}

	change, ok := req.Plan.OutputChanges[c.output]

	if !ok || change == nil {
		return
	}

	resp.Error = fmt.Errorf("unexpected output %q change", c.output)
}

This then plan check is usable for this testing situation:

ConfigPlanChecks: resource.ConfigPlanChecks{
	PreApply: []plancheck.PlanCheck{
		plancheck.ExpectOutputNoChanges("test"),
	}.
}

References

• Testing Migration (from SDKv2) of Datasources does not fail when fields are missing terraform-plugin-framework#875