google_composer_environment after initial rollout, forces new resource with no changes

[the-nando]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.0.10
on darwin_arm64
+ provider registry.terraform.io/hashicorp/google v3.90.0
+ provider registry.terraform.io/hashicorp/google-beta v3.90.0

Affected Resource(s)

• google_composer_environment

Terraform Configuration Files

I'm using this module with default values besides

image_version   = "composer-1.17.3-airflow-2.1.2"
python_version  = 3

Debug Output

  # google_composer_environment.composer_env must be replaced
-/+ resource "google_composer_environment" "composer_env" {
[...]
          ~ software_config {
              - env_variables            = {} -> null
              - pypi_packages            = {} -> null
                # (4 unchanged attributes hidden)
            }
}

Expected Behavior

Nothing should happen as no changes were done after the initial re-roll, either outside or inside Terraform.

Actual Behavior

If applied Terraform will re-roll the composer environment and will keep doing so on subquent plan / apply as well.
I have tried to ignore changes but that didn't help

lifecycle {
    ignore_changes = [ 
      config.0.software_config.0.pypi_packages,
      config.0.software_config.0.env_variables,
    ]
  }

Only workaround is to ignore the whole software_config block but that makes it more difficult to replace composer environments on image changes, which are now ignored as well:

lifecycle {
    ignore_changes = [ 
      config["software_config"]
    ]
  } 

Important Factoids

Another related issue is that any changes to env_variables or pypi_packages force a replacement of the composer environment even though it should be possible to push such changes by patching the existing environment.

References

[ScottSuarez]

Do you have the logs here? this looks like a functional change with composer api
It is returning an empty object {} if the value is not set here. Which terraform does not like.

Please set TF_LOG=DEBUG and TF_LOG_PATH=./path-to-log and post the scrubbed results as a gist

[the-nando]

I cannot share the full log at the moment but it's indeed expecting a null but gets back an empty map:

          ~ software_config {
              - env_variables            = {} -> null

2021-11-05T17:11:53.196+0100 [WARN]  Provider "registry.terraform.io/hashicorp/google" produced an unexpected new value for google_composer_environment.composer_env during refresh.
      - .config[0].software_config[0].env_variables: was null, but now cty.MapValEmpty(cty.String)
[...]

Is this log sufficent? I can setup a test env and share the full log otherwise.

Do you also happen to know why this change forces a replacement instead of an update? I currently cannot manage pypi_packages or env_variables from Terraform as any changes done there would trigger a re-provision.

[ScottSuarez]

Try ignoring changes here. Could you make a bug request against the api? I can get around to it otherwise
https://www.terraform.io/docs/language/meta-arguments/lifecycle.html#ignore_changes

[the-nando]

I tried to ignore_changes (see iniital message) but somehow it has no effect:

lifecycle {
    ignore_changes = [
      config[0].software_config[0].env_variables,
    ]
  }

Result:

     ~ software_config {
              - env_variables            = {} -> null
                # (5 unchanged attributes hidden)
            }

It still shouldn't replace the whole environment on a change to env_variables or pypi_packages.

Is this a Terraform or Google API (which one?) issue? I'm asking because the Terraform module does set both env_variables and pypi_packages to {} on initial rollout. Why does Terraform want to change that to a null on a subsequent run?

[the-nando]

with the google 4.7 provider the ignore changes now works as expected

lifecycle {
    ignore_changes = [
      config[0].software_config[0].env_variables,
      config[0].software_config[0].pypi_packages
    ]
  }

Haven't tested trying to update the env variables list.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.