Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2.20.0 breaks google_compute_region_backend_service with "backend.0.group": required field is not set #4951

Closed
korjik opened this issue Nov 20, 2019 · 19 comments

Comments

@korjik
Copy link

korjik commented Nov 20, 2019

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment
  • If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.

Terraform Version

Terraform v0.11.14

  • provider.google v2.20.0
  • provider.google-beta v2.20.0
  • provider.template v2.1.2

Affected Resource(s)

  • google_compute_region_backend_service

Terraform Configuration Files

variable "backends" {
  type = "list"
  default = [
    {
      group = "${local.instance_group[0]}"
    },
    {
      group = "${local.instance_group[1]}"
    },
    {
      group = "${local.instance_group[2]}"
    },
  ]
}

resource "google_compute_region_backend_service" "internal" {
  name                  = "${local.backend_service_name}"
  project               = "${local.project_id}"
  region                = "${var.region}"
  health_checks         = ["${local.health_checks}"]
  load_balancing_scheme = "INTERNAL"
  session_affinity      = "${var.session_affinity}"
  protocol              = "${var.ip_protocol}"
  backend               = ["${var.backends}"]
}

Debug Output

https://gist.github.com/korjik/70eb44a79671cb4d686ae3c0a0ede1c2

Expected Behavior

terraform plan works fine

Actual Behavior

Terrafrom plan breaks with:

google_compute_region_backend_service.internal: "backend.0.group": required field is not set

Steps to Reproduce

  1. Use Google provider 2.19.0 and run terraform plan over a resource google_compute_region_backend_service - all works fine
  2. Change provider to 2.20.0 and the error will be thrown

b/309764960

@ghost ghost added the bug label Nov 20, 2019
@slevenick
Copy link
Collaborator

Hey @korjik,

Are you sure that the variable that is defined for backends has the group field set? The group field is now marked as required, so if that field is not set it would throw an error.

In the past you could theoretically provide an empty block for the backend object and it would succeed (but not be actually useful). I'm able to run terraform plan with the following config:

resource "google_compute_region_backend_service" "default" {
  name                            = "region-backend-service"
  region                          = "us-central1"
  health_checks                   = ["${google_compute_health_check.default.self_link}"]
  connection_draining_timeout_sec = 10
  session_affinity                = "CLIENT_IP"
  backend = ["${var.backends}"]
}

variable "backends" {
  type = "list"
  default = [
    {
      group = "backend1-test"
    },
    {
      group = "backend2-test"
    },
  ]
}

This plans successfully, but if I remove one of the group definitions to make the backends var look like this:

variable "backends" {
  type = "list"
  default = [
    {
    },
    {
      group = "backend2-test"
    },
  ]
}

then it fails with the error message you have posted.

Can you verify that the backends variable correctly has the group field set for each member of the list, and post the result here? You can do this by adding this to a successful apply run:

output "backends" {
  value = "${var.backends}"
}

The apply needs to succeed, so you may need to comment out resources that are failing

@neilepalmer
Copy link

FWIW, I get the same error with the same TF & provider configurations. I'm passing in:
backends = [
{ group = "${module.es_client.region_instance_group}" },
]

@ghost ghost removed the waiting-response label Nov 21, 2019
@slevenick
Copy link
Collaborator

@neilepalmer are you able to output the backends variable and post the result?

Are you able to provide full debug logs for this issue? It would help track down the problem as I am unable to reproduce currently

@upodroid
Copy link
Contributor

Hi

I'm getting the same error.

Error: Error updating BackendService "athena-be": googleapi: Error 400: Invalid value for field 'resource.backends[0].group': 'https://www.googleapis.com/compute/v1/projects/REDACTED/regions/europe-west2/instanceGroupManagers/athena-ig'. Unexpected resource collection 'instanceGroupManagers'., invalid

  on ..\athena.tf line 105, in resource "google_compute_backend_service" "athena":
 105: resource "google_compute_backend_service" "athena" {

Configuration:

resource "google_compute_region_instance_group_manager" "athena" {
  name               = "athena-ig"
  base_instance_name = "athena"
  project = var.project_id

  version {
    name              = "primary"
    instance_template = google_compute_instance_template.athena.self_link
  }

  region                    = var.region
  distribution_policy_zones = var.zones
  named_port {
    name = "http"
    port = 80
  }

  target_size = lookup(var.athena_vars, "targetSize", 1)

  update_policy {
    type                  = "PROACTIVE"
    minimal_action        = "REPLACE"
    max_surge_fixed       = 30
    max_unavailable_fixed = 3
    min_ready_sec         = 30
  }
}

resource "google_compute_backend_service" "athena" {
  name        = "athena-be"
  port_name   = "http"
  protocol    = "HTTP"
  timeout_sec = 300
  backend {
    group = google_compute_region_instance_group_manager.athena.self_link
  }
  health_checks = [google_compute_health_check.athena-hc.self_link]
}

Terraform version

S C:\Git\> terraform version
Terraform v0.12.16
+ provider.google v2.20.0
+ provider.google-beta v3.0.0-beta.1

@ghost ghost removed the waiting-response label Nov 26, 2019
@upodroid
Copy link
Contributor

On a sidenote, I fixed my issue because I was referencing the self link of the IG instead of the instance group parameter.

instance_group            = "https://www.googleapis.com/compute/v1/projects/REDACTED/regions/europe-west2/instanceGroups/appserver-igm" 

self_link                 = "https://www.googleapis.com/compute/v1/projects/REDACTED/regions/europe-west2/instanceGroupManagers/appserver-igm" 

Will it be possible to use the self link to reference the IG instead of the instance group parameter?

Thanks

@slevenick
Copy link
Collaborator

@upodroid I don't understand,

You want to reference the instance group that is created by the instance group manager via self_link? self_link generally refers to the resource itself, not anything managed by it. Why do you not want to reference the created instance group via the instance_group field?

@upodroid
Copy link
Contributor

I assumed the IG and IGM are the same thing. If they aren't, then please ignore my request.

@ghost ghost removed the waiting-response label Nov 26, 2019
@billyfoss
Copy link

I am getting the same error
Error: module.my-lb.google_compute_region_backend_service.default: "backend.0.group": required field is not set

This is on an environment already created with 2.19.0 and trying to run terraform plan with 2.20.0.

The terraform.tfstate file shows the backend list is populated with full links. (names updated to be generic)

                "google_compute_region_backend_service.default": {
                    "type": "google_compute_region_backend_service",
                    "depends_on": [
                        "google_compute_health_check.http-hc.*",
                        "google_compute_health_check.https-hc.*",
                        "google_compute_health_check.tcp-hc.*"
                    ],
                    "primary": {
                        "id": "my-lb-appint",
                        "attributes": {
                            "backend.#": "3",
                            "backend.2188334471.description": "",
                            "backend.2188334471.group": "https://www.googleapis.com/compute/v1/projects/my-proj/zones/us-east1-b/instanceGroups/edge-node01-us-east1-b",
                            "backend.3379179555.description": "",
                            "backend.3379179555.group": "https://www.googleapis.com/compute/v1/projects/my-proj/zones/us-east1-c/instanceGroups/edge-node01-us-east1-c",
                            "backend.3808497182.description": "",
                            "backend.3808497182.group": "https://www.googleapis.com/compute/v1/projects/my-proj/zones/us-east1-d/instanceGroups/edge-node01-us-east1-d",
                            "connection_draining_timeout_sec": "0",
                            "description": "",

Is this an issue with using the group self link instead of the short name?

@billyfoss
Copy link

With 2.20.0 the debug ends with

2019/11/28 00:41:54 [DEBUG] Resource state not found for "module.edgenodes01.google_compute_instance_group.group[1]": module.edgenodes01.google_compute_instance_group.group[1]
2019/11/28 00:41:54 [DEBUG] Resource state not found for "module.edgenodes01.google_compute_instance_group.group[2]": module.edgenodes01.google_compute_instance_group.group[2]
2019/11/28 00:41:54 [DEBUG] Resource state not found for "module.edgenodes01.google_compute_instance_group.group[0]": module.edgenodes01.google_compute_instance_group.group[0]
2019/11/28 00:41:54 [TRACE] Graph after step *terraform.AttachStateTransformer:

module.edgenodes01.google_compute_instance_group.group[0] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[1] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[2] - *terraform.NodeValidatableResourceInstance
2019/11/28 00:41:54 [DEBUG] ReferenceTransformer: "module.edgenodes01.google_compute_instance_group.group[0]" references: []
2019/11/28 00:41:54 [DEBUG] ReferenceTransformer: "module.edgenodes01.google_compute_instance_group.group[1]" references: []
2019/11/28 00:41:54 [DEBUG] ReferenceTransformer: "module.edgenodes01.google_compute_instance_group.group[2]" references: []
2019/11/28 00:41:54 [TRACE] Graph after step *terraform.ReferenceTransformer:

module.edgenodes01.google_compute_instance_group.group[0] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[1] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[2] - *terraform.NodeValidatableResourceInstance
2019/11/28 00:41:54 [DEBUG] Setting Unknown Variable Value for computed key: instance_ids
2019/11/28 00:41:54 [DEBUG] Setting Unknown Variable Value for computed key: internal_backends.0.group
2019/11/28 00:41:54 [DEBUG] Setting Unknown Variable Value for computed key: internal_backends.1.group
2019/11/28 00:41:54 [DEBUG] Setting Unknown Variable Value for computed key: internal_backends.2.group
2019/11/28 00:41:54 [TRACE] root: eval: *terraform.EvalCoerceMapVariable
2019/11/28 00:41:54 [DEBUG] Resource state not found for "module.edgenodes01-AppInt.google_compute_region_backend_service.default": module.edgenodes01-AppInt.google_compute_region_backend_service.default
2019/11/28 00:41:54 [TRACE] Graph after step *terraform.AttachStateTransformer:

module.edgenodes01-AppInt.google_compute_region_backend_service.default - *terraform.NodeValidatableResourceInstance
2019/11/28 00:41:54 [DEBUG] Setting Unknown Variable Value for computed key: backends.0.group
2019/11/28 00:41:54 [DEBUG] Setting Unknown Variable Value for computed key: backends.1.group
2019/11/28 00:41:54 [DEBUG] Setting Unknown Variable Value for computed key: backends.2.group
2019/11/28 00:41:54 [DEBUG] ReferenceTransformer: "module.edgenodes01-AppInt.google_compute_region_backend_service.default" references: []
2019/11/28 00:41:54 [ERROR] root.edgenodes01-AppInt: eval: *terraform.EvalValidateResource, err: Warnings: []. Errors: ["backend.0.group": required field is not set]
2019/11/28 00:41:54 [ERROR] root.edgenodes01-AppInt: eval: *terraform.EvalSequence, err: Warnings: []. Errors: ["backend.0.group": required field is not set]
2019/11/28 00:41:54 [TRACE] [walkValidate] Exiting eval tree: module.edgenodes01-AppInt.google_compute_region_backend_service.default
2019/11/28 00:41:54 [DEBUG] Resource state not found for "module.edgenodes01-AppInt.google_compute_forwarding_rule.internal": module.edgenodes01-AppInt.google_compute_forwarding_rule.internal
2019/11/28 00:41:54 [DEBUG] ReferenceTransformer: "module.edgenodes01-AppInt.google_compute_forwarding_rule.internal" references: []
�[31m
�[1m�[31mError: �[0m�[0m�[1mmodule.edgenodes01-AppInt.google_compute_region_backend_service.default: "backend.0.group": required field is not set�[0m

�[0m�[0m�[0m
2019/11/28 00:41:54 [DEBUG] plugin: waiting for all plugin processes to complete...
2019-11-28T00:41:54.397Z [DEBUG] plugin.terraform-provider-template_v2.1.2_x4: 2019/11/28 00:41:54 [ERR] plugin: plugin server: accept unix /tmp/plugin191120976: use of closed network connection
2019-11-28T00:41:54.398Z [DEBUG] plugin: plugin process exited: path=/home/deployuser/.terraform.d/plugins/terraform-provider-template_v2.1.2_x4
2019-11-28T00:41:54.399Z [DEBUG] plugin.terraform-provider-google_v2.20.0_x4: 2019/11/28 00:41:54 [ERR] plugin: plugin server: accept unix /tmp/plugin156205178: use of closed network connection
2019-11-28T00:41:54.402Z [DEBUG] plugin: plugin process exited: path=/home/deployuser/.terraform.d/plugins/terraform-provider-google_v2.20.0_x4

With 2.19.0 the same section looks like

2019/11/28 00:51:44 [TRACE] Graph after step *terraform.AttachStateTransformer:

module.edgenodes01.google_compute_instance_group.group[0] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[1] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[2] - *terraform.NodeValidatableResourceInstance
2019/11/28 00:51:44 [TRACE] Graph after step *terraform.TargetsTransformer:

module.edgenodes01.google_compute_instance_group.group[0] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[1] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[2] - *terraform.NodeValidatableResourceInstance
2019/11/28 00:51:44 [DEBUG] ReferenceTransformer: "module.edgenodes01.google_compute_instance_group.group[0]" references: []
2019/11/28 00:51:44 [DEBUG] ReferenceTransformer: "module.edgenodes01.google_compute_instance_group.group[1]" references: []
2019/11/28 00:51:44 [DEBUG] ReferenceTransformer: "module.edgenodes01.google_compute_instance_group.group[2]" references: []
2019/11/28 00:51:44 [TRACE] Graph after step *terraform.ReferenceTransformer:

module.edgenodes01.google_compute_instance_group.group[0] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[1] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[2] - *terraform.NodeValidatableResourceInstance
2019/11/28 00:51:44 [TRACE] Graph after step *terraform.RootTransformer:

module.edgenodes01.google_compute_instance_group.group[0] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[1] - *terraform.NodeValidatableResourceInstance
module.edgenodes01.google_compute_instance_group.group[2] - *terraform.NodeValidatableResourceInstance
root - terraform.graphNodeRoot
  module.edgenodes01.google_compute_instance_group.group[0] - *terraform.NodeValidatableResourceInstance
  module.edgenodes01.google_compute_instance_group.group[1] - *terraform.NodeValidatableResourceInstance
  module.edgenodes01.google_compute_instance_group.group[2] - *terraform.NodeValidatableResourceInstance
2019/11/28 00:51:44 [TRACE] dag/walk: added new vertex: "module.edgenodes01.google_compute_instance_group.group[1]"
2019/11/28 00:51:44 [TRACE] dag/walk: added new vertex: "module.edgenodes01.google_compute_instance_group.group[2]"
2019/11/28 00:51:44 [TRACE] dag/walk: added new vertex: "root"
2019/11/28 00:51:44 [TRACE] dag/walk: added new vertex: "module.edgenodes01.google_compute_instance_group.group[0]"
2019/11/28 00:51:44 [TRACE] dag/walk: added edge: "root" waiting on "module.edgenodes01.google_compute_instance_group.group[1]"
2019/11/28 00:51:2019/11/28 00:51:44 [DEBUG] Setting Unknown Variable Value for computed key: backends.0.group
2019/11/28 00:51:44 [DEBUG] Setting Unknown Variable Value for computed key: backends.1.group
2019/11/28 00:51:44 [DEBUG] Setting Unknown Variable Value for computed key: backends.2.group
2019/11/28 00:51:44 [DEBUG] Setting Unknown Variable Value for computed key: internal_backends.0.group
2019/11/28 00:51:44 [DEBUG] Setting Unknown Variable Value for computed key: internal_backends.1.group
2019/11/28 00:51:44 [DEBUG] Setting Unknown Variable Value for computed key: internal_backends.2.group
2019/11/28 00:51:44 [TRACE] root: eval: *terraform.EvalCoerceMapVariable
2019/11/28 00:51:44 [DEBUG] Resource state not found for "module.edgenodes01-AppInt.google_compute_region_backend_service.default": module.edgenodes01-AppInt.google_compute_region_backend_service.default
2019/11/28 00:51:44 [DEBUG] ReferenceTransformer: "module.edgenodes01-AppInt.google_compute_region_backend_service.default" references: []
2019/11/28 00:51:44 [INFO] backend/local: plan calling Refresh
�[0m�[1mRefreshing Terraform state in-memory prior to plan...�[0m
2019/11/28 00:51:44 [INFO] terraform: building graph: GraphTypeRefresh
2019/11/28 00:51:44 [TRACE] No managed resources in state during refresh, skipping managed resource transformer
2019/11/28 00:51:44 [TRACE] ConfigTransformer: Starting for path: []
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
�[0m

@stevenaldinger
Copy link

This is happening to us as well. We're trying to upgrade from 1.17.1 to 2.20.0 with terraform code we know works and running into this issue. We're using Terraform v0.11.11

@slevenick
Copy link
Collaborator

@billyfoss and/or @stevenaldinger can you provide configs that are causing the failure?

Sorry for taking so long to get back on this, there seem to be several different reports of issues around backend.group in here

@billyfoss
Copy link

@slevenick I recreated the issue this evening. I'll have to see if I can find a smaller segment that recreates the issue. I remember doing quite a bit to get the logs I copied above.

@danawillow
Copy link
Contributor

@emilymye could this be related to the other regionbackendservice stuff?

@billyfoss
Copy link

I ran into #5218 which required me to update to 2.20, but this issue was breaking the move to 2.20.

Thanks to @danawillow for helping me workaround by using the google-beta provider at 2.20 on only the resources that needed the newer version while remaining on 2.19 for the remaining resources until this is fixed. Details of the workaround are described at: #5218 (comment)

@slevenick
Copy link
Collaborator

Hey @billyfoss if you move to terraform 0.12 do you still notice the same error? I'm wondering if this is a similar issue to #3937 which may show up differently because they are using 0.12

@marianne-butaye
Copy link

Hello, for my part I couldn't test with 0.12, but this is what I have seen:

  • when I pass directly a variable reference into the backends, it works
backend = ["${var.backends}"]
  • when I pass the variables into dictionaries declared directly, it works
backend {
        group                        = "${element(data.google_compute_instance_group.instance_groups.*.self_link, count.index)}"
    description                  = "${trimspace(lookup(var.backends[count.index], "description", ""))}"
    balancing_mode               = "${trimspace(lookup(var.backends[count.index], "balancing_mode", "UTILIZATION"))}"
    max_utilization              = "${trimspace(lookup(var.backends[count.index], "max_utilization", 0.8))}"
    max_rate                     = "${trimspace(lookup(var.backends[count.index], "max_rate", 0))}"
    max_rate_per_instance        = "${trimspace(lookup(var.backends[count.index], "max_rate_per_instance", 0))}"
    max_connections              = "${trimspace(lookup(var.backends[count.index], "max_connections", 0))}"
    max_connections_per_instance = "${trimspace(lookup(var.backends[count.index], "max_connections_per_instance", 0))}"
    capacity_scaler              = "${trimspace(lookup(var.backends[count.index], "capacity_scaler", 1))}"
    }
  • when I pass the variables into dictionaries declared directly inside a list, it works
backend = [
{
        group                        = "${element(data.google_compute_instance_group.instance_groups.*.self_link, count.index)}"
    description                  = "${trimspace(lookup(var.backends[count.index], "description", ""))}"
    balancing_mode               = "${trimspace(lookup(var.backends[count.index], "balancing_mode", "UTILIZATION"))}"
    max_utilization              = "${trimspace(lookup(var.backends[count.index], "max_utilization", 0.8))}"
    max_rate                     = "${trimspace(lookup(var.backends[count.index], "max_rate", 0))}"
    max_rate_per_instance        = "${trimspace(lookup(var.backends[count.index], "max_rate_per_instance", 0))}"
    max_connections              = "${trimspace(lookup(var.backends[count.index], "max_connections", 0))}"
    max_connections_per_instance = "${trimspace(lookup(var.backends[count.index], "max_connections_per_instance", 0))}"
    capacity_scaler              = "${trimspace(lookup(var.backends[count.index], "capacity_scaler", 1))}"
    },
]
  • but when I try to use a data source such as null_data_source, I get the "backend.0.group" error, even though the content should be exactly the same as above
backend = ["${data.null_data_source.backends.*.outputs}"]

(in my example, the content of data.null_data_source.backends.*.outputs is exactly a copy paste of the examples above, just moved from the data source to the resource directly.

In my case this problem is blocking, as I need the "INTERNAL_MANAGED" feature of the resource which has been added from v2.20 in beta and is not available in v2.19 (which has no backends broken).

modular-magician added a commit to modular-magician/terraform-provider-google that referenced this issue Nov 30, 2021
Co-authored-by: upodroid <[email protected]>
Co-authored-by: Cameron Thornton <[email protected]>
Signed-off-by: Modular Magician <[email protected]>
modular-magician added a commit that referenced this issue Nov 30, 2021
Co-authored-by: upodroid <[email protected]>
Co-authored-by: Cameron Thornton <[email protected]>
Signed-off-by: Modular Magician <[email protected]>

Co-authored-by: upodroid <[email protected]>
Co-authored-by: Cameron Thornton <[email protected]>
@github-actions github-actions bot added the forward/review In review; remove label to forward label Oct 25, 2023
@pawelJas
Copy link

@rileykarson The breaking change was introduced 4 major releases ago, can we close this bug as probably nobody is using 2.20 version anymore as well as the group parameter is indeed required by the Google Backend Service?

@slevenick
Copy link
Collaborator

That seems reasonable

Copy link

github-actions bot commented Sep 2, 2024

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests