aws_elasticsearch_domain Advanced options are not applied properly

[hashibot]

This issue was originally opened by @robertfirek as hashicorp/terraform#3980. It was migrated here as part of the provider split. The original body of the issue is below.

---

When tried to apply the following advanced option on aws_elasticsearch_domain

(...)
advanced_options {
    "rest.action.multi.allow_explicit_index" = true
}
(...)

it was not applied properly in amazon (rest.action.multi.allow_explicit_index option was set to false).

Changing this option doesn't change plan (I've changed true to false) and it is impossible to control options.

State file:

(...)
"advanced_options.#": "1",
"advanced_options.rest.action.multi.allow_explicit_index": "false", 
(...)

Terraform version:
v0.6.5

[hashibot]

This comment was originally opened by @radeksimko as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

I've got a hunch that this may be caused by the way we process the map for advanced_options:
https://github.com/hashicorp/terraform/blob/master/builtin/providers/aws/structure.go#L639

I'll try adding some extra logging and reproduce it.
Thanks for the report!

[hashibot]

This comment was originally opened by @radeksimko as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

@robertfirek You can try specifying options as strings in the meantime:

advanced_options {
    "rest.action.multi.allow_explicit_index" = "true"
}

[hashibot]

This comment was originally opened by @robertfirek as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

@radeksimko I've tried this approach before I wrote this issue. It doesn't work.

[hashibot]

This comment was originally opened by @gmorpheme as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

I'm having the same issue - have tried with both booleans and strings with no luck.

[hashibot]

This comment was originally opened by @radeksimko as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

After trying to reproduce this & running a few tests, I believe this is a core issue, described here hashicorp/terraform#2143

Tags were something that a user could choose/change, but we cannot choose the map keys here, so the bug is affecting this resource more significantly. advanced_options are completely ignored at the moment due to this bug.

[hashibot]

This comment was originally opened by @genofedanzo as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

Ran into this today. Would love a fix.

[hashibot]

This comment was originally opened by @genofedanzo as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

Also wanted to mention that from I testing I noticed that if you try to set advanced_options they end up being set to false For instance, rest.action.multi.allow_explicit_index normally defaults to true but if you happen to manually set this to true it ends up being set as false.

[hashibot]

This comment was originally opened by @genofedanzo as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

@chiefy My work around was to not include advanced_options in terraform. This made rest.action.multi.allow_explicit_index equal true (normally the default) and it looked like I could modify advanced_options in the AWS console without causing idempotent issues with the resource in terraform. Not ideal but for now it won't prevent me from deploying ES to all of our environments.

[hashibot]

This comment was originally opened by @wnkz as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

Also have this issue (Terraform v0.6.9)

[hashibot]

This comment was originally opened by @brentley as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

+1 for trackinng purposes.

[hashibot]

This comment was originally opened by @radeksimko as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

The work around until this gets fixed may include local-exec provisioner attached to this resource executing some AWS CLI commands to manage the advanced options.

It's probably not as portable as terraform code, but it does the job.

[hashibot]

This comment was originally opened by @chiefy as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

@genofedanzo 's fix worked for me (just not including advanced_options in the template)

[hashibot]

This comment was originally opened by @moesy as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

Confirmed as of today in v0.6.16 this bug still exists.

[hashibot]

This comment was originally opened by @jbryan as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

I observed today in 0.7.0. Not including advanced_options worked for me as well.

[hashibot]

This comment was originally opened by @radeksimko as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

I just checked 0.7.0 and it may not work exactly per the example in documentation, but dots in map keys are not the cause anymore.

By default we cast map values to strings before sending those to the API and (boolean) true is casted to (string) "1" which AWS API in turn translates into false.

The following example does set rest.action.multi.allow_explicit_index to true (i.e. works as expected):

resource "aws_elasticsearch_domain" "es" {
    domain_name = "tf-test"
    advanced_options {
        "rest.action.multi.allow_explicit_index" = "true"
        "indices.fielddata.cache.size" = 40
    }
}

I'll relabel this to resource bug as we can still fix the map conversion in the context of this resource (i.e. probably turn "1" to "true"), but there's a simple workaround for this bug now in 0.7. The other option would be to ask Amazon whether they can just take "1" as true 😃

[hashibot]

This comment was originally opened by @donnoman as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

still a problem in

› terraform --version
Terraform v0.8.5

[hashibot]

This comment was originally opened by @furhouse as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

I've just ran into this issue with v0.8.5 as well, the following didn't work for me either:

advanced_options {
    "rest.action.multi.allow_explicit_index" = "true"
}

[hashibot]

This comment was originally opened by @pracucci as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

The following works for us on terraform v. 0.8.8:

advanced_options {
    "rest.action.multi.allow_explicit_index" = "true"
}

[hashibot]

This comment was originally opened by @wjessop as hashicorp/terraform#3980 (comment). It was migrated here as part of the provider split. The original comment is below.

---

Can confirm that @pracucci's fix works on v0.8.6

[radeksimko]

Hi folks,
to provide some background:

"Advanced Options" need to be represented as map with varying value types (map[string]interface{}) as it may contain both boolean and integer per docs.

There's currently no way to express this in the core schema and TypeMap is represented as map[string]string by default, hence the HCL config mentioned in the first comments comes back as "1". We can't simply translate "1" to true without any context because some other options expect integer and listing option-by-option doesn't look like a very scalable solution as options (with different value types) will grow with time.

With that in mind I'm adding appropriate label.

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!