Bug introduced v3.29.0 with RDS Validation and Snapshots

[tedivm]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

v3.29.0 for AWS Provider
v0.14.5 for Terraform

Affected Resource(s)

• aws_db_instance

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_db_instance" "main" {
  engine                                = "postgres"
  engine_version                        = var.engine_version
  apply_immediately                     = true
  identifier                            = "db-${var.name}-pg-instance"
  name                                  = var.database_name
  username                              = var.database_name
  password                              = random_password.password.result
  storage_encrypted                     = true
  allocated_storage                     = var.allocated_storage
  max_allocated_storage                 = var.max_allocated_storage
  instance_class                        = var.instance_class
  snapshot_identifier                   = var.snapshot_identifier
  storage_type                          = var.storage_type
  backup_retention_period               = var.backup_retention_period
  backup_window                         = var.backup_window
  maintenance_window                    = var.maintenance_window
  auto_minor_version_upgrade            = true
  final_snapshot_identifier             = "${var.name}-rds-pg-final-snapshot"
  multi_az                              = true
  vpc_security_group_ids                = [aws_security_group.main.id]
  db_subnet_group_name                  = aws_db_subnet_group.main.name
  parameter_group_name                  = length(var.db_parameters) > 0 ? aws_db_parameter_group.main[0].name : null

  monitoring_interval                   = 30
  monitoring_role_arn                   = aws_iam_role.monitor_role.arn

  performance_insights_enabled          = true
  performance_insights_retention_period = 7

  enabled_cloudwatch_logs_exports       = [
    "postgresql",
    "upgrade"
  ]

  deletion_protection                   = var.deletion_protection
  tags                                  = local.common_data_tags

  lifecycle {
    ignore_changes = [
      username
    ]
  }

  #depends_on = [aws_iam_policy_attachment.monitor_role]
}

Debug Output

Panic Output

Expected Behavior

Because we're passing null in for the snapshot we'd expect validation to succeed.

Actual Behavior

Validation fails because it does not take the null case into account.

Steps to Reproduce

1. Create a aws_db_instance resource that includes a username and passes in null for the snapshot id.
2. terraform apply

Important Factoids

This is a bug introduced in #17156

References

• RDS - Throw error when name or username supplied on DB snapshot restore #17156

[theophilechevalier]

We got the bug using username = "something" and snapshot_identifier = ""

[ewbankkit]

@tedivm Thanks for raising this issue.
By we're passing null in for the snapshot do you mean var.snapshot_identifier is null?

[tedivm]

Yes, that's what i mean.

[mrobinson513]

Running Terraform 0.13.6 and provider 3.29.0.

We've encountered this bug with username = "a_value" and snapshot_identifier = "", this exception was thrown:

Error: ConflictsWith

  on .terraform/modules/rds/main.tf line 71, in resource "aws_db_instance" "main":
  71:   snapshot_identifier = var.snapshot_identifier

"snapshot_identifier": conflicts with username

I just tested setting snapshot_identifier to null and it planned fine;. @tedivm what is your default value for variable snapshot_identifier in your module?

[tedivm]

The variable is defined here-

variable "snapshot_identifier" {
  description = "in case you are recovering from a snapshot."
  type        = string
  default     = null
}

Downstream we have a module using it, but the variable there is also set to null by default-

variable "hubble_snapshot_identifier" {
  type    = string
  default = null
}

The hubble_snapshot_identifier variable isn't being set where we're seeing this issue.

[tedivm]

I take that back, it turns out for this project they were specifying a string! Sorry for not checking that more thoroughly on my end.

[fractos]

@mrobinson513 Similar here. Hit this straight after 3.29 release as the default for snapshot_identifier in the terraform-aws-rds community module is "".

[tgonda-discovery]

Getting this error when both username and snapshot identifier are clearly set to different values.

[konstl000]

#17722 is similar, but not quite the same though. Arguably, passing "" as a snapshot_identifier together with a name is an invalid configuration, since "" is not null.
On the other hand, the example in #17722 shows that a valid config with a snapshot_identifier and no name at all causes the same error after the initial deployment (probably, because there is still the attribute name in the state, even if the config files don't have any name at all).

[sahulyousuf]

This is not just hitting V3 but also hitting v2 i.e. we are using v0.12.29 and all the our existing reources which were built with a predefined snapshots are failing too when running terragrunt plan since last two days.

it would be great if there is a retrofit for terraform 12 too.

[konstl000]

This is not just hitting V3 but also hitting v2 i.e. we are using v0.12.29 and all the our existing reources which were built with a predefined snapshots are failing too when running terragrunt plan since last two days.

it would be great if there is a retrofit for terraform 12 too.

I don't think, the version of Terraform is important here. The question is just if you use the buggy version of the plugin or not, I would say ...

[aleon1220]

is happenning to me now and i am on terraform cloud. I believe the conflict is in
the duplicate use of the var.snapshot_identifier

Take a look at line 63 and 139

https://github.com/terraform-aws-modules/terraform-aws-rds/blob/master/modules/db_instance/main.tf#L139

[aleon1220]

is happenning to me now and i am on terraform cloud. I believe the conflict is in
the duplicate use of the var.snapshot_identifier

Take a look at line 63 and 139

https://github.com/terraform-aws-modules/terraform-aws-rds/blob/master/modules/db_instance/main.tf#L139

[aleon1220]

I think the majority of the community is tracking the issue at #17712

[jareddohrman]

Getting this too when providing a value for snapshot_identifier

[jareddohrman]

Getting this too when providing a value for snapshot_identifier

Changing the following is a workaround for me in the interim

provider "aws" {
  version = "= 3.28.0"
}

[aleon1220]

In my case i was only creating a simple RDS PostgreSQL instance. So I just used the aws_db_instance resource and disabled the db module.

[kahootali]

Getting the same issue, didn't set snapshot_identifier as it would have picked the default value of ""

[kahootali]

Changing the following is a workaround for me in the interim

provider "aws" {
  version = "= 3.28.0"
}

^This one gives warning & is deprecated in terraform 0.14.x, use

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "= 3.28.0"
    }
  }
}

[coolapso]

Being hit by this issue, using provider version 3.29 and setting name to null when planning using snapshot_identifier

------------------------------------------------------------------------

Error: 1 error occurred:
	* name attribute is not supported with snapshot_identifier when engine is postgres



  on main.tf line 39, in resource "aws_db_instance" "rds_instance":
  39: resource "aws_db_instance" "rds_instance" {

[konstl000]

Being hit by this issue, using provider version 3.29 and setting name to null when planning using snapshot_identifier

------------------------------------------------------------------------

Error: 1 error occurred:
	* name attribute is not supported with snapshot_identifier when engine is postgres



  on main.tf line 39, in resource "aws_db_instance" "rds_instance":
  39: resource "aws_db_instance" "rds_instance" {

The instance is probably already there, right?

[xorxeli]

terraform -v
Terraform v0.14.7
+ provider registry.terraform.io/hashicorp/aws v3.29.0

Error: ConflictsWith

  on ..\modules\rds\main.tf line 62, in resource "aws_db_instance" "postgresql":
  62:   snapshot_identifier       = var.snapshot_identifier

"snapshot_identifier": conflicts with username

Same here

[neugeeug]

Same here, we have default value for snapshot_identifier=""

Error: ConflictsWith

  on .terraform/modules/rds-postgres-dev.rds/main.tf line 23, in resource "aws_db_instance" "main":
  23:   snapshot_identifier       = var.snapshot_identifier

"snapshot_identifier": conflicts with username

We are kind of blocked at the moment. When we can expect a fix for that ?

[coolapso]

Being hit by this issue, using provider version 3.29 and setting name to null when planning using snapshot_identifier

------------------------------------------------------------------------

Error: 1 error occurred:
	* name attribute is not supported with snapshot_identifier when engine is postgres



  on main.tf line 39, in resource "aws_db_instance" "rds_instance":
  39: resource "aws_db_instance" "rds_instance" {

The instance is probably already there, right?

Yes, the infrastructure is already deployed!

[konstl000]

Being hit by this issue, using provider version 3.29 and setting name to null when planning using snapshot_identifier

------------------------------------------------------------------------

Error: 1 error occurred:
	* name attribute is not supported with snapshot_identifier when engine is postgres



  on main.tf line 39, in resource "aws_db_instance" "rds_instance":
  39: resource "aws_db_instance" "rds_instance" {

The instance is probably already there, right?

Yes, the infrastructure is already deployed!

This is why null does not work anymore. You can even delete the name attribute altogether, still won't work because the provider seems to get the name attribute from the state and then flag it as an error.

[konstl000]

Same here, we have default value for snapshot_identifier=""

Error: ConflictsWith

  on .terraform/modules/rds-postgres-dev.rds/main.tf line 23, in resource "aws_db_instance" "main":
  23:   snapshot_identifier       = var.snapshot_identifier

"snapshot_identifier": conflicts with username

We are kind of blocked at the moment. When we can expect a fix for that ?

You can just use an older version of the provider as suggested by kahootali

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "= 3.28.0"
    }
  }
}

[mnazir23]

Hi,

I'm getting the same error

"snapshot_identifier": conflicts with username

In my case, snapshot_identifier has a value being passed to it. I am trying to replace my RDS with the new snapshot.

[coolapso]

Being hit by this issue, using provider version 3.29 and setting name to null when planning using snapshot_identifier

------------------------------------------------------------------------

Error: 1 error occurred:
	* name attribute is not supported with snapshot_identifier when engine is postgres



  on main.tf line 39, in resource "aws_db_instance" "rds_instance":
  39: resource "aws_db_instance" "rds_instance" {

The instance is probably already there, right?

Yes, the infrastructure is already deployed!

This is why null does not work anymore. You can even delete the name attribute altogether, still won't work because the provider seems to get the name attribute from the state and then flag it as an error.

@konstl000 which is exactly what this bug is about right? :)

With 3.29 ...

you cannot define a snapshot_identifier AND name, password

but like me, If you had previously spun up a database based on a snapshot and provided the name, now using the new provider version you get a conflict, if you then send null, you will get an error.

Not providing the snapshot_identifier is not even an option otherwise terraform will want to destroy the existing database and spin up a new one.

[konstl000]

Being hit by this issue, using provider version 3.29 and setting name to null when planning using snapshot_identifier

------------------------------------------------------------------------

Error: 1 error occurred:
	* name attribute is not supported with snapshot_identifier when engine is postgres



  on main.tf line 39, in resource "aws_db_instance" "rds_instance":
  39: resource "aws_db_instance" "rds_instance" {

The instance is probably already there, right?

Yes, the infrastructure is already deployed!

This is why null does not work anymore. You can even delete the name attribute altogether, still won't work because the provider seems to get the name attribute from the state and then flag it as an error.

@konstl000 which is exactly what this bug is about right? :)

With 3.29 ...

you cannot define a snapshot_identifier AND name, password

but like me, If you had previously spun up a database based on a snapshot and provided the name, now using the new provider version you get a conflict, if you then send null, you will get an error.

Not providing the snapshot_identifier is not even an option otherwise terraform will want to destroy the existing database and spin up a new one.

The bug is even meaner than that. Even if you do not provide any name at all and deploy an instance from a snapshot, the next attempt of plan will fail. This is what I am trying to say :-) Look at my example in #17722.

[rkferreira]

Hi,

Same here, considering default empty "snapshot_identifier" value and "username" is filled with something.

  on .terraform/modules/rds.db/modules/db_instance/main.tf line 61, in resource "aws_db_instance" "this":
  61:   snapshot_identifier = var.snapshot_identifier

"snapshot_identifier": conflicts with username```

[TrangPham]

Also happening on Terraform version 0.13.5

[xorxeli]

@TrangPham problem is related to AWS provider version not Terraform version

[ghost]

This has been released in version 3.29.1 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!