[Bug]: plan always report removing serverlessv2_scaling_configuration on aws_rds_cluster

[CyrilDevOps]

Terraform Core Version

1.4.6

AWS Provider Version

5.6.2

Affected Resource(s)

aws_rds_cluster

Expected Behavior

terraform should ignore serverlessv2_scaling_configuration when engine_mode is set to provisioned

Actual Behavior

terraform plan always popup :

  # module.databases["db1"].aws_rds_cluster.aurora will be updated in-place
  ~ resource "aws_rds_cluster" "aurora" {
      + apply_immediately                   = true
      + enable_global_write_forwarding      = false
      + final_snapshot_identifier           = "test"
        id                                  = "test"
      + master_password                     = (sensitive value)
      ~ skip_final_snapshot                 = true -> false
        # (34 unchanged attributes hidden)

      ~ serverlessv2_scaling_configuration {
          - max_capacity = 3 -> null
          - min_capacity = 0.5 -> null
        }
    }

even after applying it come back at the next plan

Relevant Error/Panic Output Snippet

I have a aurora 3.x cluster with two nodes (writer, read replica) based on r6g.large setuped by terraform.
I play a little directly in AWS to switch to serverless v2 to test it, then back to provisioned.

Now at every plan, terraform want to remove/change the serverlessv2_scaling_configuration section.

When I am doing aws rds describe-clusters, the cluster that I played with serverless return a section :

            "ServerlessV2ScalingConfiguration": {
                "MinCapacity": 0.5,
                "MaxCapacity": 3.0
            },

even if it isn't serverless anymore.
The same aws rds describe-clusters on a second cluster that I didn't touch (no serverless experiment on it) don't show the ServerlessV2ScalingConfiguration section.

Both cluster are the same in AWS, one writer, one reader r6g.large, mysql aurora 3.x.

Terraform Configuration Files

aws_rds_cluster and aws_rds_cluster_instance to create a Aurora 3.x 8.0.mysql_aurora.3.03. provisionned cluster.

Steps to Reproduce

play in AWS GUI to switch to serverless v2, and back to provisionned.
aws rds describe-clusters on the cluster should return an extra section ServerlessV2ScalingConfiguration even if it isn't serverless anymore.

reapply terraform plan from the original terraform script, it will start to push a change about serverlessv2_scaling_configuration section.

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[CyrilDevOps]

Hi,
After a few days, AWS support gave me this answer. The AWS provider need to be aware of it and handle it better. Today I have a cluster that popup a serverlessv2_scaling_configuration change in my plan.
AWS Support :
Hope you're doing well. I appreciate your patience and support while my internal team was investigating this issue. To begin with, we created a test cluster with Serverless instance to observe the behavior and later modified the instance to provisioned and, checking the console and CLI, we observed the same output as you where console was showing the min and max ACU and, output of CLI was as below:
"AutoMinorVersionUpgrade": false,
"ServerlessV2ScalingConfiguration": {
"MinCapacity": 0.5,
"MaxCapacity": 16.0
},
"NetworkType": "IPV4"
}
]
}

Upon further investigation, we found that this is an expected result with Aurora Serverless v2- similarly stated in [1] i.e. Any capacity range that you previously specified for the cluster remains in place, even if all Aurora Serverless v2 DB instances are removed from the cluster (although it does not mean that the cluster is still Serverless).

I understand that this behavior seems confusing and not expected however, it is only specific to Serverless v2 by design and I hope these details help. Please let me know if you have any questions or concerns and, I will be happy to address it further:)

[marekaf]

Hi! I have the same issue for almost a year now. I tried to discuss it with AWS Support in June 2023 and got a similar response.
They said they might reimplement it to behave differently but labeled it as a "feature", not a "bug". And didn't provide any ETA on this. Told me to "read AWS News and maybe it will appear someday".

Great.

[yarka-guru]

I had the same issue.

[mpekhota]

Same issue.

[ivannedyalkov]

Same issue

09:57:35 Terraform will perform the following actions: 09:57:35 09:57:35 # module.rds-global-aurora.module.primary_cluster.aws_rds_cluster.primary_cluster[0] will be updated in-place 09:57:35 ~ resource "aws_rds_cluster" "primary_cluster" { 09:57:35 ___ id________________ = "*****" 09:57:35 ___ tags_______________ = { 09:57:35 _____ "Environment"_ = "dev" 09:57:35 _____ "IAC_managed"_ = "true" 09:57:35 _____ "IAC_repository" = "*****" 09:57:35 _____ "Owner"____ = "*****" 09:57:35 _____ "Project"___ = "*****" 09:57:35 ___ } 09:57:35 ___ # (49 unchanged attributes hidden) 09:57:35 09:57:35 __ ~ serverlessv2_scaling_configuration { 09:57:35 ____ - max_capacity = 8 -> null 09:57:35 _____ # (1 unchanged attribute hidden) 09:57:35 ___ } 09:57:35 _ } 09:57:35 09:57:35 # module.rds-global-aurora.module.replica_cluster_1.aws_rds_cluster.secondary_cluster[0] will be updated in-place 09:57:35 ~ resource "aws_rds_cluster" "secondary_cluster" { 09:57:35 ___ id________________ = "*****" 09:57:35 ___ tags_______________ = { 09:57:35 _____ "Environment"_ = "dev" 09:57:35 _____ "IAC_managed"_ = "true" 09:57:35 _____ "IAC_repository" = "*****" 09:57:35 _____ "Owner"____ = "*****" 09:57:35 _____ "Project"___ = "*****" 09:57:35 ___ } 09:57:35 ___ # (44 unchanged attributes hidden) 09:57:35 09:57:35 __ ~ serverlessv2_scaling_configuration { 09:57:35 ____ - max_capacity = 8 -> null 09:57:35 _____ # (1 unchanged attribute hidden) 09:57:35 ___ } 09:57:35 _ } 09:57:35 09:57:35 Plan: 0 to add, 2 to change, 0 to destroy.

[tyrannosaurus-becks]

We're encountering this too.

It matters because we have a daily job that just runs a plan and apply on our workspace, to prevent drift where people manually change things but don't encode them in Terraform.

This matters because we would like to notify if we detect and wipe out drift. This problem means it would notify incorrectly every day. Because of that, we can't notify if our job wipes out drift, it just runs silently and can be hard to reason about.

[marekaf]

@tyrannosaurus-becks welcome to the club.
I'm currently celebrating 2 years looking at terraform drift on all our producting db clusters 🎉