resource/aws_lamba_function - possible regression w/3.17.0 on GovCloud

[pjaudiomv]

After the release of 3.17.0 all my pipelines using lambdas started to fail. I suspect this has to do with the new lambda code signing resource/data source however I am not using this or any new feature from current release.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform 0.13.5
AWS Provider 3.17.0

Affected Resource(s)

• aws_lambda_function

Terraform Configuration Files

data archive_file test_lambda {
  type        = "zip"
  source_file = "test.py"
  output_path = "test.zip"
}

resource aws_lambda_function test {
  filename                       = data.archive_file.test_lambda.output_path
  function_name                  = "test-lamb"
  role                           = aws_iam_role.test_lambda.arn
  handler                        = "test.lambda_handler"
  source_code_hash               = data.archive_file.test_lambda.output_base64sha256
  runtime                        = "python3.8"
}

data aws_iam_policy_document test_lambda_assume_role {
  statement {
    effect  = "Allow"
    actions = ["sts:AssumeRole"]

    principals {
      type        = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }
  }
}

resource aws_iam_role test_lambda {
  name               = "test-event-lambda-iam-role"
  description        = "test event role for lambda"
  assume_role_policy = data.aws_iam_policy_document.test_lambda_assume_role.json
}

data aws_iam_policy_document test_lambda {
  statement {
    effect = "Allow"
    actions = ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"]
    resources = ["*"]
  }
}

resource aws_iam_role_policy test_lambda {
  name   = "test-event-lambda-iam-policy"
  role   = aws_iam_role.test_lambda.id
  policy = data.aws_iam_policy_document.test_lambda.json
}

Debug Output

https://gist.github.com/pjaudiomv/775454e354ed98776cce6b191b7d019a

Panic Output

Expected Behavior

the provider should process my lambda without error

Actual Behavior

terraform dies with error

Error: error getting Lambda Function (test-lambda) code signing config AccessDeniedException: 
	status code: 403, request id: fghce9-fhjde1-4394-8e11-04fhjd174612

Steps to Reproduce

1. Run a plan or apply with above code snippet on GovCloud

Important Factoids

This happens on a plan or an apply and did not happen previous to 3.17.0 release.

References

• #16384

[rayjlinden]

Terraform has also blown up for me when using localstack. However, I get a different error of 404 because the CodeSigning related endpoints do not exist in Localstack.

Is there a way to by-pass this checking?

[rayjlinden]

I am able to work around this issue by pinning to last version of the provider:

terraform {
  required_providers {
    aws = "<= 3.16.0"
  }
}

[dawidmalina]

I have same issue with AWS China and I can confirm that pinning to 3.16.0 solve the issue.

[bflad]

Please note that we have a fix for this pending (#16412), we are just working with the Lambda service team to confirm some details before merging. We hope to have this resolved in some manner later today or tomorrow. 👍

[bflad]

The fix for this resource to (for now) skip the problematic API call in AWS GovCloud (US) and AWS China has been merged and will release with version 3.18.0 of the Terraform AWS Provider, likely in the next 24 hours.

For folks using LocalStack, we will probably need a separate fix due to the implementation. Please create a new bug report following the issue template (most importantly with the full terraform apply error output) and we can take a look at resolving that issue.

[rayjlinden]

Damn. At least I have the workaround of using a pinned version of aws provider. The bug is obvious if you even attempt to load a lambda into localstack - which clearly must not be part of your test suite. Do you have any testing with localstack? In any case, if simply complains it is getting 404 errors for the new endpoints. I’ll try to create a new issue this evening. Of course, I tried yesterday and the template never let me submit.... guess I’ll debug that next...

…

On Tue, Nov 24, 2020 at 1:35 PM Brian Flad ***@***.***> wrote: The fix for this resource to (for now) skip the problematic API call in AWS GovCloud (US) and AWS China has been merged and will release with version 3.18.0 of the Terraform AWS Provider, likely in the next 24 hours. For folks using LocalStack, we will probably need a separate fix due to the implementation. Please create a new bug report following the issue template (most importantly with the full terraform apply error output) and we can take a look at resolving that issue. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#16398 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKE5LWUNPFZ5KRLFXDPZIVTSRQRLZANCNFSM4UAKRHDA> .

[ghost]

This has been released in version 3.18.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[kyleckf]

We have just pulled 3.18 but are still getting the same error. We are moving back to 3.16 in the meantime.

[frostymelonade]

Can confirm we are still getting error in US East 1 with 3.18

[thomasmoormann1903]

Same with eu-central-1. 3.16 works

[pjaudiomv]

3.18.0 works with GovCloud

[shellac85]

Where do I change the provider.aws: version? Its not in out Terraform code and I didn't set our environment up. Any help greatly appreciated

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!