Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Access token has expired or is not yet valid #918

Closed
alejandro-gallardo-ntt opened this issue Oct 31, 2022 · 8 comments · Fixed by #1100
Closed

Access token has expired or is not yet valid #918

alejandro-gallardo-ntt opened this issue Oct 31, 2022 · 8 comments · Fixed by #1100

Comments

@alejandro-gallardo-ntt
Copy link

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritise this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritise the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureAD Provider) Version

Terraform v1.3.2

provider registry.terraform.io/hashicorp/azuread v2.29.0
provider registry.terraform.io/hashicorp/azurerm v3.16.0

Affected Resource(s)

  • azuread_XXXXX

Terraform Configuration Files

resource "azurerm_mssql_managed_instance" "mssqlmi" {
  name                = var.instance_name
  resource_group_name = var.resource_group_name
  location            = var.location
  subnet_id           = var.subnet_id

  license_type       = var.license_type
  sku_name           = var.sku_name
  storage_size_in_gb = var.storage_size_in_gb
  vcores             = var.vcores

  administrator_login          = var.sql_admin_login
  administrator_login_password = var.sql_admin_password

  identity {
    type = "SystemAssigned"
  }

  tags = var.tags
}

resource "azuread_directory_role" "reader" {
  count        = var.ad_admin_enabled ? 1 : 0
  display_name = "Directory readers"
}

resource "azuread_directory_role_assignment" "ra_mssqlmi" {
  count = var.ad_admin_enabled ? 1 : 0

  role_id             = azuread_directory_role.reader[0].template_id
  principal_object_id = azurerm_mssql_managed_instance.mssqlmi.identity.0.principal_id

}

resource "azurerm_mssql_managed_instance_active_directory_administrator" "adadmin_mssqlmi" {
  count                       = var.ad_admin_enabled ? 1 : 0
  managed_instance_id         = azurerm_mssql_managed_instance.mssqlmi.id
  login_username              = var.ad_admin_login_name
  object_id                   = var.ad_admin_object_id
  tenant_id                   = var.ad_admin_tenant_id
  azuread_authentication_only = var.ad_azuread_auth_only
  depends_on = [
    time_sleep.wait_30_seconds
  ]
}

Description

Happening in this particular case with azuread_directory_role_assignment, but it could happen with any AzureAD resource.

When creating a SQL Managed Instance and the corresponding Role Assignment, this last one fails because the Managed Instance creation lasts almost 4 hours and it seems like by then the AzureAD token is expired:

module.mssql-managed.azurerm_mssql_managed_instance.mssqlmi: Creation complete after 3h56m33s [id=/subscriptions/1341fec5-a980-41e9-a3b8-e99f509917aa/resourceGroups/rg-test-sqlmi-8877ed690f8725c0/providers/Microsoft.Sql/managedInstances/managedsqlinstance8877ed690f8725c0]
module.mssql-managed.azuread_directory_role_assignment.ra_mssqlmi[0]: Creating...
│ Error: Assigning directory role "88d8e3e3-8f55-4a1e-953a-9b9898b8876b" to directory principal "b0e175de-8013-4949-af5e-a8f849d30270", received 401 with error: RoleAssignmentsClient.BaseClient.Post(): unexpected status 401 with OData error: InvalidAuthenticationToken: Access token has expired or is not yet valid.
│ 
│   with module.mssql-managed.azuread_directory_role_assignment.ra_mssqlmi[0],
│   on ../../main.tf line 49, in resource "azuread_directory_role_assignment" "ra_mssqlmi":
│   49: resource "azuread_directory_role_assignment" "ra_mssqlmi" {
│ 
│ RoleAssignmentsClient.BaseClient.Post(): unexpected status 401 with OData error: InvalidAuthenticationToken: Access token has expired or is not yet valid.
╵

When running terraform apply again once the Managed Instance is created, authentication happens and right after the Role Assignment is created without problem.

Expected Behavior

Role assignment succesully created.

Actual Behavior

Role assignment creation fails due to expired token.

Steps to Reproduce

  1. terraform apply

Important Factoids

References

  • #0000
@manicminer
Copy link
Contributor

Hi @alejandro-gallardo-ntt, thanks for reporting this! The provider is supposed to renew access tokens without interrupting a long-running apply operation, so this could well be a bug in how we are handling that.

Please can you advise the authentication method you're using when this happens? Also, if you have a debug log that includes the requests and responses leading to this error, that would be incredibly helpful - though I understand this is challenging given the provisioning times involved.

@alejandro-gallardo-ntt
Copy link
Author

Hi @manicminer ,

I´m authenticating via Azure CLI.

I have some requests and responses from previous executions, Times may not match because they are from different runs, but I hope it helps:

Token claim just after running terraform apply:

2022-10-21T07:54:43.257+0100 [INFO]  provider.terraform-provider-azuread_v2.24.0_x5: 2022/10/21 07:54:43 [DEBUG] AzureAD Provider access token claims: {"aud":"https://graph.microsoft.com","iss":"https://sts.windows.net/xxxxxxxxxxxxxxxxxxxxxxxxxxxxx/","idp":"https://sts.windows.net/xxxxxxxxxxxxxxxxxxxxxxxxxxxxx/","oid":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","roles":null,"scp":"AuditLog.Read.All Directory.AccessAsUser.All email Group.ReadWrite.All openid profile User.ReadWrite.All","sub":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","tenant_region_scope":"EU","tid":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","ver":"1.0","app_displayname":"Microsoft Azure CLI","appid":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","idtyp":"user"}: timestamp=2022-10-21T07:54:43.256+0100

And the API call after the Managed Instance is created:

[DEBUG] ============================ Begin AzureAD Request ============================
Request ID: e11aa914-2fd9-4e18-ff39-074b37fd4d65

GET /v1.0/xxxxxxxxxxxxxxxxxxxxxxxxxx/directoryRoles/88d8e3e3-8f55-4a1e-953a-9b9898b8876b HTTP/1.1
Host: graph.microsoft.com
User-Agent: HashiCorp Terraform/1.1.5 (+https://www.terraform.io) Terraform Plugin SDK/2.10.1 terraform-provider-azuread/dev Hamilton (Go-http-client/1.1) pid-222c6c49-1b0a-5959-a213-6608f9eb8820
Accept: application/json; charset=utf-8; IEEE754Compatible=false
Content-Type: application/json; charset=utf-8
Odata-Maxversion: 4.0
Odata-Version: 4.0
Accept-Encoding: gzip
============================= End AzureAD Request =============================: timestamp=2022-10-21T00:51:47.634+0100

2022-10-21T00:51:47.987+0100 [INFO]  provider.terraform-provider-azuread_v2.24.0_x5: 2022/10/21 00:51:47 

[DEBUG] ============================ Begin AzureAD Response ===========================
GET https://graph.microsoft.com/v1.0/xxxxxxxxxxxxxxxxxxxxxxxxxxxx/directoryRoles/88d8e3e3-8f55-4a1e-953a-9b9898b8876b
Request ID: e11aa914-2fd9-4e18-ff39-074b37fd4d65

HTTP/1.1 401 Unauthorized
Transfer-Encoding: chunked
Client-Request-Id: 7e7186c4-6071-4fea-8555-5618f2b3c5ed
Content-Type: application/json
Date: Thu, 20 Oct 2022 23:51:46 GMT
Request-Id: 7e7186c4-6071-4fea-8555-5618f2b3c5ed
Strict-Transport-Security: max-age=31536000
Vary: Accept-Encoding
Www-Authenticate: Bearer realm="", authorization_uri="https://login.microsoftonline.com/common/oauth2/authorize", client_id="00000003-0000-0000-c000-000000000000"
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"North Europe","Slice":"E","Ring":"4","ScaleUnit":"002","RoleInstance":"DU2PEPF000135B8"}}

104
{"error":{"code":"InvalidAuthenticationToken","message":"Access token has expired or is not yet valid.","innerError":{"date":"2022-10-20T23:51:47","request-id":"7e7186c4-6071-4fea-8555-5618f2b3c5ed","client-request-id":"7e7186c4-6071-4fea-8555-5618f2b3c5ed"}}}
0


============================= End AzureAD Response ============================

If you need more detailed logs or anything else, please let me know. I could create another instance.

@manicminer
Copy link
Contributor

@alejandro-gallardo-ntt Thanks! This should be enough for me to try out a few things.

@LaurentLesle
Copy link
Contributor

LaurentLesle commented May 10, 2023

@manicminer I am facing the same issue when destroying a sqlmi cluster with 1 replica + failover group + 2 DBs. The destroy process is taking more than 1 hour and one of the last activity I guess is to remove the directory reader role to the SQL MI instances or msi. Looks like the access token was not refreshed.

Test was done with logged in user with cli flow

2023-05-10T08:14:52.494Z [ERROR] provider.terraform-provider-azuread_v2.38.0_x5: Response contains error diagnostic: tf_req_id=740f1b0a-7cef-9c31-42c4-de3f8559c88a tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_detail="RoleAssignments.BaseClient.Get(): unexpected status 401 with OData error: InvalidAuthenticationToken: Access token has expired or is not yet valid." diagnostic_severity=ERROR diagnostic_summary="Deleting role assignment "4-PYiFWPHkqVOpuYmLiHa4w9myjwMCdNg5W3fmOK0YA-1": RoleAssignments.BaseClient.Get(): unexpected status 401 with OData error: InvalidAuthenticationToken: Access token has expired or is not yet valid." tf_proto_version=5.3 tf_provider_addr=provider tf_resource_type=azuread_directory_role_assignment @module=sdk.proto timestamp=2023-05-10T08:14:52.494Z
image

@manicminer
Copy link
Contributor

Thanks @LaurentLesle, I believe this is related to hashicorp/terraform-provider-azurerm#20867 and we should have a fix out this week.

@github-actions
Copy link

This functionality has been released in v2.39.0 of the Terraform Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@LaurentLesle
Copy link
Contributor

I can confirm it is fixed with version 2.39.0. Thanks!

@manicminer
Copy link
Contributor

Fixed with #1100

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.