oauth2: cannot fetch token: 400 Bad Request

[och10]

Hi,
I installed k8s-config-connector v1.28.0 on crcv1.15.
I have set serviceacount key as secret (from key.json file) in cnrm-system namespace.
Whatever the custom resource I try to instanciate in my k8s cluster I always get the following error from pod cnrm-controller-manager-0 :
{“severity”:“error”,“logger”:"controller-runtime.controller",“msg”:“Reconciler error”,“controller”:“storagebucket-controller”,“request”:"och-project-01-294710/och-test-gcp-k8s-01",“error”:"Update call failed: error fetching live state: error reading underlying resource: summary: Error when reading or editing Storage Bucket "och-test-gcp-k8s-01": Get "https://storage.googleapis.com/storage/v1/b/och-test-gcp-k8s-01?alt=json&prettyPrint=false\": oauth2: cannot fetch token: 400 Bad Request\nResponse: {"error":"invalid_grant","error_description":"Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim."}, detail: "}

Thanks

[jcanseco]

Hi @och10. By crc, are you referring to this? I'm not too familiar with crc, but I'm assuming it's a non-GKE Kubernetes distribution? Am I correct in understanding that you've followed the Config Connector installation instructions for non-GKE K8s distributions here?

[jcanseco]

"Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim."

Did some digging, and it seems like the above error typically arises when the system's clock is not properly synchronized. See this, this, and this (scroll down to the second invalid_grant row). The first and last link in particular mention possible solutions.

Can you please investigate if your system's clock is not properly synchronized and let us know if properly synchronizing the clock allows for the error to go away?

[och10]

Hi jcanseco, thanks for your reply.
I confirm that crc stands for codereadycontainer which is the same as minishift and you're right it's a non-GKE kubernetes distribution. I've followed the Config Connector installation instructions for non-GKE K8s distributions so I think configuration is fine. I will check the system clock even if I think it's already synchronized clock and I will try your proposed solutions. I'll keep you informed.

Thanks

[och10]

After synchronizing crc system clock all is fine, the issue is resolved.
Many thanks.

[jcanseco]

Glad to help!

[Abdoul-karim2023]

google_storage_bucket.default: Creating...
╷
│ Error: Post https://www.googleapis.com/storage/v1/b?alt=json&prettyPrint=false&project=ict-chamber-training: oauth2: cannot fetch token: 400 Bad Request
│ Response: {"error":"invalid_grant","error_description":"Invalid grant: account not found"}
│
│ with google_storage_bucket.default,
│ on main.tf line 43, in resource "google_storage_bucket" "default":
│ 43: resource "google_storage_bucket" "default" {
│
╵

[Abdoul-karim2023]

help

[darkokrstevski1976]

got same error on my side; it turns out that you must initially create a VPC and then the error dissapears.

│ Error: Post "https://storage.googleapis.com/storage/v1/b?alt=json&prettyPrint=false&project=platform-team-sandbox-h4h": oauth2: cannot fetch token: 400 Bad Request
│ Response: {
│ "error": "invalid_grant",
│ "error_description": "reauth related error (invalid_rapt)",
│ "error_uri": "https://support.google.com/a/answer/9368756",
│ "error_subtype": "invalid_rapt"
│ }
│
│ with google_storage_bucket.datastore_backup,
│ on datastore backup.tf line 29, in resource "google_storage_bucket" "datastore_backup":