RST_STREAM exception every 24 hours for gcpSecretsManager externalsecrets

[MPV]

Starting this week, we have started getting these kinds of errors for secrets with the gcpSecretsManager backendType:

$ kubectl -n kubernetes-external-secrets logs -l app.kubernetes.io/name=kubernetes-external-secrets | grep "my-secret" | jq -r
{
  "level": 50,
  "message_time": "2021-06-03T12:06:23.731Z",
  "pid": 20,
  "hostname": "kubernetes-external-secrets-6cbb86d747-fntjz",
  "payload": {
    "code": 4,
    "details": "Deadline exceeded",
    "metadata": {
      "internalRepr": {},
      "options": {}
    },
    "note": "Exception occurred in retry method that was not classified as transient"
  },
  "msg": "failure while polling the secret my-namespace/my-secret"
}
{
  "level": 50,
  "message_time": "2021-06-03T12:07:33.745Z",
  "pid": 20,
  "hostname": "kubernetes-external-secrets-6cbb86d747-fntjz",
  "payload": {
    "code": 4,
    "details": "Deadline exceeded",
    "metadata": {
      "internalRepr": {},
      "options": {}
    },
    "note": "Exception occurred in retry method that was not classified as transient"
  },
  "msg": "failure while polling the secret my-namespace/my-secret"
}
{
  "level": 50,
  "message_time": "2021-06-03T12:08:17.442Z",
  "pid": 20,
  "hostname": "kubernetes-external-secrets-6cbb86d747-fntjz",
  "payload": {
    "code": 13,
    "details": "Received RST_STREAM with code 2 triggered by internal client error: read ETIMEDOUT",
    "metadata": {
      "internalRepr": {},
      "options": {}
    },
    "note": "Exception occurred in retry method that was not classified as transient"
  },
  "msg": "failure while polling the secret my-namespace/my-secret"
}

...for an ExternalSecret of this type:

apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
  name: my-secret
  namespace: my-namespace
spec:
  backendType: gcpSecretsManager
  data:
  - key: my-secret-in-gcp
    name: service-account.json
  projectId: my-project

[MPV]

(on version 7.2.1)

[moolen]

hmm, seems like multiple users reporting issues with google grpc services here: googleapis/nodejs-datastore#679.
tho, i'm not sure if this is related.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[github-actions]

This issue was closed because it has been stalled for 30 days with no activity.