Startup failure #278

derrickburns · 2020-02-07T08:11:28Z

I am stuck. Please help.

Happens on 3.0.0 and 3.1.0
Using Kubernetes 1.14.9 on Amazon EKS

│ > [email protected] start /app                                                                                                   │
│ > ./bin/daemon.js                                                                                                                                │
│ {"level":30,"time":1581063019955,"pid":19,"hostname":"kubernetes-external-secrets-5d75db88c4-5plmp","msg":"loading kube specs","v":1}            │
│ Error: Failed to get /openapi/v2 and /swagger.json: argument fn must be a function                                                               │
│     at /app/node_modules/kubernetes-client/lib/swagger-client.js:58:15                                                                           │
│     at processTicksAndRejections (internal/process/task_queues.js:93:5)                                                                          │
│     at async main (/app/bin/daemon.js:32:3)

The text was updated successfully, but these errors were encountered:

Efrat19 · 2020-02-23T09:37:34Z

happens to me too, on EKS 1.14.9- the pods enter a CrashLoopBackOff:

npm info it worked if it ends with ok
npm WARN npm npm does not support Node.js v12.13.0
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11.
npm WARN npm You can find the latest version at https://nodejs.org/
npm info using [email protected]
npm info using [email protected]
npm info lifecycle [email protected]~prestart: [email protected]
npm info lifecycle [email protected]~start: [email protected]

> [email protected] start /app
> ./bin/daemon.js

{"level":30,"time":1582450288738,"pid":17,"hostname":"external-secrets-kubernetes-external-secrets-6d44c4d8b7-w2tj4","msg":"loading kube specs","v":1}
Error: Failed to get /openapi/v2 and /swagger.json: argument fn must be a function
    at /app/node_modules/kubernetes-client/lib/swagger-client.js:58:15
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async main (/app/bin/daemon.js:32:3)
npm info lifecycle [email protected]~start: Failed to exec start script
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `./bin/daemon.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm timing npm Completed in 765ms

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/node/.npm/_logs/2020-02-23T09_31_28_821Z-debug.log

ghostsquad · 2020-02-25T21:59:03Z

Any update on this? Have you been able to find the reason for the crash or a workaround?

derrickburns · 2020-02-25T22:03:28Z

@ghostsquad I switched to sops.

keweilu · 2020-02-27T00:48:48Z

I couldn't reproduce the error on minikube with kubernetes 1.14.9 for both 3.0.0 and 3.1.0. Does it only happen for Kubernetes 1.14.9 on Amazon EKS? Are you able to curl your cluster api endpoint for path /openapi/v2 and /swagger.json?

krao-test · 2020-02-27T23:37:40Z

me too facing this issue... any input in solving this is appreciated..

npm info it worked if it ends with ok
npm WARN npm npm does not support Node.js v12.13.0
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11.
npm WARN npm You can find the latest version at https://nodejs.org/
npm info using [email protected]
npm info using [email protected]
npm info lifecycle [email protected]~prestart: [email protected]
npm info lifecycle [email protected]~start: [email protected]
 > [email protected] start /app 
> ./bin/daemon.js
{"level":30,"time":1582827747612,"pid":18,"hostname":"external-secrets-service-kubernetes-external-secrets-5f856mc8nk","msg":"loading kube specs","v":1}
Error: Failed to get /openapi/v2 and /swagger.json: argument fn must be a function
 at /app/node_modules/kubernetes-client/lib/swagger-client.js:58:15
 at processTicksAndRejections (internal/process/task_queues.js:93:5)
   at async main (/app/bin/daemon.js:32:3)
npm info lifecycle [email protected]~start: Failed to exec start script
 npm ERR! code ELIFECYCLE
 npm ERR! errno 1
 npm ERR! [email protected] start: `./bin/daemon.js`
npm ERR! Exit status 12020-02-27T18:22:27.686006917Z npm ERR! 
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm timing npm Completed in 669ms
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/node/.npm/_logs/2020-02-27T18_22_27_687Z-debug.log

Flydiverny · 2020-02-28T08:02:45Z

I'll see if I can reproduce in a fresh EKS cluster

Flydiverny · 2020-02-28T09:26:30Z

Created a new EKS cluster with kubernetes 1.14.9 and platform version eks.9.
external-secrets pod starts without any issues here, using all default values of the helm chart.

~/Code/private/kubernetes-external-secrets
λ helm3 ls
NAME       	NAMESPACE	REVISION	UPDATED                             	STATUS  	CHART                            	APP VERSION
ext-secrets	default  	1       	2020-02-28 10:17:16.985683 +0100 CET	deployed	kubernetes-external-secrets-3.1.0	3.1.0

~/Code/private/kubernetes-external-secrets 
λ kg po
NAME                                                       READY   STATUS    RESTARTS   AGE
ext-secrets-kubernetes-external-secrets-844664976f-vspkx   1/1     Running   0          8m23s

~/Code/private/kubernetes-external-secrets 
λ k logs ext-secrets-kubernetes-external-secrets-844664976f-vspkx
npm info it worked if it ends with ok
npm WARN npm npm does not support Node.js v12.13.0
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11.
npm WARN npm You can find the latest version at https://nodejs.org/
npm info using [email protected]
npm info using [email protected]
npm info lifecycle [email protected]~prestart: [email protected]
npm info lifecycle [email protected]~start: [email protected]

> [email protected] start /app
> ./bin/daemon.js

{"level":30,"time":1582881447847,"pid":19,"hostname":"ext-secrets-kubernetes-external-secrets-844664976f-vspkx","msg":"loading kube specs","v":1}
{"level":30,"time":1582881447929,"pid":19,"hostname":"ext-secrets-kubernetes-external-secrets-844664976f-vspkx","msg":"successfully loaded kube specs","v":1}
{"level":30,"time":1582881447929,"pid":19,"hostname":"ext-secrets-kubernetes-external-secrets-844664976f-vspkx","msg":"updating CRD","v":1}
{"level":30,"time":1582881447929,"pid":19,"hostname":"ext-secrets-kubernetes-external-secrets-844664976f-vspkx","msg":"Upserting custom resource externalsecrets.kubernetes-client.io","v":1}
{"level":30,"time":1582881447960,"pid":19,"hostname":"ext-secrets-kubernetes-external-secrets-844664976f-vspkx","msg":"successfully updated CRD","v":1}
{"level":30,"time":1582881447963,"pid":19,"hostname":"ext-secrets-kubernetes-external-secrets-844664976f-vspkx","msg":"starting app","v":1}
Fri, 28 Feb 2020 09:17:27 GMT kubernetes-client deprecated .getStream use .getObjectStream, see https://github.com/godaddy/kubernetes-client/blob/master/merging-with-kubernetes.md at lib/external-secret.js:40:10
{"level":30,"time":1582881447979,"pid":19,"hostname":"ext-secrets-kubernetes-external-secrets-844664976f-vspkx","msg":"successfully started app","v":1}
{"level":30,"time":1582881447979,"pid":19,"hostname":"ext-secrets-kubernetes-external-secrets-844664976f-vspkx","msg":"MetricsServer listening on port 3001","v":1}

Efrat19 · 2020-03-01T22:36:06Z

@Flydiverny Id love to help but I need some more details.
The error comes from this line in k8s-client source, and it hadn't change recently.
If you can add some explanation I can try to figure out whats going wrong

Flydiverny · 2020-03-02T00:24:17Z

Only thing I can think of for now is that for some reason the initialization of the kube client is off.
Ie some of these calls:
https://github.com/godaddy/kubernetes-external-secrets/blob/fb284550d5987cf995bf64ca5585250714bd0cc5/config/index.js#L24-L27

but the kubernetes-client error doesn't really give much to go on, was hoping to be able to reproduce this myself.

Do you have any limitations or so where external-secrets is running or mismatching kubernetes versions or something else that might be worth evaluating to try to reproduce the issue?

krao-test · 2020-03-02T04:21:56Z

@Efrat19 , @Flydiverny , there are two issues...

If system:anonymous user does not have access to query /openapi/ endpoint, we are running into this issue..
Issue with parsing the API spec returned by k8s API server... I am not sure what exactly is the issue here... I will have to check with my colleague who helped me in debugging this issue.

Efrat19 · 2020-03-02T22:09:06Z

@krao-test got that, Ill take a look on my side too

multi-io · 2020-03-03T10:55:33Z

I can work around it in our use case (we're not using external-secrets, just stumbled over this ticket) by applying this fix to the kubernetes-client library. Not sure how safe this is (because () => {} may not fulfil the contract of this component.getStream thing). We're not using the stream api (ie. watches, which I understand are deprecated anyway?) in our client code.

Would still be nice to know the exact circumstances that cause this problem, and then maybe provide a better solution.

--- a/lib/swagger-client.js
+++ b/lib/swagger-client.js
@@ -85,6 +85,9 @@ class Root extends Component {
     // Deprecate stream API.
     //
     if (endpoint.pathItem.get) {
+      if (component.getStream === undefined) {
+        component.getStream = () => {}
+      }
       const action = endpoint.pathItem.get['x-kubernetes-action']
       if (action === 'watch' || action === 'watchlist') {
         component.getStream = deprecate.function(

Flydiverny · 2020-03-03T11:23:12Z

getStream is used by external-secrets here
https://github.com/godaddy/kubernetes-external-secrets/blob/fb284550d5987cf995bf64ca5585250714bd0cc5/lib/external-secret.js#L37-L40 not really sure what the alternative is with kubernetes-client
Theres some deprecation notices for getStream and some alternatives presented here for that. These deprecations also reference 9.0.0 which so far has yet to happen, and the PR to merge with the kubernetes client for js has been stale for quite awhile kubernetes-client/javascript#244

idobry · 2020-03-05T12:22:15Z

@krao-test There's no system:anonymous in my EKS.
In addition, I was trying to provide external-secrets' service account with admin permissions and no luck - getting the same results.

@keweilu I was able to curl into my cluster api endpoint for path /openapi/v2 and /swagger.json

mycrEEpy · 2020-03-05T13:55:59Z

Same problem here on EKS 1.14.9 with platform version eks.2 and eks.8 and tested with external-secrets 2.2.0 up to 3.1.0

mycrEEpy · 2020-03-09T16:03:41Z

I found the last log line of an instance which had started successful a few days ago and was failing at some point.

2020-03-05T01:37:54.172680293Z {"level":50,"time":1583372274172,"pid":18,"hostname":"kubernetes-external-secrets-68c5477d48-xzzjc","type":"Error","stack":"Error: connect ETIMEDOUT 172.20.0.1:443\n at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1128:14)","errno":"ETIMEDOUT","code":"ETIMEDOUT","syscall":"connect","address":"172.20.0.1","port":443,"msg":"failure while polling the secret xxx/xxx","v":1}

Now it's not starting anymore with the same error mentioned above.

dmduggan · 2020-03-10T22:52:10Z

I found that this started happening after upgrading linkerd from 2.6.1 to 2.7.0. Downgrading linkerd fixes the issue. I tried this on a new cluster with 1.14.8/eks.9. Maybe this will help as a workaround or for finding a fix.

idobry · 2020-03-11T06:50:29Z

That's it you are right @dmduggan!
We deployed linkerd as POC so we will definitely remove it.

mycrEEpy · 2020-03-11T09:59:59Z

I found that this started happening after upgrading linkerd from 2.6.1 to 2.7.0. Downgrading linkerd fixes the issue. I tried this on a new cluster with 1.14.8/eks.9. Maybe this will help as a workaround or for finding a fix.

Hummm... that might be it, we also upgraded linkerd to 2.7.0 recently, but the pod is not running with a proxy, strange... I'll dig a little deeper on this.

cpretzer · 2020-03-12T05:11:08Z

@mycrEEpy @idobry @dmduggan I dug into this a bit and the underlying issue is that kubernetes-external-secret makes a request to /openapi/v2/swagger.json which was deprecated and removed in Kubernetes 1.14.

I enabled logging on the EKS apiserver, and I don't see any errors related to this in the logs.

I think that swagger-client shouldn't try to get swagger.json, because it returns a 404.

I took a look through some of the changes between Linkerd 2.6.1 and 2.7.0 and I nothing jumped out at me as to why the request would be handled differently.

Testing with both Linkerd 2.6.0 and 2.7.1, I used kubectl proxy to curl -v http://localhost:8001/openapi/v2/swagger.json and in both cases I get a 404 error:

*   Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 8001 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8001 (#0)
> GET /openapi/v2/swagger.json HTTP/1.1
> Host: localhost:8001
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Audit-Id: 72feab4b-c7e4-4f6f-9eec-907f2dc13cd3
< Content-Length: 487
< Content-Type: application/json
< Date: Thu, 12 Mar 2020 04:38:34 GMT
< 
{
  "paths": [
    "/apis",
    "/apis/",
    "/apis/apiextensions.k8s.io",
    "/apis/apiextensions.k8s.io/v1beta1",
    "/healthz",
    "/healthz/etcd",
    "/healthz/log",
    "/healthz/ping",
    "/healthz/poststarthook/crd-informer-synced",
    "/healthz/poststarthook/generic-apiserver-start-informers",
    "/healthz/poststarthook/start-apiextensions-controllers",
    "/healthz/poststarthook/start-apiextensions-informers",
    "/metrics",
    "/openapi/v2",
    "/version"
  ]
* Connection #0 to host localhost left intact

curl -v http://localhost:8001/openapi/v2 returns the expected JSON payload

Let me know if any of you are interested in debugging this together and we can set up a video call.

domdepasquale · 2020-03-16T15:16:58Z

Strange behavior, I stood up 2 EKS clusters. Both are version 1.15/eks.1 one has been around a few days longer.

The older cluster is having issues starting KES. In the brand new one (as of today), KES came right up...

Started yet another cluster, and KES couldn't start. Can't seem to find rhyme/reason.

mycrEEpy · 2020-03-21T11:59:26Z

I'm not a nodejs developer but i managed to get a stack trace of the problem:

"TypeError: argument fn must be a function
    at Function.wrapfunction [as function] (/app/node_modules/depd/index.js:415:11)
    at Root._addEndpoint (/app/node_modules/kubernetes-client/lib/swagger-client.js:102:49)
    at /app/node_modules/swagger-fluent/lib/loader.js:149:14
    at Array.forEach (<anonymous>)
    at Root._addSpec (/app/node_modules/swagger-fluent/lib/loader.js:148:10)
    at /app/node_modules/kubernetes-client/lib/swagger-client.js:54:14
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async main (/app/bin/daemon.js:32:3)"

Seems like it's crashing at the deprecation of the stream api in swagger-client.js:102:49 while iterating over the endpoints from /openapi/v2

Anyone here to whom this is helpful?

Flydiverny · 2020-03-22T01:40:46Z

Might be helpful to have the spec from curl -v http://localhost:8001/openapi/v2 on a failing install.
From the stacktrace and reading the code it indeed looks like it fails when deprecating the getStream call here https://github.com/godaddy/kubernetes-client/blob/0f9ec26b381c8603e7727c3346edb35e1db2deb1/lib/swagger-client.js#L102

The error is thrown from here https://github.com/dougwilson/nodejs-depd/blob/master/index.js#L414-L416 so the argument here being component.getStream is apparently not a function in this case. So for some endpoint thats being iterated over the getStream function is not properly setup

So this is when its adding the spec so the error message of Failed to get /openapi/v2 and /swagger.json is abit confusing, as it has actually gotten the /openapi/v2 but fails to process it.

Even better if we could see which endpoint causes an issue and maybe see what goes wrong here and propose an update to kubernetes-client or so.

@silasbw Any thoughts on what might be causing this?

mycrEEpy · 2020-03-22T10:20:26Z

@Flydiverny here is the spec: https://gist.github.com/mycrEEpy/aa64cb5e9303870401f3029aa89cb056

I'll see if I can find out which endpoint causes the issue.

mycrEEpy · 2020-03-22T10:42:13Z

@Flydiverny it starts failing on

Endpoint {
  name: '/apis/tap.linkerd.io/v1alpha1',
  splits: [ 'apis', 'tap.linkerd.io', 'v1alpha1' ],
  pathItem: {
    get: {
      description: 'get available resources',
      consumes: [Array],
      produces: [Array],
      responses: [Object]
    }
  }
}

davidcorbin · 2020-03-23T15:13:30Z

Also running into this issue after installing in a namespace with Linkerd 2.7.0.

tzilistgoop · 2020-03-26T07:41:55Z

@Flydiverny Having the same issue here as well. I can confirm uninstalling linkerd, and allowing the service to start, then reinstalling linkerd seems to work, so if you could dig in when you have some time, that would be amazing!

davidcorbin · 2020-03-28T20:05:00Z

Dug into this issue a bit as well and seem to have discovered the issue and have a fix. The issue seems to be from the handling of URLs in silasbw/swagger-fluent, a dependency of godaddy/kubernetes-client. The issue starts on the the Endpoint that @mycrEEpy mentioned; the first linkerd endpoint that is found in the array of Endpoints.

Here's a patch that seems to fix the issue but I'd like someone else to test before putting in a PR to swagger-fluent. I'm not a js dev and am not very familiar with these individual repos so please recommend any changes. 👍🏻

From 7d0f67c1bd82e1a5f991b633e6117f983187c660 Mon Sep 17 00:00:00 2001
From: David Corbin <[email protected]>
Date: Sat, 28 Mar 2020 15:43:17 -0400
Subject: [PATCH] fix github issue

---
 lib/loader.js | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/loader.js b/lib/loader.js
index 98aec05..4713bff 100644
--- a/lib/loader.js
+++ b/lib/loader.js
@@ -114,6 +114,7 @@ class Component {
       .map(component => component.slice(1, -1))

     return pathnameParameterNames.reduce((acc, value, index) => {
+      if (!(value in acc)) return ""
       acc[value] = this.parameters[index]
       return acc
     }, {})
@@ -220,7 +221,7 @@ class Component {
     // "Expose" operations by omitting the leading _ from the method name.
     //
     Object.keys(endpoint.pathItem)
-      .filter(key => endpoint.pathItem[key].operationId)
+      .filter(key => "operationId" in endpoint.pathItem[key]? endpoint.pathItem[key].operationId : endpoint.pathItem[key])
       .forEach(method => {
         component[method] = component['_' + method]
         if (method === 'get') component.getStream = component._getStream
--
2.26.0

cc: @silasbw

davidcorbin · 2020-03-28T20:09:39Z

Also, you can easily reproduce this issue locally with minikube.

Install minikube and minikube start
Install latest version of linkerd (2.7.0) and linkerd install | kubectl apply -f - to install linkerd in minikube
Clone this repo, npm install, npm run nodemon

domdepasquale · 2020-04-01T19:16:39Z

To get back to a working state, I downgraded linkerd to 2.6.1, then deleted and reinstalled the 3 apiservices: v1alpha1.tap.linkerd.io v1alpha1.linkerd.io v1alpha2.linkerd.io.

Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.5", GitCommit:"0e9fcb426b100a2aea5ed5c25b3d8cfbb01a8acf", GitTreeState:"clean", BuildDate:"2019-08-05T09:13:08Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

KES version: 3.2.0

cpretzer · 2020-04-11T23:09:24Z

One of my colleagues dug into this and found that we can address this by adding an operationID field so that swagger-fluent will be satisfied.

We've got a pull request in progress: linkerd/linkerd2#4245

wmorgan · 2020-04-12T18:28:09Z

@cpretzer is there a workaround in the meanwhile? Or do people experiencing this issue have to wait for the next edge/stable Linkerd release?

mycrEEpy · 2020-04-12T18:34:44Z

@wmorgan current known workaround is a downgrade to linkerd 2.6.1

wmorgan · 2020-04-12T18:37:54Z

@mycrEEpy yeah I hate that workaround :)

mycrEEpy · 2020-04-12T19:45:53Z

I would also prefer something different :/

muenchdo · 2020-04-24T09:55:15Z

I'm slightly confused from reading through the comments here. Would this affect me if I have Linkerd deployed in my cluster, but external-secrets not being part of the mesh? If so, can someone explain why this would be the case?

mycrEEpy · 2020-04-24T11:41:26Z

@muenchdo linkerd registers an apiservice without an operationId set, which is perfectly valid but swagger-fluent, a dependency of external-secrets can't handle this case.

Moulick · 2020-04-24T18:10:14Z

We ran into the same issue here, Linkerd 2.7.0 is the culprit

Moulick · 2020-04-24T18:19:59Z

exactly @muenchdo , I just discovered that external-secrets did not have linkerd injected, still crashing after downgrade

wmorgan · 2020-04-24T19:25:49Z

Just to clarify a couple things here:

This is not really a Linkerd bug. Linkerd's tap service OpenAPI definition, while perfectly valid, is triggering a bug in another project (swagger-fluent) that's a dependency of kubernetes-external-secrets.
The real fix is for swagger-fluent to fix its OpenAPI parsing. But we've changed Linkerd's OpenAPI definition to avoid triggering this bug.
This change is available in the latest Linkerd edge release
This change will also go out in 2.8.

Sorry for not getting this change into Linkerd 2.7.1. I am writing this comment as penance.

Flydiverny · 2020-05-01T02:00:40Z

This issue should finally no longer occur on master! :)

Reproduction instructions in #278 (comment) now starts as expected when I try to reproduce! 🎉

matoszz · 2020-05-02T00:44:10Z

Not certain exactly what changed with 3.2 vs. 3.3, but for anyone else who runs into this, even though most folks in the comment thread are noting an environment with AWS and/or linkerd, I had the identical error on metal with no linkerd, and bumping to 3.3 resolved it. (running kube 1.17, strapped with kubeadm on metal)

Flydiverny · 2020-05-02T01:19:27Z

Changes related to this are all upstream

silasbw/swagger-fluent#69 and follow up silasbw/swagger-fluent#72

godaddy/kubernetes-client#619

#367

--

KES v3.2 -> v3.3 also (https://github.com/godaddy/kubernetes-external-secrets/blob/master/CHANGELOG.md#330-2020-05-01)

Changes GCP key format (GCP Secret Manager Backend Enhancements #347) (apparently missing in the changelog 😕)
Updates helm chart to set runAsNonRoot by default
Switches to regional STS endpoints by default
adds another prometheus metric

With the new release out I'm closing this! Please update if you still see issues with version 3.3.0 or higher.

mycrEEpy · 2020-05-02T10:07:22Z

Not certain exactly what changed with 3.2 vs. 3.3, but for anyone else who runs into this, even though most folks in the comment thread are noting an environment with AWS and/or linkerd, I had the identical error on metal with no linkerd, and bumping to 3.3 resolved it. (running kube 1.17, strapped with kubeadm on metal)

@matoszz the root issue was registering an ApiService in Kubernetes without defining an OperationId, linkerd was just a common application which did exactly that. If you have other applications with the same behaviour you get the same error for KES, no matter where your Kubernetes cluster is running.

dudicoco · 2021-01-24T12:07:55Z

We've experienced this issue as well after upgrading https://github.com/zalando-incubator/kube-metrics-adapter to the latest image. We don't have linkerd installed in our cluster.
Upgrading the version of kubernetes-external-secrets did not resolve this issue.

Any ideas?

Flydiverny · 2021-01-24T18:19:58Z

@dudicoco you sure its the exact same error? Or is it #576 ?

dudicoco · 2021-01-24T19:13:37Z

@Flydiverny looks like #576
Thanks

Flydiverny self-assigned this Feb 28, 2020

Flydiverny added bug Something isn't working help wanted Extra attention is needed labels Mar 1, 2020

Flydiverny mentioned this issue Mar 1, 2020

New error when loading kube specs #273

Closed

grampelberg mentioned this issue Mar 27, 2020

APIService swagger specs linkerd/linkerd2#4209

Closed

This was referenced Apr 26, 2020

fix: remove requirement for operationId silasbw/swagger-fluent#69

Merged

fix(deps): update dependency kubernetes-client to v9 #367

Merged

Flydiverny closed this as completed May 2, 2020

Startup failure #278

Startup failure #278

Comments

derrickburns commented Feb 7, 2020 • edited Loading

Efrat19 commented Feb 23, 2020

ghostsquad commented Feb 25, 2020

derrickburns commented Feb 25, 2020

keweilu commented Feb 27, 2020

krao-test commented Feb 27, 2020 • edited Loading

Flydiverny commented Feb 28, 2020

Flydiverny commented Feb 28, 2020

Efrat19 commented Mar 1, 2020

Flydiverny commented Mar 2, 2020

krao-test commented Mar 2, 2020

Efrat19 commented Mar 2, 2020

multi-io commented Mar 3, 2020 • edited Loading

Flydiverny commented Mar 3, 2020 • edited Loading

idobry commented Mar 5, 2020 • edited Loading

mycrEEpy commented Mar 5, 2020 • edited Loading

mycrEEpy commented Mar 9, 2020

dmduggan commented Mar 10, 2020

idobry commented Mar 11, 2020

mycrEEpy commented Mar 11, 2020

cpretzer commented Mar 12, 2020

domdepasquale commented Mar 16, 2020 • edited Loading

mycrEEpy commented Mar 21, 2020

Flydiverny commented Mar 22, 2020

mycrEEpy commented Mar 22, 2020

mycrEEpy commented Mar 22, 2020

davidcorbin commented Mar 23, 2020

tzilistgoop commented Mar 26, 2020 • edited Loading

davidcorbin commented Mar 28, 2020

davidcorbin commented Mar 28, 2020

domdepasquale commented Apr 1, 2020

cpretzer commented Apr 11, 2020

wmorgan commented Apr 12, 2020

mycrEEpy commented Apr 12, 2020

wmorgan commented Apr 12, 2020

mycrEEpy commented Apr 12, 2020

muenchdo commented Apr 24, 2020

mycrEEpy commented Apr 24, 2020

Moulick commented Apr 24, 2020

Moulick commented Apr 24, 2020

wmorgan commented Apr 24, 2020

Flydiverny commented May 1, 2020

matoszz commented May 2, 2020 • edited Loading

Flydiverny commented May 2, 2020

mycrEEpy commented May 2, 2020

dudicoco commented Jan 24, 2021

Flydiverny commented Jan 24, 2021

dudicoco commented Jan 24, 2021

derrickburns commented Feb 7, 2020 •

edited

Loading

krao-test commented Feb 27, 2020 •

edited

Loading

multi-io commented Mar 3, 2020 •

edited

Loading

Flydiverny commented Mar 3, 2020 •

edited

Loading

idobry commented Mar 5, 2020 •

edited

Loading

mycrEEpy commented Mar 5, 2020 •

edited

Loading

domdepasquale commented Mar 16, 2020 •

edited

Loading

tzilistgoop commented Mar 26, 2020 •

edited

Loading

matoszz commented May 2, 2020 •

edited

Loading