-
Notifications
You must be signed in to change notification settings - Fork 141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Node driver could not register on Windows in a little possibility #143
Comments
Following up my comments on #149, after I read how the kubelet plugin registration works, after the gRPC server is started we should get a notification from the kubelet through the callback
If we don't get the callback then as a workaround we can force a container restart, I tried a similar approach in kubernetes-csi/external-snapshotter#504, the idea is to crash the container if we don't get to a valid state after some seconds to force a restart, we could use race timer, in pseudocode it'd be something like:
In the external-snapshotter PR I also made changes to the manifests to make sure that a pod is only considered after some time, also this approach worked in the snapshotter because it's a Another idea is to use the http endpoint already exposed and a readiness probe, that endpoint would check the value of These ideas are workarounds though, the fact that sometimes we get a callback and sometimes we don't is something that we should ask in sig-windows |
cc @lizhuqi |
+1 to ask in sig-windows. I will ping sig-windows to take a look in the slack. |
I would vote for force restart if registrar does not get response from kubelet. |
We just upgraded our e2e tests to run against 1.21 windows for secrets-store-csi-driver and most of the e2e runs are failing with this error. There is no |
sig-windows asked to check the output of |
Within the next two weeks. |
https://github.com/kubernetes-csi/node-driver-registrar/releases/tag/v2.3.0 is already published. |
I have tried with
|
@andyzhangx I wrote how to use it here: https://github.com/kubernetes-csi/node-driver-registrar/blob/master/README.md#health-check-with-an-exec-probe, what's your setup? |
@mauriciopoppe thanks, it works well with that config! |
On Windows, we found that this node-driver-registrar hang on below statement forever in a little possibility, and then after restart registrar manually, it could register correctly, question: is there any way to add timeout and retry in below statement?
node-driver-registrar/cmd/csi-node-driver-registrar/node_register.go
Lines 63 to 64 in d79871c
/kind bug
/sig windows
The text was updated successfully, but these errors were encountered: