Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Watcher stopped unexpectedly, will restart" warnings when using leader election #1077

Closed
clrxbl opened this issue Sep 17, 2022 · 5 comments
Closed

Comments

@clrxbl
Copy link

clrxbl commented Sep 17, 2022

Describe the bug
On Spring Boot 2.7.2 & Spring Cloud 2021.0.3 (with Spring Cloud Kubernetes 2.1.3), the following warning appears in the logs every now and then when Kubernetes leader election is in use:

2022-09-17 11:26:03.750  WARN 7 --- [/172.30.0.1/...] o.s.c.k.f.l.Fabric8LeaderRecordWatcher   : Watcher stopped unexpectedly, will restart

io.fabric8.kubernetes.client.WatcherException: too old resource version: 64484410 (64513614)
        at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onStatus(AbstractWatchManager.java:273)
        at io.fabric8.kubernetes.client.dsl.internal.AbstractWatchManager.onMessage(AbstractWatchManager.java:257)
        at io.fabric8.kubernetes.client.dsl.internal.WatcherWebSocketListener.onMessage(WatcherWebSocketListener.java:93)
        at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.kt:333)
        at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.kt:245)
        at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.kt:106)
        at okhttp3.internal.ws.RealWebSocket.loopReader(RealWebSocket.kt:293)
        at okhttp3.internal.ws.RealWebSocket$connect$1.onResponse(RealWebSocket.kt:195)
        at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: too old resource version: 64484410 (64513614)
        ... 12 common frames omitted

This is seemingly harmless as the application functions just fine. It seems similar to #730

Sample
N/A, simply running any application with leader election will return this warning after a while.

@ryanjbaxter
Copy link
Contributor

Are you suggesting we don't print the warning?

@spring-cloud-issues
Copy link

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

@spring-cloud-issues
Copy link

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

@a-maggioni
Copy link

a-maggioni commented Feb 13, 2024

I'm using the latest version of spring-cloud-kubernetes-fabric8-leader and the warning is still there.
According to this, Spring Cloud should migrate from using raw Watchers to Informers.

@wind57
Copy link
Contributor

wind57 commented Feb 14, 2024

@a-maggioni you're right, we only switched to informers for discovery implementation, not for leader. Can you please create a defect so that we can track this properly?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants