Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

functional-tester: stresser Cancel is hanging #6221

Closed
gyuho opened this issue Aug 18, 2016 · 0 comments · Fixed by #6226
Closed

functional-tester: stresser Cancel is hanging #6221

gyuho opened this issue Aug 18, 2016 · 0 comments · Fixed by #6226

Comments

@gyuho
Copy link
Contributor

gyuho commented Aug 18, 2016

2016-08-18 19:40:55.090511 I | etcd-tester: stresser "10.240.0.19:2379" is started
2016-08-18 19:40:55.090704 I | etcd-tester: stresser "10.240.0.20:2379" is started
2016-08-18 19:40:55.090996 I | etcd-tester: stresser "10.240.0.22:2379" is started
2016-08-18 19:40:55.091798 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.20:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.20:2379" <nil>}
2016-08-18 19:40:55.091851 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.22:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.22:2379" <nil>}
2016-08-18 19:40:55.091884 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.19:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.19:2379" <nil>}
2016-08-18 19:40:55.091926 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.19:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.19:2379" <nil>}
2016-08-18 19:40:57.337018 W | etcd-tester: #0 setHealthKey error (etcdserver: not capable (http://10.240.0.19:2379))
2016-08-18 19:40:58.372445 I | etcd-tester: no failpoints found (Get http://10.240.0.19:2381: dial tcp 10.240.0.19:2381: getsockopt: connection refused)
2016-08-18 19:40:58.410892 I | etcd-tester: [round#0 case#0] injecting failure "kill all members"
2016-08-18 19:40:58.524686 I | etcd-tester: transport: http2Client.notifyError got notified that the client transport was broken read tcp 10.240.0.23:37073->10.240.0.19:2379: read: connection reset by peer.
2016-08-18 19:40:58.525198 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.19:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.19:2379" <nil>}
2016-08-18 19:40:58.641738 I | etcd-tester: transport: http2Client.notifyError got notified that the client transport was broken read tcp 10.240.0.23:59978->10.240.0.20:2379: read: connection reset by peer.
2016-08-18 19:40:58.642428 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.20:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.20:2379" <nil>}
2016-08-18 19:40:59.525520 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.19:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.19:2379" <nil>}
2016-08-18 19:40:59.642728 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.20:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.20:2379" <nil>}
2016-08-18 19:41:01.201692 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.19:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.19:2379" <nil>}
2016-08-18 19:41:01.480019 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.20:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.20:2379" <nil>}
2016-08-18 19:41:03.582114 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.19:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.19:2379" <nil>}
2016-08-18 19:41:03.642565 I | etcd-tester: transport: http2Client.notifyError got notified that the client transport was broken EOF.
2016-08-18 19:41:03.642853 I | etcd-tester: [round#0 case#0] injected failure
2016-08-18 19:41:03.642876 I | etcd-tester: [round#0 case#0] recovering failure "kill all members"
2016-08-18 19:41:03.643302 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.22:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.22:2379" <nil>}
2016-08-18 19:41:03.648355 I | etcd-tester: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 10.240.0.19:2379: getsockopt: connection refused"; Reconnecting to {"10.240.0.19:2379" <nil>}
2016-08-18 19:41:06.040063 I | etcd-tester: [round#0 case#0] recovered failure
2016-08-18 19:41:06.040100 I | etcd-tester: [round#0 case#0] canceling the stressers...
2016-08-18 19:41:06.040471 I | etcd-tester: grpc: addrConn.transportMonitor exits due to: context canceled                                                                                 

Counted canceled goroutine and only got 77 where we have 100 goroutines to cancel.
Some goroutines seem blocked. Investigating...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
1 participant