Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TestClusterIPv4 is flaky in Kind e2e test #2860

Closed
GraysonWu opened this issue Sep 29, 2021 · 3 comments · Fixed by #2866
Closed

TestClusterIPv4 is flaky in Kind e2e test #2860

GraysonWu opened this issue Sep 29, 2021 · 3 comments · Fixed by #2866
Assignees
@antoninbas
Labels
area/test Issues or PRs related to unit and integration tests. kind/bug Categorizes issue or PR as related to a bug. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test.

Comments

@GraysonWu
Copy link
Contributor

Describe the bug
TestClusterIPv4/Non-HostNetwork_Endpoints/Pods_from_all_Nodes_can_access_Service_ClusterIP and TestClusterIPv4/HostNetwork_Endpoints/Pods_from_all_Nodes_can_access_Service_ClusterIP are flaky in Kind e2e test when Antrea-proxy is disabled and Antrea-native policy is disabled.

=== RUN   TestClusterIPv4/Non-HostNetwork_Endpoints/Pods_from_all_Nodes_can_access_Service_ClusterIP
    service_test.go:92:
        	Error Trace:	service_test.go:92
        	            				service_test.go:85
        	Error:      	Received unexpected error:
        	            	command terminated with exit code 1
        	Test:       	TestClusterIPv4/Non-HostNetwork_Endpoints/Pods_from_all_Nodes_can_access_Service_ClusterIP
        	Messages:   	Service ClusterIP should be able to be connected from Pod
...
=== RUN   TestClusterIPv4/HostNetwork_Endpoints/Pods_from_all_Nodes_can_access_Service_ClusterIP
    service_test.go:92:
        	Error Trace:	service_test.go:92
        	            				service_test.go:85
        	Error:      	Received unexpected error:
        	            	command terminated with exit code 1
        	Test:       	TestClusterIPv4/HostNetwork_Endpoints/Pods_from_all_Nodes_can_access_Service_ClusterIP
        	Messages:   	Service ClusterIP should be able to be connected from Pod
@GraysonWu GraysonWu added the kind/bug Categorizes issue or PR as related to a bug. label Sep 29, 2021
@srikartati
Copy link
Member

Yes, I have seen this. Sometimes only one of the tests fails.

@GraysonWu GraysonWu mentioned this issue Sep 30, 2021
Closed
@lzhecheng
Copy link
Contributor

Saw it for a few times in different PRs.

@antoninbas antoninbas self-assigned this Oct 5, 2021
@antoninbas antoninbas added kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. area/test Issues or PRs related to unit and integration tests. labels Oct 5, 2021
@antoninbas
Copy link
Contributor

I'll take a look at this

antoninbas added a commit to antoninbas/antrea that referenced this issue Oct 6, 2021
@antoninbas
Fix flaky TestClusterIPv4 e2e tests
accee5b 
The tests checking access from the Node / hostNetwork were broken on
Kind clusters: wget is used to check connectivity, but wget is not
installed on the Kind Node image. This issue was going unnoticed however
because the tests were not checking for errors properly.

The tests checking access from the Pod network were flaky (at least with
kube-proxy) because no more than one connection was attempted, with
little delay between Service creation and the connectivity
test. Instead, we replace wget with agnhost, and we make up to 5
connection attempts.

Fixes antrea-io#2860

Signed-off-by: Antonin Bas <[email protected]>
@antoninbas antoninbas mentioned this issue Oct 6, 2021
Merged
antoninbas added a commit to antoninbas/antrea that referenced this issue Oct 13, 2021
@antoninbas
Fix flaky TestClusterIPv4 e2e tests
b888696 
The tests checking access from the Node / hostNetwork were broken on
Kind clusters: wget is used to check connectivity, but wget is not
installed on the Kind Node image. This issue was going unnoticed however
because the tests were not checking for errors properly.

The tests checking access from the Pod network were flaky (at least with
kube-proxy) because no more than one connection was attempted, with
little delay between Service creation and the connectivity
test. Instead, we replace wget with agnhost, and we make up to 5
connection attempts.

Fixes antrea-io#2860

Signed-off-by: Antonin Bas <[email protected]>
antoninbas added a commit that referenced this issue Oct 14, 2021
@antoninbas
Fix flaky TestClusterIPv4 e2e tests (#2866)
784dc4c 
The tests checking access from the Node / hostNetwork were broken on
Kind clusters: wget is used to check connectivity, but wget is not
installed on the Kind Node image. This issue was going unnoticed however
because the tests were not checking for errors properly.

The tests checking access from the Pod network were flaky (at least with
kube-proxy) because no more than one connection was attempted, with
little delay between Service creation and the connectivity
test. Instead, we replace wget with agnhost, and we make up to 5
connection attempts.

Fixes #2860

Signed-off-by: Antonin Bas <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@antoninbas antoninbas

Labels
area/test Issues or PRs related to unit and integration tests. kind/bug Categorizes issue or PR as related to a bug. kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix flaky TestClusterIPv4 e2e tests antoninbas/antrea
4 participants
@antoninbas @srikartati @lzhecheng @GraysonWu