Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x-pack/filebeat/input/cometd/TestMultiEventForEOFRetryHandlerInput test can deadlock for 10 minutes #34740

Closed
elasticmachine opened this issue Mar 6, 2023 · 3 comments
Closed

x-pack/filebeat/input/cometd/TestMultiEventForEOFRetryHandlerInput test can deadlock for 10 minutes #34740

elasticmachine opened this issue Mar 6, 2023 · 3 comments
Assignees
@kush-elastic
Labels
automation build-failures Build failures in the CI. ci-reported Issues that have been automatically reported from the CI flaky-test Unstable or unreliable test cases. Team:Service-Integrations Label for the Service Integrations team

Comments

@elasticmachine
Copy link
Collaborator

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-03-06T16:34:29.690+0000

  • Duration: 115 min 12 sec

Test stats 🧪

Test Results
Failed 1
Passed 29080
Skipped 2035
Total 31116

Test errors 1

Expand to view the tests failures

Extended / x-pack/filebeat-arm-ubuntu-2204-aarch64 / TestMultiEventForEOFRetryHandlerInput – github.com/elastic/beats/v7/x-pack/filebeat/input/cometd
    Expand to view the error details

     Failed

    Expand to view the stacktrace

     === RUN   TestMultiEventForEOFRetryHandlerInput
coverage: 79.1% of statements
panic: test timed out after 10m0s

goroutine 825 [running]:
testing.(*M).startAlarm.func1()
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/testing/testing.go:2029 +0x8c
created by time.goFunc
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/time/sleep.go:176 +0x3c

goroutine 1 [chan receive, 9 minutes]:
testing.(*T).Run(0x400030ad00, {0x127a111?, 0x1f8d77c9dc6?}, 0x12b1f78)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/testing/testing.go:1487 +0x33c
testing.runTests.func1(0x0?)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/testing/testing.go:1839 +0x74
testing.tRunner(0x400030ad00, 0x40002dfc88)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/testing/testing.go:1439 +0x110
testing.runTests(0x40002fae60?, {0x19decc0, 0xf, 0xf}, {0x3b00000000000000?, 0x8509c0?, 0x1b80c80?})
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/testing/testing.go:1837 +0x3e8
testing.(*M).Run(0x40002fae60)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/testing/testing.go:1719 +0x510
main.main()
	_testmain.go:129 +0x278

goroutine 81 [semacquire, 9 minutes]:
sync.runtime_Semacquire(0x13a93a8?)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/runtime/sema.go:56 +0x2c
sync.(*WaitGroup).Wait(0x400043c488)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/sync/waitgroup.go:136 +0x88
github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.(*cometdInput).Stop(0x400043c420)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:201 +0x7c
github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.TestMultiEventForEOFRetryHandlerInput(0x4000428d00)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input_test.go:502 +0x80c
testing.tRunner(0x4000428d00, 0x12b1f78)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/testing/testing.go:1439 +0x110
created by testing.(*T).Run
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/testing/testing.go:1486 +0x328

goroutine 68 [chan receive (nil chan), 9 minutes]:
github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.NewInput.func1()
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:172 +0x74
created by github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.NewInput
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:171 +0x3d0

goroutine 32 [chan receive (nil chan), 9 minutes]:
github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.NewInput.func1()
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:172 +0x74
created by github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.NewInput
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:171 +0x3d0

goroutine 103 [chan receive (nil chan), 9 minutes]:
github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.NewInput.func1()
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:172 +0x74
created by github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.NewInput
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:171 +0x3d0

goroutine 104 [chan receive (nil chan), 9 minutes]:
github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.NewInput.func1()
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:172 +0x74
created by github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.NewInput
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:171 +0x3d0

goroutine 137 [chan receive (nil chan), 9 minutes]:
github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.NewInput.func1()
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:172 +0x74
created by github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.NewInput
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:171 +0x3d0

goroutine 138 [chan send, 9 minutes]:
github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.TestMultiEventForEOFRetryHandlerInput.func1({{0x25ea6915, 0xedb982575, 0x0}, 0x40004d2ed0, 0x40004d2e40, {0x10de9a0, 0x40004e6060}, 0x0})
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input_test.go:407 +0x64
github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.mockedOutleter.OnEvent(...)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/client_mocked.go:46
github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.(*cometdInput).run(0x400043c420)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:128 +0x650
github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.(*cometdInput).Run.func1.1()
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:65 +0x4bc
created by github.com/elastic/beats/v7/x-pack/filebeat/input/cometd.(*cometdInput).Run.func1
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/src/github.com/elastic/beats/x-pack/filebeat/input/cometd/input.go:41 +0xd8

goroutine 821 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0xffff763602e8, 0x72)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/runtime/netpoll.go:302 +0xa4
internal/poll.(*pollDesc).wait(0x4000271900?, 0x8c79b4?, 0x0)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/internal/poll/fd_poll_runtime.go:83 +0x2c
internal/poll.(*pollDesc).waitRead(...)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/internal/poll/fd_poll_runtime.go:88
internal/poll.(*FD).Accept(0x4000271900)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/internal/poll/fd_unix.go:614 +0x1d8
net.(*netFD).accept(0x4000271900)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/net/fd_unix.go:172 +0x28
net.(*TCPListener).accept(0x40000d11a0)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/net/tcpsock_posix.go:139 +0x2c
net.(*TCPListener).Accept(0x40000d11a0)
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/net/tcpsock.go:288 +0x30
net/http.(*Server).Serve(0x40002440e0, {0x13b1e00, 0x40000d11a0})
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/net/http/server.go:3039 +0x300
net/http/httptest.(*Server).goServe.func1()
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/net/http/httptest/server.go:308 +0x64
created by net/http/httptest.(*Server).goServe
	/var/lib/jenkins/workspace/main-1325-c2b4d38c-9926-4a61-930c-83cecdd1f51b/.gvm/versions/go1.18.10.linux.arm64/src/net/http/httptest/server.go:306 +0x78

Steps errors 6

Expand to view the steps failures

libbeat-goIntegTest - mage goIntegTest
  • Took 6 min 41 sec . View more details here
  • Description: mage goIntegTest
x-pack/metricbeat-goIntegTest - mage goIntegTest
  • Took 20 min 6 sec . View more details here
  • Description: mage goIntegTest
x-pack/filebeat-arm-ubuntu-2204-aarch64 - mage build unitTest
  • Took 13 min 27 sec . View more details here
  • Description: mage build unitTest
x-pack/filebeat-arm-ubuntu-2204-aarch64 - mage build unitTest
  • Took 10 min 18 sec . View more details here
  • Description: mage build unitTest
x-pack/filebeat-arm-ubuntu-2204-aarch64 - mage build unitTest
  • Took 10 min 17 sec . View more details here
  • Description: mage build unitTest
Error signal
  • Took 0 min 0 sec . View more details here
  • Description: Error "hudson.AbortException: script returned exit code 1"

@elasticmachine elasticmachine added automation ci-reported Issues that have been automatically reported from the CI build-failures Build failures in the CI. Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team labels Mar 6, 2023
@elasticmachine
Copy link
Collaborator Author

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@cmacknz cmacknz added Team:Service-Integrations Label for the Service Integrations team and removed Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team labels Mar 6, 2023
@cmacknz
Copy link
Member

cmacknz commented Mar 6, 2023

@elastic/obs-service-integrations looks like the Salesforce cometd tests are timing out:

 === RUN   TestMultiEventForEOFRetryHandlerInput
coverage: 79.1% of statements
panic: test timed out after 10m0s

@cmacknz cmacknz changed the title Build 1325 for main with status FAILURE The cometd TestMultiEventForEOFRetryHandlerInput test can deadlocked for 10 minutes Mar 6, 2023
@cmacknz cmacknz added the flaky-test Unstable or unreliable test cases. label Mar 6, 2023
@cmacknz cmacknz changed the title The cometd TestMultiEventForEOFRetryHandlerInput test can deadlocked for 10 minutes x-pack/filebeat/input/cometd/TestMultiEventForEOFRetryHandlerInput test can deadlocked for 10 minutes Mar 6, 2023
@cmacknz cmacknz changed the title x-pack/filebeat/input/cometd/TestMultiEventForEOFRetryHandlerInput test can deadlocked for 10 minutes x-pack/filebeat/input/cometd/TestMultiEventForEOFRetryHandlerInput test can deadlock for 10 minutes Mar 6, 2023
@cmacknz cmacknz mentioned this issue Mar 9, 2023
Closed
@kush-elastic
Copy link
Collaborator

Closing this as PR for a fix is merged.
#34903

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kush-elastic kush-elastic

Labels
automation build-failures Build failures in the CI. ci-reported Issues that have been automatically reported from the CI flaky-test Unstable or unreliable test cases. Team:Service-Integrations Label for the Service Integrations team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cmacknz @elasticmachine @kush-elastic