Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: drain/early-exit-conn-wait failed #103577

Closed
cockroach-teamcity opened this issue May 18, 2023 · 4 comments · Fixed by #103609
Closed

roachtest: drain/early-exit-conn-wait failed #103577

cockroach-teamcity opened this issue May 18, 2023 · 4 comments · Fixed by #103609
Assignees
@rafiss
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Milestone
23.1

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented May 18, 2023 •
edited by cockroach-jira-scripts
Loading

roachtest.drain/early-exit-conn-wait failed with artifacts on master @ 8f10f78aed7606edd454e886183bf22c74a3153e:

test artifacts and logs in: /artifacts/drain/early-exit-conn-wait/run_1
(assertions.go:262).Fail: 
	Error Trace:	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:204
	            				github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:45
	            				main/pkg/cmd/roachtest/test_runner.go:1012
	            				GOROOT/src/runtime/asm_amd64.s:1594
	Error:      	Received unexpected error:
	            	monitor failure: monitor task failed: expected --drain-wait to be upgraded to 2m25s
	            	(1) attached stack trace
	            	  -- stack trace:
	            	  | main.(*monitorImpl).WaitE
	            	  | 	main/pkg/cmd/roachtest/monitor.go:115
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runEarlyExitInConnectionWait
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:203
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerDrain.func1
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:45
	            	  | main.(*testRunner).runTest.func2
	            	  | 	main/pkg/cmd/roachtest/test_runner.go:1012
	            	Wraps: (2) monitor failure
	            	Wraps: (3) attached stack trace
	            	  -- stack trace:
	            	  | main.(*monitorImpl).wait.func2
	            	  | 	main/pkg/cmd/roachtest/monitor.go:171
	            	Wraps: (4) monitor task failed
	            	Wraps: (5) attached stack trace
	            	  -- stack trace:
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runEarlyExitInConnectionWait.func1
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:145
	            	  | main.(*monitorImpl).Go.func1
	            	  | 	main/pkg/cmd/roachtest/monitor.go:105
	            	  | golang.org/x/sync/errgroup.(*Group).Go.func1
	            	  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75
	            	  | runtime.goexit
	            	  | 	GOROOT/src/runtime/asm_amd64.s:1594
	            	Wraps: (6) expected --drain-wait to be upgraded to 2m25s
	            	Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError
	Test:       	drain/early-exit-conn-wait
	Messages:   	error waiting for the draining to finish
(require.go:1264).NoError: FailNow called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-28086
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) labels May 18, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone May 18, 2023
@rafiss rafiss self-assigned this May 18, 2023
@rafiss rafiss removed the release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. label May 18, 2023
@rafiss rafiss mentioned this issue May 18, 2023
Merged
@cockroach-teamcity
Copy link
Member Author

roachtest.drain/early-exit-conn-wait failed with artifacts on master @ 1df2f0e8dd222a21d799c63893fed65cb6a8ba99:

test artifacts and logs in: /artifacts/drain/early-exit-conn-wait/run_1
(assertions.go:262).Fail: 
	Error Trace:	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:204
	            				github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:45
	            				main/pkg/cmd/roachtest/test_runner.go:1012
	            				GOROOT/src/runtime/asm_amd64.s:1594
	Error:      	Received unexpected error:
	            	monitor failure: monitor task failed: expected --drain-wait to be upgraded to 2m25s
	            	(1) attached stack trace
	            	  -- stack trace:
	            	  | main.(*monitorImpl).WaitE
	            	  | 	main/pkg/cmd/roachtest/monitor.go:115
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runEarlyExitInConnectionWait
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:203
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerDrain.func1
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:45
	            	  | main.(*testRunner).runTest.func2
	            	  | 	main/pkg/cmd/roachtest/test_runner.go:1012
	            	Wraps: (2) monitor failure
	            	Wraps: (3) attached stack trace
	            	  -- stack trace:
	            	  | main.(*monitorImpl).wait.func2
	            	  | 	main/pkg/cmd/roachtest/monitor.go:171
	            	Wraps: (4) monitor task failed
	            	Wraps: (5) attached stack trace
	            	  -- stack trace:
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runEarlyExitInConnectionWait.func1
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:145
	            	  | main.(*monitorImpl).Go.func1
	            	  | 	main/pkg/cmd/roachtest/monitor.go:105
	            	  | golang.org/x/sync/errgroup.(*Group).Go.func1
	            	  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75
	            	  | runtime.goexit
	            	  | 	GOROOT/src/runtime/asm_amd64.s:1594
	            	Wraps: (6) expected --drain-wait to be upgraded to 2m25s
	            	Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError
	Test:       	drain/early-exit-conn-wait
	Messages:   	error waiting for the draining to finish
(require.go:1264).NoError: FailNow called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.drain/early-exit-conn-wait failed with artifacts on master @ a226099dfee6ac0159c0c40a54aaa753d646cf6e:

test artifacts and logs in: /artifacts/drain/early-exit-conn-wait/run_1
(assertions.go:262).Fail: 
	Error Trace:	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:204
	            				github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:45
	            				main/pkg/cmd/roachtest/test_runner.go:1012
	            				GOROOT/src/runtime/asm_amd64.s:1594
	Error:      	Received unexpected error:
	            	monitor failure: monitor task failed: expected --drain-wait to be upgraded to 2m25s
	            	(1) attached stack trace
	            	  -- stack trace:
	            	  | main.(*monitorImpl).WaitE
	            	  | 	main/pkg/cmd/roachtest/monitor.go:115
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runEarlyExitInConnectionWait
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:203
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerDrain.func1
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:45
	            	  | main.(*testRunner).runTest.func2
	            	  | 	main/pkg/cmd/roachtest/test_runner.go:1012
	            	Wraps: (2) monitor failure
	            	Wraps: (3) attached stack trace
	            	  -- stack trace:
	            	  | main.(*monitorImpl).wait.func2
	            	  | 	main/pkg/cmd/roachtest/monitor.go:171
	            	Wraps: (4) monitor task failed
	            	Wraps: (5) attached stack trace
	            	  -- stack trace:
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runEarlyExitInConnectionWait.func1
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:145
	            	  | main.(*monitorImpl).Go.func1
	            	  | 	main/pkg/cmd/roachtest/monitor.go:105
	            	  | golang.org/x/sync/errgroup.(*Group).Go.func1
	            	  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75
	            	  | runtime.goexit
	            	  | 	GOROOT/src/runtime/asm_amd64.s:1594
	            	Wraps: (6) expected --drain-wait to be upgraded to 2m25s
	            	Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError
	Test:       	drain/early-exit-conn-wait
	Messages:   	error waiting for the draining to finish
(require.go:1264).NoError: FailNow called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.drain/early-exit-conn-wait failed with artifacts on master @ b34c137004f111b915fb346da9a4770264d6c22f:

test artifacts and logs in: /artifacts/drain/early-exit-conn-wait/run_1
(assertions.go:262).Fail: 
	Error Trace:	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:204
	            				github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:45
	            				main/pkg/cmd/roachtest/test_runner.go:1012
	            				GOROOT/src/runtime/asm_amd64.s:1594
	Error:      	Received unexpected error:
	            	monitor failure: monitor task failed: expected --drain-wait to be upgraded to 2m25s
	            	(1) attached stack trace
	            	  -- stack trace:
	            	  | main.(*monitorImpl).WaitE
	            	  | 	main/pkg/cmd/roachtest/monitor.go:115
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runEarlyExitInConnectionWait
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:203
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerDrain.func1
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:45
	            	  | main.(*testRunner).runTest.func2
	            	  | 	main/pkg/cmd/roachtest/test_runner.go:1012
	            	Wraps: (2) monitor failure
	            	Wraps: (3) attached stack trace
	            	  -- stack trace:
	            	  | main.(*monitorImpl).wait.func2
	            	  | 	main/pkg/cmd/roachtest/monitor.go:171
	            	Wraps: (4) monitor task failed
	            	Wraps: (5) attached stack trace
	            	  -- stack trace:
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runEarlyExitInConnectionWait.func1
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:145
	            	  | main.(*monitorImpl).Go.func1
	            	  | 	main/pkg/cmd/roachtest/monitor.go:105
	            	  | golang.org/x/sync/errgroup.(*Group).Go.func1
	            	  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75
	            	  | runtime.goexit
	            	  | 	GOROOT/src/runtime/asm_amd64.s:1594
	            	Wraps: (6) expected --drain-wait to be upgraded to 2m25s
	            	Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError
	Test:       	drain/early-exit-conn-wait
	Messages:   	error waiting for the draining to finish
(require.go:1264).NoError: FailNow called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.drain/early-exit-conn-wait failed with artifacts on master @ b34c137004f111b915fb346da9a4770264d6c22f:

test artifacts and logs in: /artifacts/drain/early-exit-conn-wait/run_1
(assertions.go:262).Fail: 
	Error Trace:	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:204
	            				github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:45
	            				main/pkg/cmd/roachtest/test_runner.go:1012
	            				GOROOT/src/runtime/asm_amd64.s:1594
	Error:      	Received unexpected error:
	            	monitor failure: monitor task failed: expected --drain-wait to be upgraded to 2m25s
	            	(1) attached stack trace
	            	  -- stack trace:
	            	  | main.(*monitorImpl).WaitE
	            	  | 	main/pkg/cmd/roachtest/monitor.go:115
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runEarlyExitInConnectionWait
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:203
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerDrain.func1
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:45
	            	  | main.(*testRunner).runTest.func2
	            	  | 	main/pkg/cmd/roachtest/test_runner.go:1012
	            	Wraps: (2) monitor failure
	            	Wraps: (3) attached stack trace
	            	  -- stack trace:
	            	  | main.(*monitorImpl).wait.func2
	            	  | 	main/pkg/cmd/roachtest/monitor.go:171
	            	Wraps: (4) monitor task failed
	            	Wraps: (5) attached stack trace
	            	  -- stack trace:
	            	  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runEarlyExitInConnectionWait.func1
	            	  | 	github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/drain.go:145
	            	  | main.(*monitorImpl).Go.func1
	            	  | 	main/pkg/cmd/roachtest/monitor.go:105
	            	  | golang.org/x/sync/errgroup.(*Group).Go.func1
	            	  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75
	            	  | runtime.goexit
	            	  | 	GOROOT/src/runtime/asm_amd64.s:1594
	            	Wraps: (6) expected --drain-wait to be upgraded to 2m25s
	            	Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError
	Test:       	drain/early-exit-conn-wait
	Messages:   	error waiting for the draining to finish
(require.go:1264).NoError: FailNow called

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

This test on roachdash | Improve this report!

@craig craig bot closed this as completed in e00172c May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rafiss rafiss

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Milestone
23.1
Development

Successfully merging a pull request may close this issue.

roachtest: fix drain test's log searching rafiss/cockroach
2 participants
@rafiss @cockroach-teamcity