Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ccl/backupccl: TestRestoreErrorPropagates failed #102796

Closed
cockroach-teamcity opened this issue May 5, 2023 · 1 comment
Closed

ccl/backupccl: TestRestoreErrorPropagates failed #102796

cockroach-teamcity opened this issue May 5, 2023 · 1 comment
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-disaster-recovery
Milestone
23.1

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented May 5, 2023 •
edited by cockroach-jira-scripts
Loading

ccl/backupccl.TestRestoreErrorPropagates failed with artifacts on master @ 918923738a73c39986c163465edb70e9af2c49b8:

F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	github.com/cockroachdb/cockroach/pkg/kv/kvpb/bazel-out/k8-fastbuild/bin/pkg/kv/kvpb/kvpb_go_proto_/github.com/cockroachdb/cockroach/pkg/kv/kvpb/api.pb.go:9791 +0x4c
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*rangefeedMuxer).receiveEventsFromNode.func2()
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_mux_rangefeed.go:426 +0x2b
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*stuckRangeFeedCanceler).do(0xc000865f90, 0xc01b6d5b50)
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed_canceler.go:135 +0x1a2
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*rangefeedMuxer).receiveEventsFromNode(0xc01c19fc00, {0x768a238, 0xc006363b40}, {0x7f37f0bcede8, 0xc00c9c3f80}, 0xaa263b0?, 0xc010eb6e70)
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_mux_rangefeed.go:425 +0x198
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*rangefeedMuxer).startNodeMuxRangeFeed(0xc01c19fc00, {0x768a238, 0xc006363b40}, {0x7687b60, 0xc00c9c3ec0}, 0x1, 0x8?)
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_mux_rangefeed.go:379 +0x574
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*rangefeedMuxer).establishMuxConnection.func1({0x768a238?, 0xc006363a40?})
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_mux_rangefeed.go:322 +0x45
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1()
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x25
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !golang.org/x/sync/errgroup.(*Group).Go.func1()
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x64
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !created by golang.org/x/sync/errgroup.(*Group).Go
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:72 +0xa5
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !goroutine 1057717 [chan receive]:
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !github.com/cockroachdb/cockroach/pkg/util/netutil.MakeTCPServer.func1({0xc008ea3800, 0xc01b1237c0})
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:126 +0x5d
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470 +0x146
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:461 +0x43b
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !For more context, check log files in: /artifacts/tmp/_tmp/95e138e66d69292427dfb9528cf06d04/logTestRestoreErrorPropagates3457703609
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !For more context, check log files in: /artifacts/tmp/_tmp/95e138e66d69292427dfb9528cf06d04/logTestRestoreErrorPropagates3457703609
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !****************************************************************************
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !This node experienced a fatal error (printed above), and as a result the
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !process is terminating.
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !Fatal errors can occur due to faulty hardware (disks, memory, clocks) or a
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !problem in CockroachDB. With your help, the support team at Cockroach Labs
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !will try to determine the root cause, recommend next steps, and we can
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !improve CockroachDB based on your report.
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !Please submit a crash report by following the instructions here:
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !    https://github.com/cockroachdb/cockroach/issues/new/choose
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !If you would rather not post publicly, please contact us directly at:
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !    [email protected]
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !
F230505 06:39:39.095267 1056849 3@pebble/db.go:802  [n2,s2,pebble] 1 !The Cockroach Labs team appreciates your feedback.

Parameters: TAGS=bazel,gss

Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/disaster-recovery

This test on roachdash | Improve this report!

Jira issue: CRDB-27655
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-disaster-recovery labels May 5, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone May 5, 2023
@stevendanna
Copy link
Collaborator

Duplicate of #102945

@stevendanna stevendanna marked this as a duplicate of #102945 May 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-disaster-recovery
Projects
No open projects
Disaster Recovery Backlog
Archived in project
Milestone
23.1
Development

No branches or pull requests
2 participants
@stevendanna @cockroach-teamcity