Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: bunch of failures #109994

Closed
cockroach-teamcity opened this issue Sep 4, 2023 · 565 comments
Closed

*: bunch of failures #109994

cockroach-teamcity opened this issue Sep 4, 2023 · 565 comments
Labels
branch-master Failures and bugs on the master branch. O-robot Originated from a bot. T-dev-inf
Milestone

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Sep 4, 2023

. failed with artifacts on master @ 8ee0bc7e4391add8c3d8ec99cddf510f581b1a8f:

[3,090 / 3,297] GoCompilePkg pkg/sql/parser/parser.a; 19s processwrapper-sandbox ... (4 actions, 3 running)
[3,092 / 3,297] GoCompilePkg pkg/sql/parser/parser.a; 21s processwrapper-sandbox ... (4 actions running)
[3,095 / 3,297] GoCompilePkg pkg/sql/parser/parser.a; 22s processwrapper-sandbox ... (3 actions running)
[3,096 / 3,297] GoCompilePkg pkg/sql/parser/parser.a; 23s processwrapper-sandbox ... (3 actions, 2 running)
[3,100 / 3,297] GoCompilePkg pkg/sql/parser/parser.a; 24s processwrapper-sandbox ... (4 actions, 3 running)
[3,105 / 3,297] GoCompilePkg pkg/sql/parser/parser.a; 26s processwrapper-sandbox ... (4 actions running)
[3,111 / 3,297] GoCompilePkg pkg/sql/parser/parser.a; 27s processwrapper-sandbox ... (4 actions, 3 running)
[3,115 / 3,297] GoCompilePkg pkg/sql/parser/parser.a; 28s processwrapper-sandbox ... (4 actions, 3 running)
[3,122 / 3,297] GoCompilePkg pkg/sql/parser/parser.a; 30s processwrapper-sandbox ... (2 actions running)
[3,123 / 3,297] GoCompilePkg pkg/sql/parser/parser.a; 32s processwrapper-sandbox
[3,124 / 3,297] GoCompilePkg pkg/sql/catalog/redact/redact.a; 0s processwrapper-sandbox ... (4 actions running)
[3,131 / 3,297] [Prepa] GoCompilePkg pkg/sql/paramparse/paramparse.a
[3,135 / 3,297] GoCompilePkg pkg/sql/exprutil/exprutil.a; 0s processwrapper-sandbox ... (4 actions, 3 running)
[3,139 / 3,297] GoCompilePkg pkg/sql/pgwire/pgwirebase/pgwirebase.a; 0s processwrapper-sandbox ... (4 actions running)
[3,144 / 3,297] GoCompilePkg pkg/sql/isql/isql.a; 0s processwrapper-sandbox ... (4 actions running)
[3,147 / 3,297] GoCompilePkg pkg/col/coldataext/coldataext.a; 1s processwrapper-sandbox ... (4 actions running)
[3,152 / 3,297] GoCompilePkg pkg/storage/storage.a; 0s processwrapper-sandbox ... (4 actions running)
[3,155 / 3,297] GoCompilePkg pkg/storage/storage.a; 1s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /go/src/github.com/cockroachdb/cockroach/pkg/server/tracedumper/BUILD.bazel:3:11: GoCompilePkg pkg/server/tracedumper/tracedumper.a failed: (Exit 1): builder failed: error executing command (from target //pkg/server/tracedumper:tracedumper) bazel-out/k8-opt-exec-2B5CBBC6/bin/external/go_sdk/builder_reset/builder compilepkg -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,deadlock,bazel,gss,deadlock -src ... (remaining 41 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
compilepkg: panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x67509b]

goroutine 119 [running]:
go/types.(*_TypeSet).Method(...)
	GOROOT/src/go/types/typeset.go:56
go/types.(*Interface).Method(...)
	GOROOT/src/go/types/interface.go:116
golang.org/x/tools/go/types/typeutil.Hasher.hashFor({0xc001142420?, 0xc001142450?, 0x0?}, {0xc1e2b0?, 0xc0020272c0?})
	golang.org/x/tools/go/types/typeutil/external/org_golang_x_tools/go/types/typeutil/map.go:334 +0x83b
golang.org/x/tools/go/types/typeutil.Hasher.Hash({0xc001142420?, 0xc001142450?, 0x0?}, {0xc1e2b0, 0xc0020272c0})
	golang.org/x/tools/go/types/typeutil/external/org_golang_x_tools/go/types/typeutil/map.go:239 +0x91
golang.org/x/tools/go/types/typeutil.(*Map).At(0xc001588078, {0xc1e2b0, 0xc0020272c0})
	golang.org/x/tools/go/types/typeutil/external/org_golang_x_tools/go/types/typeutil/map.go:86 +0x5c
golang.org/x/tools/go/ssa.(*Program).needMethods(0xc001588000, {0xc1e2b0?, 0xc0020272c0?}, 0x1, 0xfc1f20?)
	golang.org/x/tools/go/ssa/external/org_golang_x_tools/go/ssa/methods.go:186 +0x6e
golang.org/x/tools/go/ssa.(*Program).needMethods(0xc001588000, {0xc1e300?, 0xc002032850?}, 0x0, 0xfc1f20?)
	golang.org/x/tools/go/ssa/external/org_golang_x_tools/go/ssa/methods.go:251 +0x589
golang.org/x/tools/go/ssa.(*Program).needMethods(0xc001588000, {0xc1e3c8?, 0xc0020154d0?}, 0x0, 0xfc1f20?)
	golang.org/x/tools/go/ssa/external/org_golang_x_tools/go/ssa/methods.go:263 +0x7f5
golang.org/x/tools/go/ssa.(*Program).needMethods(0xc001588000, {0xc1e300?, 0xc000b80ee0?}, 0x0, 0xfc1f20?)
	golang.org/x/tools/go/ssa/external/org_golang_x_tools/go/ssa/methods.go:212 +0x1f7
golang.org/x/tools/go/ssa.(*Program).needMethods(0xc001588000, {0xc1e3c8?, 0xc000b7fa10?}, 0x0, 0xfc1f20?)
	golang.org/x/tools/go/ssa/external/org_golang_x_tools/go/ssa/methods.go:263 +0x7f5
golang.org/x/tools/go/ssa.(*Program).needMethods(0xc001588000, {0xc1e328?, 0xc000b848d0?}, 0x0, 0xfc1f20?)
	golang.org/x/tools/go/ssa/external/org_golang_x_tools/go/ssa/methods.go:211 +0x1d0
golang.org/x/tools/go/ssa.(*Program).needMethods(0xc001588000, {0xc1e3c8?, 0xc000bacdf8?}, 0x0, 0xfc1f20?)
	golang.org/x/tools/go/ssa/external/org_golang_x_tools/go/ssa/methods.go:263 +0x7f5
golang.org/x/tools/go/ssa.(*Program).needMethods(0xc001588000, {0xc1e

Parameters: TAGS=bazel,gss,deadlock , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-31207

@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-testeng TestEng Team labels Sep 4, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.2 milestone Sep 4, 2023
@renatolabs
Copy link
Contributor

@rickystewart seems like these panics are still happening on master.

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 3fb3b588117a2f790ba9bd7fcc0b93874822b695:

stdout: 
, stderr: Loading: 
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --//build/toolchains:cross_flag, --platforms, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/acceptance:acceptance_test (1 packages loaded, 0 targets configured)
Analyzing: target //pkg/acceptance:acceptance_test (117 packages loaded, 13964 targets configured)
Analyzing: target //pkg/acceptance:acceptance_test (604 packages loaded, 17378 targets configured)
Analyzing: target //pkg/acceptance:acceptance_test (701 packages loaded, 18416 targets configured)
Analyzing: target //pkg/acceptance:acceptance_test (952 packages loaded, 20603 targets configured)
Analyzing: target //pkg/acceptance:acceptance_test (1013 packages loaded, 21859 targets configured)
Analyzing: target //pkg/acceptance:acceptance_test (1076 packages loaded, 22528 targets configured)
INFO: Repository com_github_apache_thrift instantiated at:
  /home/agent/work/.go/src/github.com/cockroachdb/cockroach/WORKSPACE:59:8: in <toplevel>
  /home/agent/work/.go/src/github.com/cockroachdb/cockroach/DEPS.bzl:297:18: in go_deps
Repository rule go_repository defined at:
  /home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/bazel_gazelle/internal/go_repository.bzl:278:32: in <toplevel>
WARNING: Download from https://storage.googleapis.com/cockroach-godeps/gomod/github.com/apache/thrift/com_github_apache_thrift-v0.16.0.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.ContentLengthMismatchException Bytes read 2883584 but wanted 5017867
ERROR: An error occurred during the fetch of repository 'com_github_apache_thrift':
   Traceback (most recent call last):
	File "/home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/bazel_gazelle/internal/go_repository.bzl", line 113, column 33, in _go_repository_impl
		ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://storage.googleapis.com/cockroach-godeps/gomod/github.com/apache/thrift/com_github_apache_thrift-v0.16.0.zip] to /home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/com_github_apache_thrift/temp17107791169598267418/com_github_apache_thrift-v0.16.0.zip: Bytes read 2883584 but wanted 5017867
ERROR: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/WORKSPACE:59:8: fetching go_repository rule //external:com_github_apache_thrift: Traceback (most recent call last):
	File "/home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/bazel_gazelle/internal/go_repository.bzl", line 113, column 33, in _go_repository_impl
		ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://storage.googleapis.com/cockroach-godeps/gomod/github.com/apache/thrift/com_github_apache_thrift-v0.16.0.zip] to /home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/com_github_apache_thrift/temp17107791169598267418/com_github_apache_thrift-v0.16.0.zip: Bytes read 2883584 but wanted 5017867
ERROR: /home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/com_github_fraugster_parquet_go/parquet/BUILD.bazel:3:11: @com_github_fraugster_parquet_go//parquet:parquet depends on @com_github_apache_thrift//lib/go/thrift:thrift in repository @com_github_apache_thrift which failed to fetch. no such package '@com_github_apache_thrift//lib/go/thrift': java.io.IOException: Error downloading [https://storage.googleapis.com/cockroach-godeps/gomod/github.com/apache/thrift/com_github_apache_thrift-v0.16.0.zip] to /home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/com_github_apache_thrift/temp17107791169598267418/com_github_apache_thrift-v0.16.0.zip: Bytes read 2883584 but wanted 5017867
ERROR: Analysis of target '//pkg/acceptance:acceptance_test' failed; build aborted: 
INFO: Elapsed time: 7.304s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1077 packages loaded, 22528 targets configured)
ERROR: Couldn't start the build. 
Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110157 : failed [C-test-failure O-robot T-testeng branch-release-23.1 release-blocker]

This test on roachdash | Improve this report!

@renatolabs renatolabs added T-dev-inf and removed T-testeng TestEng Team labels Sep 8, 2023
@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b7840ea58fc6db38528e1b499a0357d8ca51501e:

honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e328?, 0xc001514c60?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:183 +0x1a9
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e300?, 0xc000e9db20?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:217 +0x585
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e3a0?, 0xc000493fb0?}, 0x1)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:230 +0x65d
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e300?, 0xc000e9dab0?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:223 +0x5ac
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e328?, 0xc000482e10?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:195 +0x49c
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e3c8?, 0xc000867e78?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:235 +0x775
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e328?, 0xc00119b930?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:184 +0x1c8
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e3c8?, 0xc000b2d5a8?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:235 +0x775
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e328?, 0xc000482d60?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:183 +0x1a9
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e3c8?, 0xc001108d68?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:235 +0x775
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e300?, 0xc000e9d9d0?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:184 +0x1c8
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e3c8?, 0xc0015fa930?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:235 +0x775
honnef.co/go/tools/go/ir.(*Program).needMethods(0xc000399a40, {0xc1e350?, 0xc0015fcd00?}, 0x0)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:211 +0x2d8
honnef.co/go/tools/go/ir.(*Program).needMethodsOf(0xc000399a40, {0xc1e350?, 0xc0015fcd00?})
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/methods.go:148 +0x70
honnef.co/go/tools/go/ir.(*Package).build(0xc001117a70)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/builder.go:2568 +0x111
sync.(*Once).doSlow(0xc000399a40?, 0xc00012f900?)
	GOROOT/src/sync/once.go:74 +0xc2
sync.(*Once).Do(...)
	GOROOT/src/sync/once.go:65
honnef.co/go/tools/go/ir.(*Package).Build(...)
	honnef.co/go/tools/go/ir/external/co_honnef_go_tools/go/ir/builder.go:2556
honnef.co/go/tools/internal/passes/buildir.run(0xc0010ca340)
	honnef.co/go/tools/internal/passes/buildir/external/co_honnef_go_tools/internal/passes/buildir/buildir.go:86 +0x308
main.(*action).execOnce(0xc0001a8e60)
	main/external/io_bazel_rules_go/go/tools/builders/nogo_main.go:353 +0x843
sync.(*Once).doSlow(0x6c71732f676b702f?, 0x6572742f6d65732f?)
	GOROOT/src/sync/once.go:74 +0xc2
sync.(*Once).Do(...)
	GOROOT/src/sync/once.go:65
main.(*action).exec(0x632f62646863616f?)
	main/external/io_bazel_rules_go/go/tools/builders/nogo_main.go:280 +0x3d
main.execAll.func1(0x726b636f632f6d6f?)
	main/external/io_bazel_rules_go/go/tools/builders/nogo_main.go:274 +0x54
created by main.execAll
	main/external/io_bazel_rules_go/go/tools/builders/nogo_main.go:272 +0x47

Parameters: TAGS=bazel,gss,deadlock , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/types:types_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/types:types_test (88 packages loaded, 11351 targets configured)
Analyzing: target //pkg/sql/types:types_test (118 packages loaded, 12306 targets configured)
Analyzing: target //pkg/sql/types:types_test (170 packages loaded, 14433 targets configured)
Analyzing: target //pkg/sql/types:types_test (216 packages loaded, 16420 targets configured)
Analyzing: target //pkg/sql/types:types_test (246 packages loaded, 17918 targets configured)
Analyzing: target //pkg/sql/types:types_test (273 packages loaded, 20538 targets configured)
INFO: Analyzed target //pkg/sql/types:types_test (286 packages loaded, 21654 targets configured).
 checking cached actions
INFO: Found 1 test target...
[0 / 8] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[130 / 510] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[130 / 510] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[131 / 510] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 16s processwrapper-sandbox ... (4 actions, 3 running)
[358 / 1,162] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[362 / 1,176] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[362 / 1,176] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 13s processwrapper-sandbox ... (4 actions running)
[363 / 1,176] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (3 actions running)
[367 / 1,176] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 27s processwrapper-sandbox ... (4 actions, 3 running)
[372 / 1,176] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 28s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __t

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/ttl/ttlbase:ttlbase_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/ttl/ttlbase:ttlbase_test (87 packages loaded, 11267 targets configured)
Analyzing: target //pkg/sql/ttl/ttlbase:ttlbase_test (104 packages loaded, 13697 targets configured)
Analyzing: target //pkg/sql/ttl/ttlbase:ttlbase_test (190 packages loaded, 14410 targets configured)
Analyzing: target //pkg/sql/ttl/ttlbase:ttlbase_test (234 packages loaded, 15483 targets configured)
Analyzing: target //pkg/sql/ttl/ttlbase:ttlbase_test (311 packages loaded, 17647 targets configured)
Analyzing: target //pkg/sql/ttl/ttlbase:ttlbase_test (364 packages loaded, 18868 targets configured)
Analyzing: target //pkg/sql/ttl/ttlbase:ttlbase_test (387 packages loaded, 21296 targets configured)
INFO: Analyzed target //pkg/sql/ttl/ttlbase:ttlbase_test (404 packages loaded, 22513 targets configured).
 checking cached actions
INFO: Found 1 test target...
[0 / 8] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[4 / 90] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 0s processwrapper-sandbox ... (4 actions running)
[4 / 90] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[6 / 90] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 17s processwrapper-sandbox ... (3 actions, 2 running)
[230 / 667] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[248 / 702] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[248 / 702] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 13s processwrapper-sandbox ... (4 actions running)
[249 / 702] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire no

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/tests:tests_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (96 packages loaded, 1086 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (265 packages loaded, 9770 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (439 packages loaded, 10294 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (465 packages loaded, 10302 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (474 packages loaded, 10606 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (493 packages loaded, 11004 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (628 packages loaded, 11499 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (693 packages loaded, 11830 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (771 packages loaded, 14065 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (824 packages loaded, 14481 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (903 packages loaded, 14642 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (992 packages loaded, 15802 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (1026 packages loaded, 16447 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (1085 packages loaded, 18059 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (1131 packages loaded, 19540 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (1153 packages loaded, 20452 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (1209 packages loaded, 20991 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (1320 packages loaded, 21981 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (1381 packages loaded, 23134 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (1412 packages loaded, 24252 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (1466 packages loaded, 25720 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (1477 packages loaded, 27290 targets configured)
Analyzing: target //pkg/sql/tests:tests_test (1517 packages loaded, 29518 targets configured)
INFO: Analyzed target //pkg/sql/tests:tests_test (1522 packages loaded, 30458 targets configured).
INFO: Found 1 test target...
[0 / 7] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[131 / 535] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[131 / 535] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[133 / 535] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 16s processwrapper-sandbox ... (3 actions, 2 running)
[356 / 1,261] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[366 / 1,272] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[366 / 1,272] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[367 / 1,272] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 24s processwrapper-sandbox ... (4 actions, 3 running)
[371 / 1,272] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions, 3 running)
[375 / 1,272] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 27s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: e

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/storage:storage_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/storage:storage_test (104 packages loaded, 3567 targets configured)
Analyzing: target //pkg/storage:storage_test (175 packages loaded, 10389 targets configured)
Analyzing: target //pkg/storage:storage_test (184 packages loaded, 10703 targets configured)
Analyzing: target //pkg/storage:storage_test (256 packages loaded, 12240 targets configured)
Analyzing: target //pkg/storage:storage_test (340 packages loaded, 15011 targets configured)
Analyzing: target //pkg/storage:storage_test (402 packages loaded, 16556 targets configured)
Analyzing: target //pkg/storage:storage_test (437 packages loaded, 18599 targets configured)
Analyzing: target //pkg/storage:storage_test (466 packages loaded, 21911 targets configured)
Analyzing: target //pkg/storage:storage_test (476 packages loaded, 23106 targets configured)
INFO: Analyzed target //pkg/storage:storage_test (479 packages loaded, 23136 targets configured).
INFO: Found 1 test target...
[0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[130 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[130 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[131 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 16s processwrapper-sandbox ... (4 actions, 3 running)
[280 / 1,016] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 0s processwrapper-sandbox ... (3 actions, 2 running)
[360 / 1,309] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[360 / 1,309] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 13s processwrapper-sandbox ... (4 actions running)
[361 / 1,309] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 24s processwrapper-sandbox ... (4 actions, 3 running)
[365 / 1,309] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 25s processwrapper-sandbox ... (4 actions, 3 running)
[369 / 1,309] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_sha

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (106 packages loaded, 9622 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (310 packages loaded, 9711 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (492 packages loaded, 10262 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (534 packages loaded, 11382 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (578 packages loaded, 11950 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (596 packages loaded, 12048 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (628 packages loaded, 12353 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (731 packages loaded, 15126 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (765 packages loaded, 16299 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (806 packages loaded, 17275 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (874 packages loaded, 19510 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (924 packages loaded, 20704 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (970 packages loaded, 22410 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (985 packages loaded, 25279 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (1012 packages loaded, 26882 targets configured)
Analyzing: target //pkg/sql/ttl/ttljob:ttljob_test (1055 packages loaded, 27646 targets configured)
INFO: Analyzed target //pkg/sql/ttl/ttljob:ttljob_test (1056 packages loaded, 27684 targets configured).
INFO: Found 1 test target...
[0 / 3] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[131 / 556] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[131 / 556] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[132 / 556] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 17s processwrapper-sandbox ... (4 actions, 3 running)
[259 / 708] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 1s processwrapper-sandbox ... (4 actions running)
[259 / 708] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 11s processwrapper-sandbox ... (4 actions running)
[260 / 708] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 32s processwrapper-sandbox ... (4 actions, 3 running)
[262 / 708] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 33s processwrapper-sandbox ... (4 actions, 3 running)
[265 / 708] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 34s processwrapper-sandbox ... (4 actions, 3 running)
[268 / 708] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 36s processwrapper-sandbox ... (4 actions running)
[273 / 708] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 37s processwrapper-sandbox ... (4 actions running)
[274 / 708] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 38s processwrapper-sandbox ... (4 actions, 3 running)
[278 / 708] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 39s processwrapper-sandbox ... (4 actions, 3 running)
[285 / 708] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 40s processwrapper-sandbox ... (4 actions, 3 running)
[294 / 708] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 41s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (fr

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/storage/enginepb:enginepb_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/storage/enginepb:enginepb_test (52 packages loaded, 10856 targets configured)
Analyzing: target //pkg/storage/enginepb:enginepb_test (74 packages loaded, 13446 targets configured)
Analyzing: target //pkg/storage/enginepb:enginepb_test (138 packages loaded, 15083 targets configured)
Analyzing: target //pkg/storage/enginepb:enginepb_test (182 packages loaded, 15239 targets configured)
Analyzing: target //pkg/storage/enginepb:enginepb_test (252 packages loaded, 15816 targets configured)
Analyzing: target //pkg/storage/enginepb:enginepb_test (315 packages loaded, 17041 targets configured)
Analyzing: target //pkg/storage/enginepb:enginepb_test (386 packages loaded, 19180 targets configured)
Analyzing: target //pkg/storage/enginepb:enginepb_test (431 packages loaded, 21678 targets configured)
Analyzing: target //pkg/storage/enginepb:enginepb_test (447 packages loaded, 22825 targets configured)
INFO: Analyzed target //pkg/storage/enginepb:enginepb_test (447 packages loaded, 22826 targets configured).
INFO: Found 1 test target...
[0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[130 / 523] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[130 / 523] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[133 / 523] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 16s processwrapper-sandbox ... (2 actions, 1 running)
[313 / 1,130] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 0s processwrapper-sandbox ... (3 actions, 2 running)
[389 / 1,316] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[413 / 1,390] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 3s processwrapper-sandbox ... (4 actions running)
[413 / 1,390] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 12s processwrapper-sandbox ... (4 actions running)
[414 / 1,390] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 24s processwrapper-sandbox ... (4 actions, 3 running)
[418 / 1,390] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions, 3 running)
[422 / 1,390] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 27s processwrapper-sandbox ... (4 actions, 3 running)
[426 / 1,390] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 28s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/storage/fs:fs_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/storage/fs:fs_test (31 packages loaded, 10524 targets configured)
Analyzing: target //pkg/storage/fs:fs_test (53 packages loaded, 11922 targets configured)
Analyzing: target //pkg/storage/fs:fs_test (82 packages loaded, 14051 targets configured)
Analyzing: target //pkg/storage/fs:fs_test (149 packages loaded, 16362 targets configured)
Analyzing: target //pkg/storage/fs:fs_test (200 packages loaded, 18413 targets configured)
INFO: Analyzed target //pkg/storage/fs:fs_test (209 packages loaded, 20303 targets configured).
 checking cached actions
INFO: Found 1 test target...
[0 / 3] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[131 / 547] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[131 / 547] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[132 / 547] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 17s processwrapper-sandbox ... (4 actions, 3 running)
[314 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 1s processwrapper-sandbox ... (3 actions running)
[314 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 2s processwrapper-sandbox ... (4 actions running)
[314 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 10s processwrapper-sandbox ... (4 actions running)
[315 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 34s processwrapper-sandbox ... (4 actions, 3 running)
[318 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 35s processwrapper-sandbox ... (4 actions, 3 running)
[322 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 36s processwrapper-sandbox ... (4 actions, 3 running)
[328 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 37s processwrapper-sandbox ... (4 actions, 3 running)
[332 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 39s processwrapper-sandbox ... (4 actions, 3 running)
[332 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 40s processwrapper-sandbox ... (4 actions running)
[341 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 41s processwrapper-sandbox ... (4 actions running)
[353 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 42s processwrapper-sandbox ... (4 actions, 3 running)
[363 / 752] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 43s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not 

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/storage/metamorphic:metamorphic_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/storage/metamorphic:metamorphic_test (61 packages loaded, 10995 targets configured)
Analyzing: target //pkg/storage/metamorphic:metamorphic_test (132 packages loaded, 12166 targets configured)
Analyzing: target //pkg/storage/metamorphic:metamorphic_test (202 packages loaded, 13280 targets configured)
Analyzing: target //pkg/storage/metamorphic:metamorphic_test (279 packages loaded, 14031 targets configured)
Analyzing: target //pkg/storage/metamorphic:metamorphic_test (314 packages loaded, 14896 targets configured)
Analyzing: target //pkg/storage/metamorphic:metamorphic_test (417 packages loaded, 18784 targets configured)
Analyzing: target //pkg/storage/metamorphic:metamorphic_test (429 packages loaded, 20880 targets configured)
Analyzing: target //pkg/storage/metamorphic:metamorphic_test (447 packages loaded, 22750 targets configured)
INFO: Analyzed target //pkg/storage/metamorphic:metamorphic_test (448 packages loaded, 22835 targets configured).
INFO: Found 1 test target...
[0 / 5] [Prepa] BazelWorkspaceStatusAction stable-status.txt
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire not defined
runtime.racereleaseg: relocation target __tsan_release not defined
runtime.racereleaseacquireg: relocation target __tsan_release_acquire not defined
runtime.racereleasemergeg: relocation target __tsan_release_merge not defined
external/go_sdk/pkg/tool/linux_amd64/link: too many errors
link: error running subcommand external/go_sdk/pkg/tool/linux_amd64/link: exit status 2
[311 / 825] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 1s processwrapper-sandbox ... (2 actions running)
Target //pkg/storage/metamorphic:metamorphic_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d3669

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/testutils/floatcmp:floatcmp_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/testutils/floatcmp:floatcmp_test (8 packages loaded, 11332 targets configured)
Analyzing: target //pkg/testutils/floatcmp:floatcmp_test (18 packages loaded, 14049 targets configured)
INFO: Analyzed target //pkg/testutils/floatcmp:floatcmp_test (31 packages loaded, 14122 targets configured).
INFO: Found 1 test target...
[0 / 7] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[271 / 809] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 0s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire not defined
runtime.racereleaseg: relocation target __tsan_release not defined
runtime.racereleaseacquireg: relocation target __tsan_release_acquire not defined
runtime.racereleasemergeg: relocation target __tsan_release_merge not defined
external/go_sdk/pkg/tool/linux_amd64/link: too many errors
link: error running subcommand external/go_sdk/pkg/tool/linux_amd64/link: exit status 2
Target //pkg/testutils/floatcmp:floatcmp_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/com_github_cockroachdb_errors/errorspb/BUILD.bazel:30:11 GoCompilePkg external/com_github_cockroachdb_errors/errorspb/errorspb.a failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
INFO: Elapsed time: 4.874s, Critical Path: 1.60s
IN

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (134 packages loaded, 9638 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (247 packages loaded, 9764 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (312 packages loaded, 11104 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (381 packages loaded, 11569 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (436 packages loaded, 12586 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (452 packages loaded, 12717 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (542 packages loaded, 13697 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (618 packages loaded, 15920 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (643 packages loaded, 16094 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (697 packages loaded, 18081 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (823 packages loaded, 19854 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (881 packages loaded, 21451 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (937 packages loaded, 23481 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (969 packages loaded, 25440 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (988 packages loaded, 26872 targets configured)
Analyzing: target //pkg/testutils/fingerprintutils:fingerprintutils_test (1046 packages loaded, 27609 targets configured)
INFO: Analyzed target //pkg/testutils/fingerprintutils:fingerprintutils_test (1047 packages loaded, 27646 targets configured).
INFO: Found 1 test target...
[0 / 7] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[217 / 553] [Prepa] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool] ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable:

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/testutils/echotest:echotest_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/testutils/echotest:echotest_test (5 packages loaded, 12025 targets configured)
INFO: Analyzed target //pkg/testutils/echotest:echotest_test (11 packages loaded, 13153 targets configured).
INFO: Found 1 test target...
[0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[129 / 528] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[129 / 528] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[130 / 528] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 18s processwrapper-sandbox ... (4 actions, 3 running)
[263 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 1s processwrapper-sandbox ... (3 actions running)
[263 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 11s processwrapper-sandbox ... (3 actions running)
[264 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 24s processwrapper-sandbox ... (3 actions, 2 running)
[271 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 25s processwrapper-sandbox ... (4 actions, 3 running)
[277 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions running)
[284 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 28s processwrapper-sandbox ... (4 actions, 3 running)
[295 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 29s processwrapper-sandbox ... (4 actions, 3 running)
[303 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 30s processwrapper-sandbox ... (4 actions, 3 running)
[318 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 31s processwrapper-sandbox ... (4 actions, 3 running)
[323 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 32s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not de

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/storage/pebbleiter:pebbleiter_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/storage/pebbleiter:pebbleiter_test (4 packages loaded, 11375 targets configured)
Analyzing: target //pkg/storage/pebbleiter:pebbleiter_test (34 packages loaded, 13015 targets configured)
Analyzing: target //pkg/storage/pebbleiter:pebbleiter_test (54 packages loaded, 13892 targets configured)
INFO: Analyzed target //pkg/storage/pebbleiter:pebbleiter_test (62 packages loaded, 14118 targets configured).
INFO: Found 1 test target...
[0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[130 / 557] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[130 / 557] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[131 / 557] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 16s processwrapper-sandbox ... (4 actions, 3 running)
[264 / 818] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 1s processwrapper-sandbox ... (3 actions, 2 running)
[264 / 818] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 2s processwrapper-sandbox ... (3 actions running)
[264 / 818] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 11s processwrapper-sandbox ... (3 actions running)
[265 / 818] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 24s processwrapper-sandbox ... (3 actions, 2 running)
[274 / 818] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions, 3 running)
[282 / 818] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 27s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire not defined
runtime.racereleaseg: relocation target __tsan_release not defined
runtime.racereleaseacquireg: relocation target __tsan_release_acquire not defined
runtime.racereleasemergeg: relocation 

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/testutils:testutils_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/testutils:testutils_test (37 packages loaded, 10330 targets configured)
Analyzing: target //pkg/testutils:testutils_test (70 packages loaded, 13648 targets configured)
Analyzing: target //pkg/testutils:testutils_test (135 packages loaded, 15155 targets configured)
Analyzing: target //pkg/testutils:testutils_test (194 packages loaded, 16606 targets configured)
Analyzing: target //pkg/testutils:testutils_test (246 packages loaded, 17696 targets configured)
Analyzing: target //pkg/testutils:testutils_test (276 packages loaded, 19529 targets configured)
Analyzing: target //pkg/testutils:testutils_test (296 packages loaded, 20826 targets configured)
INFO: Analyzed target //pkg/testutils:testutils_test (305 packages loaded, 21110 targets configured).
INFO: Found 1 test target...
[0 / 3] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[131 / 561] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[131 / 561] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[133 / 561] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 16s processwrapper-sandbox ... (3 actions, 2 running)
[259 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 0s processwrapper-sandbox ... (4 actions, 3 running)
[259 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 2s processwrapper-sandbox ... (4 actions running)
[259 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 10s processwrapper-sandbox ... (4 actions running)
[260 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 30s processwrapper-sandbox ... (4 actions, 3 running)
[264 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 32s processwrapper-sandbox ... (4 actions, 3 running)
[268 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 33s processwrapper-sandbox ... (4 actions, 3 running)
[272 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 34s processwrapper-sandbox ... (4 actions, 3 running)
[276 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 35s processwrapper-sandbox ... (4 actions running)
[279 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 36s processwrapper-sandbox ... (4 actions running)
[281 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 37s processwrapper-sandbox ... (4 actions, 3 running)
[291 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 38s processwrapper-sandbox ... (4 actions, 3 running)
[303 / 712] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 39s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_ma

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/testutils/keysutils:keysutils_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/testutils/keysutils:keysutils_test (76 packages loaded, 9990 targets configured)
Analyzing: target //pkg/testutils/keysutils:keysutils_test (115 packages loaded, 12073 targets configured)
Analyzing: target //pkg/testutils/keysutils:keysutils_test (172 packages loaded, 14569 targets configured)
Analyzing: target //pkg/testutils/keysutils:keysutils_test (229 packages loaded, 16704 targets configured)
Analyzing: target //pkg/testutils/keysutils:keysutils_test (243 packages loaded, 19274 targets configured)
INFO: Analyzed target //pkg/testutils/keysutils:keysutils_test (250 packages loaded, 20631 targets configured).
INFO: Found 1 test target...
[0 / 2] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[358 / 1,197] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 1s processwrapper-sandbox ... (4 actions, 3 running)
[368 / 1,197] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 2s processwrapper-sandbox ... (4 actions, 3 running)
[372 / 1,197] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 3s processwrapper-sandbox ... (4 actions, 3 running)
[385 / 1,197] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 4s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire not defined
runtime.racereleaseg: relocation target __tsan_release not defined
runtime.racereleaseacquireg: relocation target __tsan_release_acquire not defined
runtime.racereleasemergeg: relocation target __tsan_release_merge not defined
external/go_sdk/pkg/tool/linux_amd64/link: too many errors
link: error running subcommand external/go_sdk/pkg/tool/linux_amd64/link: exit status 2
Target //pkg/testutils/keysutils:keysutils_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/testutils/lint/passes/forbiddenmethod:forbiddenmethod_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/testutils/lint/passes/forbiddenmethod:forbiddenmethod_test (21 packages loaded, 12364 targets configured)
INFO: Analyzed target //pkg/testutils/lint/passes/forbiddenmethod:forbiddenmethod_test (22 packages loaded, 13223 targets configured).
INFO: Found 1 test target...
[0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire not defined
runtime.racereleaseg: relocation target __tsan_release not defined
runtime.racereleaseacquireg: relocation target __tsan_release_acquire not defined
runtime.racereleasemergeg: relocation target __tsan_release_merge not defined
external/go_sdk/pkg/tool/linux_amd64/link: too many errors
link: error running subcommand external/go_sdk/pkg/tool/linux_amd64/link: exit status 2
[129 / 224] 1 / 1 tests, 1 failed; checking cached actions
Target //pkg/testutils/lint/passes/forbiddenmethod:forbiddenmethod_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/com_github_cockroachdb_errors/errorspb/BUILD.bazel:30:11 GoCompilePkg external/com_github_cockroachdb_errors/errorspb/errorspb.a failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
INFO: Elapsed time: 4.038s, Critical Path: 1.34s
INFO: 10 processes: 10 internal.
FAILED: Build did NOT c

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/testutils/lint/passes/errwrap:errwrap_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/testutils/lint/passes/errwrap:errwrap_test (16 packages loaded, 10889 targets configured)
INFO: Analyzed target //pkg/testutils/lint/passes/errwrap:errwrap_test (22 packages loaded, 13218 targets configured).
 checking cached actions
INFO: Found 1 test target...
[0 / 3] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[129 / 495] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[132 / 539] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[132 / 539] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[133 / 539] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 18s processwrapper-sandbox ... (4 actions, 3 running)
[143 / 539] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 17s processwrapper-sandbox ... (4 actions, 3 running)
[160 / 539] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 1s processwrapper-sandbox ... (4 actions, 3 running)
[173 / 539] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 2s processwrapper-sandbox ... (4 actions running)
[186 / 539] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 3s processwrapper-sandbox ... (4 actions running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire not defined
runtime.racereleaseg: relocation target __tsan_release not defined
runtime.racereleaseacquireg: relocation target __tsan_release_acquire not defined
runtime.racereleasemergeg: relocation target __tsan_release_merge not defined
external/go_sdk/pkg/tool/linux_amd64/link: too many errors
link: error running subcommand external/go_sdk/pkg/tool/linux_amd64/link: exit status 2
Target //pkg/testutils/lint/passes/errwrap:errwrap_test failed to build
Use

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ 397dc0b48ba959278c0ab9e0042bf5b78940b279:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/testutils/lint/passes/errcmp:errcmp_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/testutils/lint/passes/errcmp:errcmp_test (19 packages loaded, 10371 targets configured)
INFO: Analyzed target //pkg/testutils/lint/passes/errcmp:errcmp_test (22 packages loaded, 13214 targets configured).
 checking cached actions
INFO: Found 1 test target...
[0 / 6] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[129 / 495] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[131 / 546] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[131 / 546] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[132 / 546] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 17s processwrapper-sandbox ... (4 actions, 3 running)
[140 / 546] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 0s processwrapper-sandbox ... (4 actions, 3 running)
[263 / 767] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 2s processwrapper-sandbox ... (3 actions running)
[263 / 767] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 12s processwrapper-sandbox ... (3 actions running)
[264 / 767] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 25s processwrapper-sandbox ... (3 actions, 2 running)
[272 / 767] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions, 3 running)
[284 / 767] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 27s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire not defined
runtime.racereleaseg: relocation target __tsan_release not defined
runtime.racereleaseacquireg: relocation target __tsan_release_acquire not defined
runtime.racereleasemergeg: relocation target __tsan_release_merge not defined
externa

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/lexbase:lexbase_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/lexbase:lexbase_test (63 packages loaded, 11916 targets configured)
Analyzing: target //pkg/sql/lexbase:lexbase_test (145 packages loaded, 15415 targets configured)
INFO: Analyzed target //pkg/sql/lexbase:lexbase_test (157 packages loaded, 19514 targets configured).
 checking cached actions
INFO: Found 1 test target...
[0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[149 / 757] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[149 / 757] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[150 / 757] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
[160 / 771] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[160 / 771] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[160 / 771] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 10s processwrapper-sandbox ... (4 actions running)
[161 / 771] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 25s processwrapper-sandbox ... (4 actions, 3 running)
[165 / 771] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 26s processwrapper-sandbox ... (4 actions, 3 running)
[171 / 771] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 27s processwrapper-sandbox ... (4 actions, 3 running)
[177 / 771] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 28s processwrapper-sandbox ... (4 actions running)
[185 / 771] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 29s processwrapper-sandbox ... (4 actions running)
[195 / 771] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 30s processwrapper-sandbox ... (4 actions running)
[200 / 771] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 31s processwrapper-sandbox ... (4 actions, 3 running)
[209 / 771] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 32s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation 

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/lex:lex_test (0 packages loaded, 0 targets configured)
INFO: Analyzed target //pkg/sql/lex:lex_test (9 packages loaded, 13112 targets configured).
 checking cached actions
INFO: Found 1 test target...
[0 / 2] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[130 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 0s processwrapper-sandbox ... (4 actions running)
[130 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[131 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
[267 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 1s processwrapper-sandbox ... (3 actions running)
[267 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 11s processwrapper-sandbox ... (3 actions running)
[268 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 21s processwrapper-sandbox ... (3 actions, 2 running)
[273 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 22s processwrapper-sandbox ... (4 actions, 3 running)
[281 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 24s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire not defined
runtime.racereleaseg: relocation target __tsan_release not defined
runtime.racereleaseacquireg: relocation target __tsan_release_acquire not defined
runtime.racereleasemergeg: relocation target __tsan_release_merge not defined
external/go_sdk/pkg/tool/linux_amd64/link: too many errors
link: error running subcommand external/go_sdk/pkg/tool/linux_amd64/link: exit status 2
[287 / 734] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 25s processwrapper-sandbox ... (2 actions running)
Target //pkg/sql/lex:lex_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/inverted:inverted_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/inverted:inverted_test (39 packages loaded, 12308 targets configured)
INFO: Analyzed target //pkg/sql/inverted:inverted_test (66 packages loaded, 14653 targets configured).
INFO: Found 1 test target...
[0 / 7] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[10 / 333] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[12 / 333] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[18 / 333] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[25 / 333] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 4s processwrapper-sandbox ... (4 actions running)
[26 / 333] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 5s processwrapper-sandbox ... (4 actions running)
[27 / 333] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
[29 / 333] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 16s processwrapper-sandbox ... (4 actions, 3 running)
[37 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[38 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[40 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 4s processwrapper-sandbox ... (4 actions, 3 running)
[41 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 5s processwrapper-sandbox ... (4 actions running)
[42 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 8s processwrapper-sandbox ... (4 actions, 3 running)
[43 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 9s processwrapper-sandbox ... (4 actions running)
[45 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 10s processwrapper-sandbox ... (4 actions running)
[46 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 13s processwrapper-sandbox ... (4 actions, 3 running)
[47 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 14s processwrapper-sandbox ... (4 actions, 3 running)
[48 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 16s processwrapper-sandbox ... (4 actions, 3 running)
[51 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 17s processwrapper-sandbox ... (4 actions, 3 running)
[51 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 18s processwrapper-sandbox ... (4 actions running)
[53 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 19s processwrapper-sandbox ... (4 actions, 3 running)
[57 / 333] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 20s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not def

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

[21 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 8s processwrapper-sandbox ... (4 actions, 3 running)
[21 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 9s processwrapper-sandbox ... (4 actions running)
[24 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 10s processwrapper-sandbox ... (4 actions, 3 running)
[30 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 12s processwrapper-sandbox ... (4 actions, 3 running)
[34 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 13s processwrapper-sandbox ... (4 actions running)
[35 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 14s processwrapper-sandbox ... (4 actions running)
[42 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
[52 / 223] Compiling src/google/protobuf/extension_set.cc [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[56 / 223] Compiling src/google/protobuf/extension_set.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[62 / 223] Compiling src/google/protobuf/util/message_differencer.cc [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[66 / 223] Compiling src/google/protobuf/text_format.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[69 / 223] Compiling src/google/protobuf/text_format.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[74 / 223] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[77 / 223] Compiling src/google/protobuf/type.pb.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[80 / 223] Compiling src/google/protobuf/compiler/parser.cc [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[83 / 223] Compiling src/google/protobuf/util/internal/datapiece.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[85 / 223] Compiling src/google/protobuf/map_field.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[87 / 223] Compiling src/google/protobuf/generated_message_reflection.cc [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[93 / 223] Compiling src/google/protobuf/io/tokenizer.cc [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[98 / 223] Compiling src/google/protobuf/struct.pb.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[99 / 223] Compiling src/google/protobuf/descriptor_database.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[102 / 223] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[103 / 223] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 3s processwrapper-sandbox ... (4 actions running)
[104 / 223] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 4s processwrapper-sandbox ... (4 actions running)
[107 / 223] Compiling src/google/protobuf/descriptor.cc [for tool]; 5s processwrapper-sandbox ... (4 actions, 3 running)
[111 / 226] Compiling src/google/protobuf/descriptor.cc [for tool]; 7s processwrapper-sandbox ... (4 actions, 3 running)
[126 / 495] Compiling src/google/protobuf/descriptor.cc [for tool]; 8s processwrapper-sandbox ... (4 actions running)
[129 / 746] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 3s processwrapper-sandbox ... (4 actions running)
[130 / 746] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 7s processwrapper-sandbox ... (4 actions, 3 running)
[141 / 857] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 9s processwrapper-sandbox ... (4 actions, 3 running)
[141 / 857] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 10s processwrapper-sandbox ... (4 actions running)
[142 / 857] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 14s processwrapper-sandbox ... (4 actions, 3 running)
[153 / 857] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtim

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

[37 / 212] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 13s processwrapper-sandbox ... (4 actions, 3 running)
[40 / 212] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 14s processwrapper-sandbox ... (4 actions, 3 running)
[45 / 212] Compiling src/google/protobuf/repeated_field.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[51 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[55 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[57 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[59 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 5s processwrapper-sandbox ... (4 actions running)
[63 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 6s processwrapper-sandbox ... (4 actions, 3 running)
[64 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 8s processwrapper-sandbox ... (4 actions, 3 running)
[66 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 9s processwrapper-sandbox ... (4 actions, 3 running)
[69 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 10s processwrapper-sandbox ... (4 actions running)
[70 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[73 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 12s processwrapper-sandbox ... (4 actions, 3 running)
[76 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 13s processwrapper-sandbox ... (4 actions, 3 running)
[77 / 212] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 14s processwrapper-sandbox ... (4 actions, 3 running)
[94 / 212] Compiling src/google/protobuf/text_format.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire not defined
runtime.racereleaseg: relocation target __tsan_release not defined
runtime.racereleaseacquireg: relocation target __tsan_release_acquire not defined
runtime.racereleasemergeg: relocation target __tsan_release_merge not defined
external/go_sdk/pkg/tool/linux_amd64/link: too many errors
link: error running subcommand external/go_sdk/pkg/tool/linux_amd64/link: exit status 2
Target //pkg/sql/logictest/tests/5node-disk:5node-disk_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /go/src/github.com/cockroachdb/cockroach/pkg/util/log/logpb/BUILD.bazel:5:11 GoCompilePkg pkg/util/log/logpb/logpb.a [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
INFO: Elapsed time: 44.210s, Critical Path: 30.78s
INFO: 107 processes: 13 internal, 94 processwrapper-sandbox.
FAILED: Buil

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

[9 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[16 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[20 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 3s processwrapper-sandbox ... (4 actions running)
[22 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 4s processwrapper-sandbox ... (4 actions, 3 running)
[24 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 6s processwrapper-sandbox ... (4 actions running)
[27 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 7s processwrapper-sandbox ... (4 actions, 3 running)
[30 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 9s processwrapper-sandbox ... (4 actions running)
[32 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 10s processwrapper-sandbox ... (4 actions running)
[36 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions, 3 running)
[38 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 12s processwrapper-sandbox ... (4 actions, 3 running)
[40 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 13s processwrapper-sandbox ... (4 actions, 3 running)
[46 / 234] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
[52 / 234] Compiling src/google/protobuf/generated_message_table_driven_lite.cc [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[53 / 234] Compiling src/google/protobuf/generated_message_table_driven_lite.cc [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[55 / 234] Compiling src/google/protobuf/util/message_differencer.cc [for tool]; 4s processwrapper-sandbox ... (4 actions running)
[61 / 234] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 4s processwrapper-sandbox ... (4 actions running)
[63 / 234] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 5s processwrapper-sandbox ... (4 actions running)
[64 / 234] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 7s processwrapper-sandbox ... (4 actions running)
[67 / 234] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 9s processwrapper-sandbox ... (4 actions, 3 running)
[68 / 234] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 10s processwrapper-sandbox ... (4 actions running)
[69 / 234] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 12s processwrapper-sandbox ... (4 actions, 3 running)
[70 / 234] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 13s processwrapper-sandbox ... (4 actions running)
[74 / 234] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[78 / 234] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[83 / 234] Compiling src/google/protobuf/compiler/importer.cc [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[86 / 234] Compiling src/google/protobuf/compiler/parser.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[92 / 234] Compiling src/google/protobuf/io/tokenizer.cc [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[94 / 234] Compiling src/google/protobuf/map_field.cc [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[95 / 234] Compiling src/google/protobuf/generated_message_table_driven.cc [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[100 / 234] Compiling src/google/protobuf/generated_message_reflection.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[102 / 234] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[103 / 234] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 3s processwrapper-sandbox ... (4 actions running)
[105 / 234] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 4s processwrapper-sandbox ... (4 actions, 3 running)
[108 / 234] Compiling src/google/protobuf/descriptor.cc [for tool]; 5s processwrapper-sandbox ... (4 actions, 3 running)
[110 / 234] Compiling src/google/protobuf/descriptor.cc [for tool]; 6s processwrapper-sandbox ... (4 actions, 3 running)
[130 / 492] Compiling src/google/protobuf/descriptor.cc [for tool]; 7s processwrapper-sandbox ... (4 actions, 3 running)
[141 / 492] Compiling src/google/protobuf/descriptor.cc [for tool]; 8s processwrapper-sandbox ... (4 actions, 3 running)
[166 / 492] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[187 / 492] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 4s processwrapper-sandbox ... (4 actions, 3 running)
[207 / 492] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 5s processwrapper-sandbox ... (4 actions, 3 running)
[244 / 492] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 6s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_m

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (252 packages loaded, 10117 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (275 packages loaded, 10117 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (410 packages loaded, 11664 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (603 packages loaded, 12026 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (701 packages loaded, 13082 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (741 packages loaded, 13648 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (800 packages loaded, 14471 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (987 packages loaded, 18751 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (1103 packages loaded, 20552 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (1269 packages loaded, 22808 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (1373 packages loaded, 25048 targets configured)
Analyzing: target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (1488 packages loaded, 27720 targets configured)
INFO: Analyzed target //pkg/sql/logictest/tests/cockroach-go-testserver-upgrade-to-master:cockroach-go-testserver-upgrade-to-master_test (1525 packages loaded, 31279 targets configured).
 checking cached actions
INFO: Found 1 test target...
[0 / 5] [Prepa] BazelWorkspaceStatusAction stable-status.txt
ERROR: /go/src/github.com/cockroachdb/cockroach/pkg/cmd/protoc-gen-gogoroach/BUILD.bazel:16:10: GoLink pkg/cmd/protoc-gen-gogoroach/protoc-gen-gogoroach_/protoc-gen-gogoroach [for tool] failed: (Exit 1): builder failed: error executing command (from target //pkg/cmd/protoc-gen-gogoroach:protoc-gen-gogoroach) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -arc ... (remaining 65 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).d

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

[13 / 225] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[16 / 225] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 3s processwrapper-sandbox ... (4 actions running)
[18 / 225] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 4s processwrapper-sandbox ... (4 actions running)
[20 / 225] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 5s processwrapper-sandbox ... (4 actions running)
[22 / 225] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 6s processwrapper-sandbox ... (4 actions, 3 running)
[25 / 225] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 8s processwrapper-sandbox ... (4 actions, 3 running)
[28 / 225] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 9s processwrapper-sandbox ... (4 actions, 3 running)
[31 / 225] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 10s processwrapper-sandbox ... (4 actions, 3 running)
[34 / 225] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[38 / 225] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 12s processwrapper-sandbox ... (4 actions running)
[43 / 225] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 14s processwrapper-sandbox ... (4 actions, 3 running)
[49 / 225] Compiling src/google/protobuf/util/time_util.cc [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[52 / 225] Compiling src/google/protobuf/extension_set.cc [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[58 / 225] Compiling src/google/protobuf/util/message_differencer.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[63 / 225] Compiling src/google/protobuf/util/internal/json_stream_parser.cc [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[67 / 225] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[72 / 225] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[76 / 225] Compiling src/google/protobuf/api.pb.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[79 / 225] Compiling src/google/protobuf/compiler/parser.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[82 / 225] Compiling src/google/protobuf/struct.pb.cc [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[85 / 225] Compiling src/google/protobuf/generated_message_reflection.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[86 / 225] Compiling src/google/protobuf/generated_message_reflection.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[89 / 225] Compiling src/google/protobuf/text_format.cc [for tool]; 3s processwrapper-sandbox ... (4 actions running)
[95 / 225] Compiling src/google/protobuf/map_field.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[98 / 225] Compiling src/google/protobuf/type.pb.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[100 / 225] Compiling src/google/protobuf/descriptor_database.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[102 / 225] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[104 / 225] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 4s processwrapper-sandbox ... (4 actions running)
[107 / 225] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 5s processwrapper-sandbox ... (4 actions running)
[110 / 225] Compiling src/google/protobuf/descriptor.cc [for tool]; 6s processwrapper-sandbox ... (4 actions running)
[127 / 495] Compiling src/google/protobuf/descriptor.cc [for tool]; 8s processwrapper-sandbox ... (4 actions running)
[129 / 499] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 4s processwrapper-sandbox ... (3 actions running)
[129 / 499] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 5s processwrapper-sandbox ... (4 actions running)
[130 / 499] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 6s processwrapper-sandbox ... (4 actions, 3 running)
[137 / 553] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 7s processwrapper-sandbox ... (4 actions running)
[138 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 16s processwrapper-sandbox ... (4 actions, 3 running)
[144 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 17s processwrapper-sandbox ... (4 actions, 3 running)
[147 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 18s processwrapper-sandbox ... (4 actions running)
[151 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 19s processwrapper-sandbox ... (4 actions running)
[155 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 20s processwrapper-sandbox ... (4 actions running)
[160 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 21s processwrapper-sandbox ... (4 actions, 3 running)
[170 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 22s processwrapper-sandbox ... (4 actions, 3 running)
[175 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 24s processwrapper-sandbox ... (4 actions, 3 running)
[190 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 25s processwrapper-sandbox ... (4 actions, 3 running)
[215 / 553] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 23s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

[31 / 386] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
[38 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 1s processwrapper-sandbox ... (4 actions running)
[39 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 2s processwrapper-sandbox ... (4 actions running)
[40 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 3s processwrapper-sandbox ... (4 actions running)
[41 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 7s processwrapper-sandbox ... (4 actions, 3 running)
[41 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 9s processwrapper-sandbox ... (4 actions running)
[42 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 11s processwrapper-sandbox ... (4 actions running)
[43 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 16s processwrapper-sandbox ... (4 actions, 3 running)
[43 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 18s processwrapper-sandbox ... (4 actions running)
[44 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 20s processwrapper-sandbox ... (4 actions, 3 running)
[44 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 21s processwrapper-sandbox ... (4 actions running)
[45 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 24s processwrapper-sandbox ... (4 actions, 3 running)
[51 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 25s processwrapper-sandbox ... (4 actions, 2 running)
[65 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions, 3 running)
[87 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 27s processwrapper-sandbox ... (4 actions, 3 running)
[101 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 29s processwrapper-sandbox ... (4 actions, 3 running)
[115 / 386] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 30s processwrapper-sandbox ... (4 actions running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire not defined
runtime.racereleaseg: relocation target __tsan_release not defined
runtime.racereleaseacquireg: relocation target __tsan_release_acquire not defined
runtime.racereleasemergeg: relocation target __tsan_release_merge not defined
external/go_sdk/pkg/tool/linux_amd64/link: too many errors
link: error running subcommand external/go_sdk/pkg/tool/linux_amd64/link: exit status 2
[131 / 386] 1 / 1 tests, 1 failed; checking cached actions
Target //pkg/sql/logictest/tests/fakedist:fakedist_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /go/src/github.com/cockroachdb/cockroach/pkg/util/log/logpb/BUILD.bazel:5:11 GoCompilePkg pkg/util/log/logpb/logpb.a [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
INFO: Elapsed time: 59.464s, Critic

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (246 packages loaded, 10069 targets configured)
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (511 packages loaded, 11377 targets configured)
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (552 packages loaded, 11539 targets configured)
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (595 packages loaded, 12087 targets configured)
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (638 packages loaded, 14310 targets configured)
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (749 packages loaded, 16525 targets configured)
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (845 packages loaded, 17795 targets configured)
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (997 packages loaded, 20968 targets configured)
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (1098 packages loaded, 23074 targets configured)
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (1195 packages loaded, 27275 targets configured)
Analyzing: target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (1223 packages loaded, 28882 targets configured)
INFO: Analyzed target //pkg/sql/logictest/tests/fakedist-vec-off:fakedist-vec-off_test (1267 packages loaded, 29559 targets configured).
INFO: Found 1 test target...
[0 / 6] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[131 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[131 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[132 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
[287 / 845] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[303 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[303 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[304 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions, 3 running)
[307 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 27s processwrapper-sandbox ... (4 actions running)
[311 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 28s processwrapper-sandbox ... (4 actions, 3 running)
[314 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 29s processwrapper-sandbox ... (4 actions, 3 running)
[315 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 31s processwrapper-sandbox ... (4 actions running)
[316 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 33s processwrapper-sandbox ... (4 actions, 3 running)
[321 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 34s processwrapper-sandbox ... (4 actions, 3 running)
[329 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 35s processwrapper-sandbox ... (4 actions, 3 running)
[332 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 36s processwrapper-sandbox ... (4 actions running)
[339 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 38s processwrapper-sandbox ... (4 actions, 3 running)
[344 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 39s processwrapper-sandbox ... (4 actions, 3 running

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/logictest/tests/local:local_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local:local_test (231 packages loaded, 10125 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local:local_test (367 packages loaded, 12269 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local:local_test (406 packages loaded, 12387 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local:local_test (595 packages loaded, 14528 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local:local_test (691 packages loaded, 15397 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local:local_test (806 packages loaded, 18973 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local:local_test (890 packages loaded, 20481 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local:local_test (976 packages loaded, 22662 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local:local_test (1059 packages loaded, 25303 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local:local_test (1195 packages loaded, 28025 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local:local_test (1256 packages loaded, 29405 targets configured)
INFO: Analyzed target //pkg/sql/logictest/tests/local:local_test (1267 packages loaded, 29559 targets configured).
INFO: Found 1 test target...
[0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[12 / 423] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[16 / 423] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[19 / 423] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[23 / 423] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 5s processwrapper-sandbox ... (4 actions running)
[24 / 423] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 6s processwrapper-sandbox ... (4 actions, 3 running)
[24 / 423] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 8s processwrapper-sandbox ... (4 actions running)
[26 / 423] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 10s processwrapper-sandbox ... (4 actions running)
[27 / 423] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[30 / 423] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 12s processwrapper-sandbox ... (4 actions, 3 running)
[32 / 423] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 13s processwrapper-sandbox ... (4 actions, 3 running)
[34 / 423] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 14s processwrapper-sandbox ... (4 actions, 3 running)
[44 / 423] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 0s processwrapper-sandbox ... (4 actions running)
[45 / 423] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 2s processwrapper-sandbox ... (4 actions running)
[46 / 423] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 4s processwrapper-sandbox ... (4 actions, 3 running)
[48 / 423] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 5s processwrapper-sandbox ... (4 actions, 3 running)
[48 / 423] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 6s processwrapper-sandbox ... (4 actions running)
[49 / 423] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 11s processwrapper-sandbox ... (4 actions, 3 running)
[49 / 423] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 12s processwrapper-sandbox ... (4 actions running)
[50 / 423] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 14s processwrapper-sandbox ... (4 actions, 3 running)
[50 / 423] GoStdlib external/io_bazel_ru

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (226 packages loaded, 10079 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (336 packages loaded, 10922 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (358 packages loaded, 10963 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (553 packages loaded, 12720 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (661 packages loaded, 14591 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (768 packages loaded, 16461 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (858 packages loaded, 19237 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (990 packages loaded, 22301 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (1102 packages loaded, 25148 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (1195 packages loaded, 27873 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (1266 packages loaded, 29521 targets configured)
INFO: Analyzed target //pkg/sql/logictest/tests/local-legacy-schema-changer:local-legacy-schema-changer_test (1267 packages loaded, 29559 targets configured).
INFO: Found 1 test target...
[0 / 5] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[131 / 542] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[131 / 542] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[133 / 542] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (3 actions, 2 running)
[262 / 741] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[262 / 741] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[263 / 741] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions, 3 running)
[267 / 741] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 27s processwrapper-sandbox ... (4 actions, 3 running)
[271 / 741] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 28s processwrapper-sandbox ... (4 actions, 3 running)
[273 / 741] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 30s processwrapper-sandbox ... (4 actions running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages fro

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/logictest/tests/local-vec-off:local-vec-off_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-vec-off:local-vec-off_test (293 packages loaded, 11184 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-vec-off:local-vec-off_test (316 packages loaded, 12449 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-vec-off:local-vec-off_test (567 packages loaded, 13776 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-vec-off:local-vec-off_test (756 packages loaded, 16459 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-vec-off:local-vec-off_test (959 packages loaded, 19296 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-vec-off:local-vec-off_test (1087 packages loaded, 22242 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-vec-off:local-vec-off_test (1229 packages loaded, 25983 targets configured)
INFO: Analyzed target //pkg/sql/logictest/tests/local-vec-off:local-vec-off_test (1267 packages loaded, 29559 targets configured).
 checking cached actions
INFO: Found 1 test target...
[0 / 2] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[131 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 0s processwrapper-sandbox ... (4 actions running)
[131 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[132 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
[148 / 540] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 0s processwrapper-sandbox ... (4 actions, 3 running)
[167 / 540] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 1s processwrapper-sandbox ... (4 actions, 3 running)
[176 / 540] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 3s processwrapper-sandbox ... (4 actions, 3 running)
[183 / 540] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 4s processwrapper-sandbox ... (4 actions, 3 running)
[192 / 540] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 5s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /go/src/github.com/cockroachdb/cockroach/pkg/cmd/protoc-gen-gogoroach/BUILD.bazel:16:10: GoLink pkg/cmd/protoc-gen-gogoroach/protoc-gen-gogoroach_/protoc-gen-gogoroach [for tool] failed: (Exit 1): builder failed: error executing command (from target //pkg/cmd/protoc-gen-gogoroach:protoc-gen-gogoroach) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -arc ... (remaining 65 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/oidext:oidext_test (0 packages loaded, 0 targets configured)
INFO: Analyzed target //pkg/sql/oidext:oidext_test (4 packages loaded, 12269 targets configured).
 checking cached actions
INFO: Found 1 test target...
[0 / 7] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[8 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[10 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[11 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 3s processwrapper-sandbox ... (4 actions running)
[13 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 4s processwrapper-sandbox ... (4 actions, 3 running)
[16 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 6s processwrapper-sandbox ... (4 actions running)
[18 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 7s processwrapper-sandbox ... (4 actions running)
[19 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 8s processwrapper-sandbox ... (4 actions running)
[21 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 10s processwrapper-sandbox ... (4 actions, 3 running)
[22 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[24 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 13s processwrapper-sandbox ... (4 actions, 3 running)
[24 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 14s processwrapper-sandbox ... (4 actions running)
[28 / 217] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (3 actions, 2 running)
[37 / 217] Compiling src/google/protobuf/util/type_resolver_util.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[41 / 217] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[46 / 217] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 3s processwrapper-sandbox ... (4 actions running)
[48 / 217] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 5s processwrapper-sandbox ... (4 actions, 3 running)
[49 / 217] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 6s processwrapper-sandbox ... (4 actions running)
[51 / 217] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 8s processwrapper-sandbox ... (4 actions, 3 running)
[52 / 217] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 9s processwrapper-sandbox ... (4 actions, 3 running)
[53 / 217] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 10s processwrapper-sandbox ... (4 actions running)
[54 / 217] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[56 / 217] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 12s processwrapper-sandbox ... (4 actions running)
[60 / 217] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 14s processwrapper-sandbox ... (4 actions, 3 running)
[65 / 217] Compiling src/google/protobuf/wire_format.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[85 / 217] Compiling src/google/protobuf/wire_format.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[105 / 217] Compiling src/google/protobuf/util/message_differencer.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing comman

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

[58 / 404] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 7s processwrapper-sandbox ... (4 actions running)
[59 / 404] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 8s processwrapper-sandbox ... (4 actions, 3 running)
[60 / 404] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 10s processwrapper-sandbox ... (4 actions running)
[61 / 404] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 11s processwrapper-sandbox ... (4 actions, 3 running)
[64 / 404] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 13s processwrapper-sandbox ... (4 actions, 3 running)
[68 / 404] Compiling src/google/protobuf/util/internal/protostream_objectwriter.cc [for tool]; 0s processwrapper-sandbox ... (4 actions running)
[71 / 404] Compiling src/google/protobuf/util/internal/protostream_objectwriter.cc [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[76 / 404] Compiling src/google/protobuf/wire_format.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[79 / 404] Compiling src/google/protobuf/util/message_differencer.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[82 / 404] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[86 / 404] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[86 / 404] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 4s processwrapper-sandbox ... (4 actions running)
[91 / 404] Compiling src/google/protobuf/source_context.pb.cc [for tool]; 0s processwrapper-sandbox ... (4 actions running)
[94 / 404] Compiling src/google/protobuf/map_field.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[96 / 404] Compiling src/google/protobuf/text_format.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[101 / 404] Compiling src/google/protobuf/text_format.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[108 / 404] Compiling src/google/protobuf/util/internal/json_objectwriter.cc [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[113 / 404] Compiling src/google/protobuf/wrappers.pb.cc [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[130 / 492] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[153 / 492] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[185 / 492] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 3s processwrapper-sandbox ... (4 actions running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-9e4de9a90ea1/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation target __tsan_malloc not defined
runtime.runfinq: relocation target __tsan_finalizer_goroutine not defined
runtime.(*sweepLocked).sweep: relocation target __tsan_free not defined
runtime.goexit1: relocation target __tsan_go_end not defined
runtime.gfget: relocation target __tsan_malloc not defined
runtime.(*p).init: relocation target __tsan_proc_create not defined
runtime.(*p).destroy: relocation target __tsan_go_end not defined
runtime.(*p).destroy: relocation target __tsan_proc_destroy not defined
runtime.RaceDisable: relocation target __tsan_go_ignore_sync_begin not defined
runtime.RaceEnable: relocation target __tsan_go_ignore_sync_end not defined
runtime.raceinit: relocation target __tsan_init not defined
runtime.raceinit: relocation target __tsan_map_shadow not defined
runtime.racefini: relocation target __tsan_fini not defined
runtime.racemapshadow: relocation target __tsan_map_shadow not defined
runtime.racegostart: relocation target __tsan_go_start not defined
runtime.raceacquireg: relocation target __tsan_acquire not defined
runtime.raceacquirectx: relocation target __tsan_acquire not defined
runtime.racereleaseg: relocation target __tsan_release not defined
runtime.racereleaseacquireg: relocation target __tsan_release_acquire not defined
runtime.racereleasemergeg: relocation target __tsan_release_merge not defined
external/go_sdk/pkg/tool/linux_amd64/link: too many errors
link: error running subcommand external/go_sdk/pkg/tool/linux_amd64/link: exit status 2
Target //pkg/sql/opt:opt_test failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /go/src/github.com/cockroachdb/cockroach/pkg/util/log/logpb/BUIL

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (231 packages loaded, 10418 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (349 packages loaded, 11317 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (567 packages loaded, 12792 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (625 packages loaded, 13810 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (666 packages loaded, 15135 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (758 packages loaded, 17093 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (841 packages loaded, 19394 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (933 packages loaded, 21885 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (1073 packages loaded, 24409 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (1196 packages loaded, 27559 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (1252 packages loaded, 29026 targets configured)
INFO: Analyzed target //pkg/sql/logictest/tests/multiregion-9node-3region-3azs:multiregion-9node-3region-3azs_test (1267 packages loaded, 29559 targets configured).
INFO: Found 1 test target...
[0 / 5] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[131 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[131 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[132 / 540] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
[303 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[303 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[303 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[304 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 24s processwrapper-sandbox ... (4 actions, 3 running)
[308 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions, 3 running)
[312 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 27s processwrapper-sandbox ... (4 actions, 3 running)
[315 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 28s processwrapper-sandbox ... (4 actions running)
[316 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 31s processwrapper-sandbox ... (4 actions, 3 running)
[322 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 33s processwrapper-sandbox ... (4 actions, 3 running)
[326 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 34s processwrapper-sandbox ... (4 actions, 3 running)
[327 / 885] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 35s processwrapper-sandbox 

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/opt/bench:bench_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/opt/bench:bench_test (300 packages loaded, 10144 targets configured)
Analyzing: target //pkg/sql/opt/bench:bench_test (427 packages loaded, 11583 targets configured)
Analyzing: target //pkg/sql/opt/bench:bench_test (527 packages loaded, 12752 targets configured)
Analyzing: target //pkg/sql/opt/bench:bench_test (603 packages loaded, 14198 targets configured)
Analyzing: target //pkg/sql/opt/bench:bench_test (671 packages loaded, 15404 targets configured)
Analyzing: target //pkg/sql/opt/bench:bench_test (704 packages loaded, 16665 targets configured)
Analyzing: target //pkg/sql/opt/bench:bench_test (758 packages loaded, 19950 targets configured)
Analyzing: target //pkg/sql/opt/bench:bench_test (770 packages loaded, 22768 targets configured)
Analyzing: target //pkg/sql/opt/bench:bench_test (954 packages loaded, 25959 targets configured)
Analyzing: target //pkg/sql/opt/bench:bench_test (983 packages loaded, 26861 targets configured)
INFO: Analyzed target //pkg/sql/opt/bench:bench_test (1044 packages loaded, 27646 targets configured).
 checking cached actions
INFO: Found 1 test target...
[0 / 7] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[131 / 598] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[131 / 598] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[132 / 598] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
[145 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 1s processwrapper-sandbox ... (4 actions, 3 running)
[148 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 2s processwrapper-sandbox ... (4 actions, 3 running)
[149 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 3s processwrapper-sandbox ... (4 actions running)
[151 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 4s processwrapper-sandbox ... (4 actions, 3 running)
[153 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 6s processwrapper-sandbox ... (4 actions running)
[155 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 8s processwrapper-sandbox ... (4 actions, 3 running)
[157 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 9s processwrapper-sandbox ... (4 actions running)
[158 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 13s processwrapper-sandbox ... (4 actions, 3 running)
[158 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 15s processwrapper-sandbox ... (4 actions running)
[159 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 18s processwrapper-sandbox ... (4 actions, 3 running)
[159 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 19s processwrapper-sandbox ... (4 actions running)
[160 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 20s processwrapper-sandbox ... (4 actions running)
[161 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 24s processwrapper-sandbox ... (4 actions, 3 running)
[165 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 25s processwrapper-sandbox ... (4 actions, 3 running)
[171 / 709] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 26s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/b

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

stdout: 
, stderr: Loading: 
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
Loading: 
Loading: 0 packages loaded
INFO: Build options --//build/toolchains:crdb_test_flag, --@io_bazel_rules_go//go/config:race, --define, and 2 more have changed, discarding analysis cache.
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (0 packages loaded, 0 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (236 packages loaded, 10460 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (329 packages loaded, 10905 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (357 packages loaded, 11177 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (627 packages loaded, 13562 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (711 packages loaded, 16537 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (768 packages loaded, 17833 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (887 packages loaded, 21519 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (975 packages loaded, 23411 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (1076 packages loaded, 25813 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (1122 packages loaded, 28058 targets configured)
Analyzing: target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (1254 packages loaded, 29237 targets configured)
INFO: Analyzed target //pkg/sql/logictest/tests/local-mixed-22.2-23.1:local-mixed-22_2-23_1_test (1267 packages loaded, 29559 targets configured).
INFO: Found 1 test target...
[0 / 3] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[129 / 541] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[129 / 541] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[130 / 541] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 15s processwrapper-sandbox ... (4 actions, 3 running)
[260 / 740] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 1s processwrapper-sandbox ... (4 actions running)
[260 / 740] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 11s processwrapper-sandbox ... (4 actions running)
[261 / 740] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 28s processwrapper-sandbox ... (4 actions, 3 running)
[264 / 740] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 29s processwrapper-sandbox ... (4 actions, 3 running)
[268 / 740] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 30s processwrapper-sandbox ... (4 actions, 3 running)
[271 / 740] GoStdlib external/io_bazel_rules_go/stdlib_/pkg; 31s processwrapper-sandbox ... (4 actions, 3 running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go//go/tools/builders:go-protoc-bin) bazel-out/k8-opt-exec-2B5CBBC6-ST-529502155522/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,race,bazel,gss,race -package_list ... (remaining 15 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
runtime.(*mspan).setUserArenaChunkToFault: relocation target __tsan_free not defined
runtime.mallocgc: relocation

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

. failed with artifacts on master @ b5836a5b4ab6c4fe58ada6002eb8c56c4e35438e:

Analyzing: target //pkg/sql/logictest/tests/multiregion-invalid-locality:multiregion-invalid-locality_test (1079 packages loaded, 27896 targets configured)
Analyzing: target //pkg/sql/logictest/tests/multiregion-invalid-locality:multiregion-invalid-locality_test (1265 packages loaded, 29519 targets configured)
INFO: Analyzed target //pkg/sql/logictest/tests/multiregion-invalid-locality:multiregion-invalid-locality_test (1267 packages loaded, 29559 targets configured).
INFO: Found 1 test target...
[0 / 3] [Prepa] BazelWorkspaceStatusAction stable-status.txt
[7 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 0s processwrapper-sandbox ... (4 actions running)
[8 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[12 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[15 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 4s processwrapper-sandbox ... (4 actions running)
[17 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 5s processwrapper-sandbox ... (4 actions running)
[19 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 6s processwrapper-sandbox ... (4 actions running)
[21 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 8s processwrapper-sandbox ... (4 actions, 3 running)
[22 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 9s processwrapper-sandbox ... (4 actions, 3 running)
[24 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 10s processwrapper-sandbox ... (4 actions, 3 running)
[29 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 11s processwrapper-sandbox ... (4 actions running)
[34 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 13s processwrapper-sandbox ... (4 actions running)
[37 / 223] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 14s processwrapper-sandbox ... (4 actions running)
[48 / 223] Compiling src/google/protobuf/wire_format_lite.cc [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[53 / 223] Compiling src/google/protobuf/extension_set.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[58 / 223] Compiling src/google/protobuf/util/message_differencer.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[64 / 223] Compiling src/google/protobuf/util/message_differencer.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[68 / 223] Compiling src/google/protobuf/text_format.cc [for tool]; 2s processwrapper-sandbox ... (4 actions running)
[69 / 223] Compiling src/google/protobuf/text_format.cc [for tool]; 3s processwrapper-sandbox ... (4 actions running)
[75 / 223] Compiling src/google/protobuf/compiler/command_line_interface.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[78 / 223] Compiling src/google/protobuf/compiler/importer.cc [for tool]; 1s processwrapper-sandbox ... (4 actions, 3 running)
[81 / 223] Compiling src/google/protobuf/compiler/parser.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[84 / 223] Compiling src/google/protobuf/message.cc [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[86 / 223] Compiling src/google/protobuf/map_field.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[88 / 223] Compiling src/google/protobuf/generated_message_reflection.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[95 / 223] Compiling src/google/protobuf/struct.pb.cc [for tool]; 0s processwrapper-sandbox ... (4 actions, 3 running)
[98 / 223] Compiling src/google/protobuf/struct.pb.cc [for tool]; 1s processwrapper-sandbox ... (4 actions running)
[99 / 223] Compiling src/google/protobuf/descriptor_database.cc [for tool]; 2s processwrapper-sandbox ... (4 actions, 3 running)
[102 / 223] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[104 / 223] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 4s processwrapper-sandbox ... (4 actions, 3 running)
[106 / 223] Compiling src/google/protobuf/descriptor.pb.cc [for tool]; 5s processwrapper-sandbox ... (4 actions running)
[109 / 223] Compiling src/google/protobuf/descriptor.cc [for tool]; 6s processwrapper-sandbox ... (4 actions, 3 running)
[126 / 495] Compiling src/google/protobuf/descriptor.cc [for tool]; 8s processwrapper-sandbox ... (4 actions running)
[129 / 499] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 3s processwrapper-sandbox ... (4 actions, 3 running)
[129 / 499] Compiling src/google/protobuf/compiler/js/js_generator.cc [for tool]; 5s processwrapper-sandbox ... (4 actions running)
[130 / 499] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 7s processwrapper-sandbox ... (4 actions, 3 running)
[137 / 553] GoStdlib external/io_bazel_rules_go/stdlib_/pkg [for tool]; 9s processwrapper-sandbox ... (4 actions running)
[138 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 16s processwrapper-sandbox ... (4 actions, 3 running)
[143 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 17s processwrapper-sandbox ... (4 actions, 3 running)
[147 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 18s processwrapper-sandbox ... (4 actions running)
[151 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 19s processwrapper-sandbox ... (4 actions running)
[157 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 21s processwrapper-sandbox ... (4 actions, 3 running)
[168 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 22s processwrapper-sandbox ... (4 actions, 3 running)
[170 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 23s processwrapper-sandbox ... (4 actions, 3 running)
[181 / 553] GoToolchainBinaryBuild external/go_sdk/builder [for tool]; 24s processwrapper-sandbox ... (4 actions running)
ERROR: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:145:10: GoLink external/io_bazel_rules_go/go/tools/builders/go-protoc-bin_/go-protoc-bin [for tool] failed: (Exit 1): builder failed: error executing command (from target @io_bazel_rules_go

Parameters: TAGS=bazel,gss,race , stress=true

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

  • : failed #110276 : failed [C-test-failure O-robot T-testeng branch-release-23.1.9-rc.FROZEN release-blocker]

This test on roachdash | Improve this report!

healthy-pod pushed a commit to healthy-pod/cockroach that referenced this issue Sep 9, 2023
By running under `-race`, the go command defines the `race`
build tag for us [1].

Previously, we defined it under `TAGS` to let the issue poster
know that this is a failure under `race` and indicate that in
the issue.

At the time, defining the tag twice didn't cause issues
but after cockroachdb#109773, it led to build failures [2].

To reproduce locally:
```
bazel test -s --config=race pkg/util/ctxgroup:all --test_env=GOTRACEBACK=all --define gotags=bazel,gss,race
```

As a follow-up, we should find another way to let the issue
poster know that a failure was running under `race`.

[1] https://go.dev/doc/articles/race_detector#Excluding_Tests
[2] cockroachdb#109994 (comment)

Epic: none
Release note: None
craig bot pushed a commit that referenced this issue Sep 11, 2023
108175: kvserver: unskip lease preferences during outage  r=andrewbaptist a=kvoli

Previously, `TestLeasePreferenceDuringOutage` would force replication
queue processing of the test range, then assert that the range
up-replicated and lease transferred to a preferred locality.

This test was skipped, and two of the assumptions it relied on to pass
were no longer true.

After #85219, the replicate queue no longer
re-processes replicas. Instead, the queue requeues replicas after
processing, at the appropriate priority. This broke the test due to the
replicate queue being disabled, making the re-queue a no-op.

After #94023, the replicate queue no longer looked for lease transfers,
after processing a replication action. Combined with #85219, the queue
would now be guaranteed to not process both up-replication and lease
transfers from a single enqueue.

Update the test to not require a manual process, instead using a queue
range filter, which allows tests which disable automatic replication, to
still process filtered ranges via the various replica queues. Also,
ensure that the non-stopped stores are considered live targets, after
simulating an outage (bumping manual clocks, stopping servers) -- so
that the expected up-replication, then lease transfer can proceed.

Fixes: #88769
Release note: None

109432: cluster-ui: handle partial response errors on the database details page r=THardy98 a=THardy98

Part of: #102386

**Demos** (Note: these demos show this same logic applied to both the databases and database table pages as well):
DB-Console
- https://www.loom.com/share/5108dd655ad342f28323e72eaf68219c
- https://www.loom.com/share/1973383dacd7494a84e10bf39e5b85a3

This change applies the same error handling ideas from #109245 to the
database details page, enabling non-admin users to use the database
details page and providing better transparency to data fetching issues.

Errors encountered while fetching table details can be viewed via the
tooltip provided by the `Caution` icon at the table's name.
`unavailable` cells also provide a tooltip that displays the error
impacting that exact cell.

Release note (ui change): Non-admin users are able to use the database
details page.

110292: c2c: use seperate spanConfigEventStreamSpec in the span config event stream r=stevendanna a=msbutler

Previously, the spanConfigEventStream used a streamPartitionSpec, which contained a bunch of fields unecessary for span config streaming. This patch creates a new spanConfigEventStreamSpec which contains the fields only necessary for span config event streaming.

Informs #109059

Release note: None

110309: teamcity-trigger: ensure that `race` tag is only passed once r=healthy-pod a=healthy-pod

By running under `-race`, the go command defines the `race` build tag for us [1].

Previously, we defined it under `TAGS` to let the issue poster know that this is a failure under `race` and indicate that in the issue.

At the time, defining the tag twice didn't cause issues but after #109773, it led to build failures [2].

To reproduce locally:
```
bazel test -s --config=race pkg/util/ctxgroup:all --test_env=GOTRACEBACK=all --define gotags=bazel,gss,race
```

As a follow-up, we should find another way to let the issue poster know that a failure was running under `race`.

[1] https://go.dev/doc/articles/race_detector#Excluding_Tests
[2] #109994 (comment)

Epic: none
Release note: None

Co-authored-by: Austen McClernon <[email protected]>
Co-authored-by: Thomas Hardy <[email protected]>
Co-authored-by: Michael Butler <[email protected]>
Co-authored-by: healthy-pod <[email protected]>
@jlinder jlinder removed C-test-failure Broken test (automatically or manually discovered). release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Sep 13, 2023
@jlinder
Copy link
Collaborator

jlinder commented Sep 13, 2023

This issue was partly due to the race tag fix in #110309. The rest is the same problem as #99988.

@yuzefovich yuzefovich changed the title : failed *: bunch of failures Sep 15, 2023
@rickystewart
Copy link
Collaborator

Closing as dupe of #99988

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-master Failures and bugs on the master branch. O-robot Originated from a bot. T-dev-inf
Projects
None yet
Development

No branches or pull requests

4 participants