Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachtest: tpccbench/nodes=6/cpu=16/multi-az failed #69751

Closed
cockroach-teamcity opened this issue Sep 2, 2021 · 0 comments · Fixed by #69747
Closed

roachtest: tpccbench/nodes=6/cpu=16/multi-az failed #69751

cockroach-teamcity opened this issue Sep 2, 2021 · 0 comments · Fixed by #69747
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.

Comments

@cockroach-teamcity
Copy link
Member

roachtest.tpccbench/nodes=6/cpu=16/multi-az failed with artifacts on master @ 63679e7bf4a0244c6e46381751543cfe3f32388c:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/tpccbench/nodes=6/cpu=16/multi-az/run_1
	monitor.go:128,tpcc.go:1121,tpcc.go:955,test_runner.go:777: monitor failure: monitor task failed: failed with output "I210902 10:09:20.053156 1 workload/cli/run.go:408  [-] 1  creating load generator...\nInitializing 5000 connections...\nInitializing 0 idle connections...\nInitializing 5000 workers and preparing statements...\nfatal error: concurrent map writes\nfatal error: concurrent map writes\n\ngoroutine 15359 [running]:\nruntime.throw(0x5249fe4, 0x15)\n\t/usr/local/go/src/runtime/panic.go:1117 +0x72 fp=0xc002083c60 sp=0xc002083c30 pc=0x48b672\nruntime.mapassign_faststr(0x4a822a0, 0xc00065a1b0, 0xc010209ad0, 0xb, 0x2)\n\t/usr/local/go/src/runtime/map_faststr.go:211 +0x3f1 fp=0xc002083cc8 sp=0xc002083c60 pc=0x467bd1\ngithub.com/cockroachdb/cockroach/pkg/workload.(*SQLRunner).Init(0xc00a6ac940, 0x8a99a38, 0xc000c9e360, 0x5218701, 0x9, 0xc00065a180, 0xc000aea840, 0x8, 0x0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/sql_runner.go:122 +0x20b fp=0xc002083d80 sp=0xc002083cc8 pc=0x3aa030b\ngithub.com/cockroachdb/cockroach/pkg/workload/tpcc.createNewOrder(0x8a99a38, 0xc000c9e360, 0xc00000d0e0, 0xc00065a180, 0x17, 0xc0009fe300, 0x0, 0x18)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/tpcc/new_order.go:126 +0x38a fp=0xc002083e18 sp=0xc002083d80 pc=0x4204a2a\ngithub.com/cockroachdb/cockroach/pkg/workload/tpcc.newWorker(0x8a99a38, 0xc000c9e360, 0xc00000d0e0, 0xc00065a180, 0xc010212a20, 0xc000cd8360, 0x64, 0xc008b22600, 0xc0038c6e08, 0xc001cc9f94)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/tpcc/worker.go:193 +0x21b fp=0xc002083ea8 sp=0xc002083e18 pc=0x421159b\ngithub.com/cockroachdb/cockroach/pkg/workload/tpcc.(*tpcc).Ops.func3(0x495801, 0x0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/tpcc/tpcc.go:862 +0x145 fp=0xc002083f78 sp=0xc002083ea8 pc=0x421cfc5\ngolang.org/x/sync/errgroup.(*Group).Go.func1(0xc0102129f0, 0xc008261380)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59 fp=0xc002083fd0 sp=0xc002083f78 pc=0x12dd439\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc002083fd8 sp=0xc002083fd0 pc=0x4c5cc1\ncreated by golang.org/x/sync/errgroup.(*Group).Go\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66\n\ngoroutine 1 [runnable]:\nruntime.gopark(0x7aa4308, 0xc00fbb7f18, 0x160f, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc001facd18 sp=0xc001faccf8 pc=0x48e2e5\nruntime.chansend(0xc00fbb7ec0, 0xc001face30, 0xc001facd01, 0x420f89e, 0x9b6ad27f58a3180c)\n\t/usr/local/go/src/runtime/chan.go:257 +0x29a fp=0xc001facda0 sp=0xc001facd18 pc=0x458cba\nruntime.chansend1(0xc00fbb7ec0, 0xc001face30)\n\t/usr/local/go/src/runtime/chan.go:143 +0x35 fp=0xc001facdd8 sp=0xc001facda0 pc=0x458a15\ngithub.com/cockroachdb/cockroach/pkg/workload/tpcc.(*tpcc).Ops(0xc00000d0e0, 0x8a99a38, 0xc000c9e360, 0xc000c810a0, 0x6, 0xe, 0xc000c9e180, 0x0, 0x0, 0x0, ...)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/tpcc/tpcc.go:860 +0xc7e fp=0xc001fad0a0 sp=0xc001facdd8 pc=0x420f89e\ngithub.com/cockroachdb/cockroach/pkg/workload/cli.runRun.func2(0x7ff9110836f8, 0xc00000d0e0, 0xc000c810a0, 0x6, 0xe, 0xc000c9e180, 0xc000c0e040, 0x8a99a38, 0xc000c9e360, 0x1, ...)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/cli/run.go:419 +0x207 fp=0xc001fad218 sp=0xc001fad0a0 pc=0x3b19967\ngithub.com/cockroachdb/cockroach/pkg/workload/cli.runRun(0x8a501d0, 0xc00000d0e0, 0xc000c810a0, 0x6, 0xe, 0x5208372, 0x4, 0x0, 0x0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/cli/run.go:436 +0x685 fp=0xc001fad710 sp=0xc001fad218 pc=0x3b16425\ngithub.com/cockroachdb/cockroach/pkg/workload/cli.CmdHelper.func1(0xc000afa000, 0xc000c810a0, 0x6, 0xe, 0x4316377, 0x8a99a00)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/cli/run.go:223 +0x1e4 fp=0xc001fadc58 sp=0xc001fad710 pc=0x3b18d24\ngithub.com/cockroachdb/cockroach/pkg/workload/cli.HandleErrs.func1(0xc000afa000, 0xc000c810a0, 0x6, 0xe)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/cli/cli.go:87 +0x5d fp=0xc001fadcd8 sp=0xc001fadc58 pc=0x3b17f1d\ngithub.com/spf13/cobra.(*Command).execute(0xc000afa000, 0xc000c80fc0, 0xe, 0xe, 0xc000afa000, 0xc000c80fc0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:856 +0x2c2 fp=0xc001fadd98 sp=0xc001fadcd8 pc=0x3accf82\ngithub.com/spf13/cobra.(*Command).ExecuteC(0xb0b0ac0, 0xc00013a038, 0xc00039556a, 0x11)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:960 +0x375 fp=0xc001fade78 sp=0xc001fadd98 pc=0x3acdcb5\ngithub.com/spf13/cobra.(*Command).Execute(...)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:897\ngithub.com/cockroachdb/cockroach/pkg/cli.Run(...)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cli/cli.go:278\ngithub.com/cockroachdb/cockroach/pkg/cli.doMain(0xc000afa000, 0xc00039556a, 0x11, 0x0, 0x0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cli/cli.go:125 +0x11a fp=0xc001faded8 sp=0xc001fade78 pc=0x42b9bda\ngithub.com/cockroachdb/cockroach/pkg/cli.Main()\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cli/cli.go:59 +0x13a fp=0xc001fadf78 sp=0xc001faded8 pc=0x42b98ba\nmain.main()\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/cockroach/main.go:26 +0x25 fp=0xc001fadf88 sp=0xc001fadf78 pc=0x4462ba5\nruntime.main()\n\t/usr/local/go/src/runtime/proc.go:225 +0x256 fp=0xc001fadfe0 sp=0xc001fadf88 pc=0x48deb6\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc001fadfe8 sp=0xc001fadfe0 pc=0x4c5cc1\n\ngoroutine 2 [force gc (idle)]:\nruntime.gopark(0x7aa4510, 0xb44e2b0, 0x1411, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000aefb0 sp=0xc0000aef90 pc=0x48e2e5\nruntime.goparkunlock(...)\n\t/usr/local/go/src/runtime/proc.go:342\nruntime.forcegchelper()\n\t/usr/local/go/src/runtime/proc.go:276 +0xc5 fp=0xc0000aefe0 sp=0xc0000aefb0 pc=0x48e145\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000aefe8 sp=0xc0000aefe0 pc=0x4c5cc1\ncreated by runtime.init.6\n\t/usr/local/go/src/runtime/proc.go:264 +0x35\n\ngoroutine 3 [GC sweep wait]:\nruntime.gopark(0x7aa4510, 0xb456ce0, 0x140c, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000af7a8 sp=0xc0000af788 pc=0x48e2e5\nruntime.goparkunlock(...)\n\t/usr/local/go/src/runtime/proc.go:342\nruntime.bgsweep(0xc00007e070)\n\t/usr/local/go/src/runtime/mgcsweep.go:182 +0x13b fp=0xc0000af7d8 sp=0xc0000af7a8 pc=0x4781fb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000af7e0 sp=0xc0000af7d8 pc=0x4c5cc1\ncreated by runtime.gcenable\n\t/usr/local/go/src/runtime/mgc.go:217 +0x5c\n\ngoroutine 4 [GC scavenge wait]:\nruntime.gopark(0x7aa4510, 0xb457de0, 0x140d, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000aff78 sp=0xc0000aff58 pc=0x48e2e5\nruntime.goparkunlock(...)\n\t/usr/local/go/src/runtime/proc.go:342\nruntime.bgscavenge(0xc00007e070)\n\t/usr/local/go/src/runtime/mgcscavenge.go:314 +0x29c fp=0xc0000affd8 sp=0xc0000aff78 pc=0x47637c\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000affe0 sp=0xc0000affd8 pc=0x4c5cc1\ncreated by runtime.gcenable\n\t/usr/local/go/src/runtime/mgc.go:218 +0x7e\n\ngoroutine 18 [finalizer wait]:\nruntime.gopark(0x7aa4510, 0xb4b19b8, 0xc000071410, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000ae758 sp=0xc0000ae738 pc=0x48e2e5\nruntime.goparkunlock(...)\n\t/usr/local/go/src/runtime/proc.go:342\nruntime.runfinq()\n\t/usr/local/go/src/runtime/mfinal.go:175 +0xa9 fp=0xc0000ae7e0 sp=0xc0000ae758 pc=0x46d309\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000ae7e8 sp=0xc0000ae7e0 pc=0x4c5cc1\ncreated by runtime.createfing\n\t/usr/local/go/src/runtime/mfinal.go:156 +0x65\n\ngoroutine 69 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc000488980, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000396760 sp=0xc000396740 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc0003967e0 sp=0xc000396760 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0003967e8 sp=0xc0003967e0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 68 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc0002b6380, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000aaf60 sp=0xc0000aaf40 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc0000aafe0 sp=0xc0000aaf60 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000aafe8 sp=0xc0000aafe0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 21 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010e5f8, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000ab5f8 sp=0xc0000ab5d8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e5a0, 0xc0000ab6df, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc0000ab688 sp=0xc0000ab5f8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e5a0, 0xc0000ab6df, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc0000ab6b8 sp=0xc0000ab688 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc0002551e0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc0000ab7d8 sp=0xc0000ab6b8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000ab7e0 sp=0xc0000ab7d8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 22 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010e658, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000abdf8 sp=0xc0000abdd8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e600, 0xc0000abedf, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc0000abe88 sp=0xc0000abdf8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e600, 0xc0000abedf, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc0000abeb8 sp=0xc0000abe88 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc0002552b0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc0000abfd8 sp=0xc0000abeb8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000abfe0 sp=0xc0000abfd8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 23 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010e6b8, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000290df8 sp=0xc000290dd8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e660, 0xc000290edf, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000290e88 sp=0xc000290df8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e660, 0xc0000ac6df, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc000290eb8 sp=0xc000290e88 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255380)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc000290fd8 sp=0xc000290eb8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000290fe0 sp=0xc000290fd8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 24 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010e718, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000acdf8 sp=0xc0000acdd8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e6c0, 0xc0000acedf, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc0000ace88 sp=0xc0000acdf8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e6c0, 0xc0000acedf, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc0000aceb8 sp=0xc0000ace88 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255450)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc0000acfd8 sp=0xc0000aceb8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000acfe0 sp=0xc0000acfd8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 25 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010e778, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000ad5f8 sp=0xc0000ad5d8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e720, 0xc0000ad6df, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc0000ad688 sp=0xc0000ad5f8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e720, 0xc0000ad6df, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc0000ad6b8 sp=0xc0000ad688 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255520)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc0000ad7d8 sp=0xc0000ad6b8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000ad7e0 sp=0xc0000ad7d8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 26 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010e7d8, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000addf8 sp=0xc0000addd8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e780, 0xc0000adedf, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc0000ade88 sp=0xc0000addf8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e780, 0xc0000adedf, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc0000adeb8 sp=0xc0000ade88 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc0002555f0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc0000adfd8 sp=0xc0000adeb8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000adfe0 sp=0xc0000adfd8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 27 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010e838, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0003145f8 sp=0xc0003145d8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e7e0, 0xc0003146df, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000314688 sp=0xc0003145f8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e7e0, 0xc0003146df, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc0003146b8 sp=0xc000314688 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc0002556c0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc0003147d8 sp=0xc0003146b8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0003147e0 sp=0xc0003147d8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 28 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010e898, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000314df8 sp=0xc000314dd8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e840, 0xc000314edf, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000314e88 sp=0xc000314df8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e840, 0xc000314edf, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc000314eb8 sp=0xc000314e88 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255790)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc000314fd8 sp=0xc000314eb8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000314fe0 sp=0xc000314fd8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 29 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010e8f8, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0003155f8 sp=0xc0003155d8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e8a0, 0xc0003156df, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000315688 sp=0xc0003155f8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e8a0, 0xc0003156df, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc0003156b8 sp=0xc000315688 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255860)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc0003157d8 sp=0xc0003156b8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0003157e0 sp=0xc0003157d8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 30 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010e958, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000315df8 sp=0xc000315dd8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e900, 0xc000315edf, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000315e88 sp=0xc000315df8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e900, 0xc000315edf, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc000315eb8 sp=0xc000315e88 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255930)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc000315fd8 sp=0xc000315eb8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000315fe0 sp=0xc000315fd8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 31 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010e9b8, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0003165f8 sp=0xc0003165d8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e960, 0xc0003166df, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000316688 sp=0xc0003165f8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e960, 0xc0003166df, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc0003166b8 sp=0xc000316688 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255a00)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc0003167d8 sp=0xc0003166b8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0003167e0 sp=0xc0003167d8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 32 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010ea18, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000316df8 sp=0xc000316dd8 pc=0x48e2e5\nruntime.chanrecv(0xc00010e9c0, 0xc000316edf, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000316e88 sp=0xc000316df8 pc=0x459b8f\nruntime.chanrecv2(0xc00010e9c0, 0xc000316edf, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc000316eb8 sp=0xc000316e88 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255ad0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc000316fd8 sp=0xc000316eb8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000316fe0 sp=0xc000316fd8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 33 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010ea78, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0003175f8 sp=0xc0003175d8 pc=0x48e2e5\nruntime.chanrecv(0xc00010ea20, 0xc0003176df, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000317688 sp=0xc0003175f8 pc=0x459b8f\nruntime.chanrecv2(0xc00010ea20, 0xc0003176df, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc0003176b8 sp=0xc000317688 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255ba0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc0003177d8 sp=0xc0003176b8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0003177e0 sp=0xc0003177d8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 34 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010ead8, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000317df8 sp=0xc000317dd8 pc=0x48e2e5\nruntime.chanrecv(0xc00010ea80, 0xc000317edf, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000317e88 sp=0xc000317df8 pc=0x459b8f\nruntime.chanrecv2(0xc00010ea80, 0xc000317edf, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc000317eb8 sp=0xc000317e88 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255c70)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc000317fd8 sp=0xc000317eb8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000317fe0 sp=0xc000317fd8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 35 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010eb38, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0003105f8 sp=0xc0003105d8 pc=0x48e2e5\nruntime.chanrecv(0xc00010eae0, 0xc0003106df, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000310688 sp=0xc0003105f8 pc=0x459b8f\nruntime.chanrecv2(0xc00010eae0, 0xc0003106df, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc0003106b8 sp=0xc000310688 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255d40)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc0003107d8 sp=0xc0003106b8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0003107e0 sp=0xc0003107d8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 36 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00010eb98, 0x170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000310df8 sp=0xc000310dd8 pc=0x48e2e5\nruntime.chanrecv(0xc00010eb40, 0xc000310edf, 0x1, 0x0)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000310e88 sp=0xc000310df8 pc=0x459b8f\nruntime.chanrecv2(0xc00010eb40, 0xc000310edf, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc000310eb8 sp=0xc000310e88 pc=0x45980b\ngithub.com/klauspost/compress/zstd.(*blockDec).startDecoder(0xc000255e10)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:215 +0x149 fp=0xc000310fd8 sp=0xc000310eb8 pc=0xcf17e9\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000310fe0 sp=0xc000310fd8 pc=0x4c5cc1\ncreated by github.com/klauspost/compress/zstd.newBlockDec\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/klauspost/compress/zstd/blockdec.go:118 +0x173\n\ngoroutine 37 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc00028e3b8, 0x49170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0003116a8 sp=0xc000311688 pc=0x48e2e5\nruntime.chanrecv(0xc00028e360, 0xc0003117b8, 0xc00029c001, 0xc00028e360)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000311738 sp=0xc0003116a8 pc=0x459b8f\nruntime.chanrecv2(0xc00028e360, 0xc0003117b8, 0x0)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc000311768 sp=0xc000311738 pc=0x45980b\ngithub.com/cockroachdb/cockroach/pkg/util/log.flushDaemon()\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:75 +0x74 fp=0xc0003117e0 sp=0xc000311768 pc=0xf3eed4\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0003117e8 sp=0xc0003117e0 pc=0x4c5cc1\ncreated by github.com/cockroachdb/cockroach/pkg/util/log.init.6\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:41 +0x35\n\ngoroutine 38 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc0000da2f8, 0xb4b170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000311e90 sp=0xc000311e70 pc=0x48e2e5\nruntime.chanrecv(0xc0000da2a0, 0xc000311fc0, 0xb45e601, 0x7aa46d8)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000311f20 sp=0xc000311e90 pc=0x459b8f\nruntime.chanrecv2(0xc0000da2a0, 0xc000311fc0, 0x1)\n\t/usr/local/go/src/runtime/chan.go:444 +0x2b fp=0xc000311f50 sp=0xc000311f20 pc=0x45980b\ngithub.com/cockroachdb/cockroach/pkg/util/log.signalFlusher()\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:98 +0x12c fp=0xc000311fe0 sp=0xc000311f50 pc=0xf3f16c\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000311fe8 sp=0xc000311fe0 pc=0x4c5cc1\ncreated by github.com/cockroachdb/cockroach/pkg/util/log.init.6\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:42 +0x4d\n\ngoroutine 5 [select, locked to thread]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000aa610 sp=0xc0000aa5f0 pc=0x48e2e5\nruntime.selectgo(0xc0000aa7a8, 0xc0000aa7a0, 0x0, 0x0, 0x2, 0x1, 0x1, 0x1)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc0000aa748 sp=0xc0000aa610 pc=0x49f557\nruntime.ensureSigM.func1()\n\t/usr/local/go/src/runtime/signal_unix.go:889 +0x1cd fp=0xc0000aa7e0 sp=0xc0000aa748 pc=0x4be08d\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000aa7e8 sp=0xc0000aa7e0 pc=0x4c5cc1\ncreated by runtime.ensureSigM\n\t/usr/local/go/src/runtime/signal_unix.go:872 +0xd5\n\ngoroutine 50 [syscall]:\nruntime.notetsleepg(0xb4b2cc0, 0xffffffffffffffff, 0x0)\n\t/usr/local/go/src/runtime/lock_futex.go:235 +0x34 fp=0xc0000ac798 sp=0xc0000ac768 pc=0x45fa54\nos/signal.signal_recv(0x7aa4700)\n\t/usr/local/go/src/runtime/sigqueue.go:168 +0xa5 fp=0xc0000ac7c0 sp=0xc0000ac798 pc=0x4c21e5\nos/signal.loop()\n\t/usr/local/go/src/os/signal/signal_unix.go:23 +0x25 fp=0xc0000ac7e0 sp=0xc0000ac7c0 pc=0xf217c5\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000ac7e8 sp=0xc0000ac7e0 pc=0x4c5cc1\ncreated by os/signal.Notify.func1.1\n\t/usr/local/go/src/os/signal/signal.go:151 +0x45\n\ngoroutine 67 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc000488940, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000312760 sp=0xc000312740 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc0003127e0 sp=0xc000312760 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0003127e8 sp=0xc0003127e0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 40 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc000488960, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000312f60 sp=0xc000312f40 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc000312fe0 sp=0xc000312f60 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000312fe8 sp=0xc000312fe0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 41 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc000394000, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000313760 sp=0xc000313740 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc0003137e0 sp=0xc000313760 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0003137e8 sp=0xc0003137e0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 51 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc000394020, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc00039a760 sp=0xc00039a740 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc00039a7e0 sp=0xc00039a760 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00039a7e8 sp=0xc00039a7e0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 52 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc0002b6360, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc00039af60 sp=0xc00039af40 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc00039afe0 sp=0xc00039af60 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00039afe8 sp=0xc00039afe0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 42 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc0004889a0, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000313f60 sp=0xc000313f40 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc000313fe0 sp=0xc000313f60 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000313fe8 sp=0xc000313fe0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 43 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc000394040, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000516760 sp=0xc000516740 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc0005167e0 sp=0xc000516760 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005167e8 sp=0xc0005167e0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 53 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc000394060, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc00039b760 sp=0xc00039b740 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc00039b7e0 sp=0xc00039b760 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00039b7e8 sp=0xc00039b7e0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 54 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc0002b63a0, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc00039bf60 sp=0xc00039bf40 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc00039bfe0 sp=0xc00039bf60 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00039bfe8 sp=0xc00039bfe0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 70 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc0002b63c0, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000396f60 sp=0xc000396f40 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc000396fe0 sp=0xc000396f60 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000396fe8 sp=0xc000396fe0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 71 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc0004889c0, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000397760 sp=0xc000397740 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc0003977e0 sp=0xc000397760 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0003977e8 sp=0xc0003977e0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 55 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc0004889e0, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc00039c760 sp=0xc00039c740 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc00039c7e0 sp=0xc00039c760 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00039c7e8 sp=0xc00039c7e0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 44 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc0002b63e0, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000516f60 sp=0xc000516f40 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc000516fe0 sp=0xc000516f60 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000516fe8 sp=0xc000516fe0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 72 [GC worker (idle)]:\nruntime.gopark(0x7aa43a8, 0xc0002b6400, 0x1418, 0x0)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000397f60 sp=0xc000397f40 pc=0x48e2e5\nruntime.gcBgMarkWorker()\n\t/usr/local/go/src/runtime/mgc.go:1911 +0xfb fp=0xc000397fe0 sp=0xc000397f60 pc=0x47105b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000397fe8 sp=0xc000397fe0 pc=0x4c5cc1\ncreated by runtime.gcBgMarkStartWorkers\n\t/usr/local/go/src/runtime/mgc.go:1835 +0x37\n\ngoroutine 146 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000b1e20 sp=0xc0000b1e00 pc=0x48e2e5\nruntime.selectgo(0xc0000b1fa8, 0xc0000b1f98, 0x0, 0x0, 0x2, 0x1, 0x0, 0x57ef871)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc0000b1f58 sp=0xc0000b1e20 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc0001fa000)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc0000b1fd8 sp=0xc0000b1f58 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000b1fe0 sp=0xc0000b1fd8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 7 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000b1600 sp=0xc0000b15e0 pc=0x48e2e5\nruntime.selectgo(0xc0000b1798, 0xc0000b177c, 0x0, 0x0, 0x3, 0x1, 0x0, 0x0)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc0000b1738 sp=0xc0000b1600 pc=0x49f557\ngo.opencensus.io/stats/view.(*worker).start(0xc00052e080)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/go.opencensus.io/stats/view/worker.go:276 +0xcd fp=0xc0000b17d8 sp=0xc0000b1738 pc=0x39af94d\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000b17e0 sp=0xc0000b17d8 pc=0x4c5cc1\ncreated by go.opencensus.io/stats/view.init.0\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/go.opencensus.io/stats/view/worker.go:34 +0x68\n\ngoroutine 45 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000398620 sp=0xc000398600 pc=0x48e2e5\nruntime.selectgo(0xc0003987a8, 0xc000398798, 0x0, 0x0, 0x2, 0x1, 0x0, 0x547f1de)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000398758 sp=0xc000398620 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc000874070)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc0003987d8 sp=0xc000398758 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0003987e0 sp=0xc0003987d8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 9 [chan receive]:\nruntime.gopark(0x7aa4308, 0xc0001405f8, 0x45170e, 0x2)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000398e30 sp=0xc000398e10 pc=0x48e2e5\nruntime.chanrecv(0xc0001405a0, 0xc000398f70, 0xc000d98a01, 0xc000140101)\n\t/usr/local/go/src/runtime/chan.go:576 +0x36f fp=0xc000398ec0 sp=0xc000398e30 pc=0x459b8f\nruntime.chanrecv1(0xc0001405a0, 0xc000398f70)\n\t/usr/local/go/src/runtime/chan.go:439 +0x2b fp=0xc000398ef0 sp=0xc000398ec0 pc=0x4597cb\ngithub.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0.func1()\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/goschedstats/runnable.go:165 +0x16b fp=0xc000398fe0 sp=0xc000398ef0 pc=0x3b5ba6b\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000398fe8 sp=0xc000398fe0 pc=0x4c5cc1\ncreated by github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/goschedstats/runnable.go:157 +0x35\n\ngoroutine 10 [IO wait]:\nruntime.gopark(0x7aa44e8, 0x7ff93c23b718, 0x1b02, 0x5)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000297b50 sp=0xc000297b30 pc=0x48e2e5\nruntime.netpollblock(0x7ff93c23b6f0, 0x72, 0xd0)\n\t/usr/local/go/src/runtime/netpoll.go:438 +0x9b fp=0xc000297b88 sp=0xc000297b50 pc=0x48687b\ninternal/poll.runtime_pollWait(0x7ff93c23b6f0, 0x72, 0x0)\n\t/usr/local/go/src/runtime/netpoll.go:222 +0x55 fp=0xc000297bb0 sp=0xc000297b88 pc=0x4bfe35\ninternal/poll.(*pollDesc).wait(0xc001442018, 0x72, 0x0, 0x0, 0x5210cbf)\n\t/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45 fp=0xc000297be0 sp=0xc000297bb0 pc=0x539dc5\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/local/go/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Accept(0xc001442000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)\n\t/usr/local/go/src/internal/poll/fd_unix.go:401 +0x212 fp=0xc000297c60 sp=0xc000297be0 pc=0x53cc32\nnet.(*netFD).accept(0xc001442000, 0x30, 0x30, 0x7ff93c2ab108)\n\t/usr/local/go/src/net/fd_unix.go:172 +0x45 fp=0xc000297d28 sp=0xc000297c60 pc=0x5ee805\nnet.(*TCPListener).accept(0xc000647698, 0xc000297da0, 0x461658, 0x30)\n\t/usr/local/go/src/net/tcpsock_posix.go:139 +0x32 fp=0xc000297d68 sp=0xc000297d28 pc=0x60d7b2\nnet.(*TCPListener).Accept(0xc000647698, 0x4ee40a0, 0xc00143c510, 0x4a4b980, 0xb066520)\n\t/usr/local/go/src/net/tcpsock.go:261 +0x65 fp=0xc000297da8 sp=0xc000297d68 pc=0x60c5a5\nnet/http.(*onceCloseListener).Accept(0xc00143c450, 0xc00013a038, 0x4a4b980, 0xb066520, 0x517c340)\n\t<autogenerated>:1 +0x3c fp=0xc000297de0 sp=0xc000297da8 pc=0x95d8bc\nnet/http.(*Server).Serve(0xc000c81260, 0x8a73d40, 0xc000647698, 0x0, 0x0)\n\t/usr/local/go/src/net/http/server.go:2961 +0x285 fp=0xc000297f08 sp=0xc000297de0 pc=0x9355e5\nnet/http.(*Server).ListenAndServe(0xc000c81260, 0xc000c81260, 0x1)\n\t/usr/local/go/src/net/http/server.go:2890 +0xba fp=0xc000297f58 sp=0xc000297f08 pc=0x9352fa\nnet/http.ListenAndServe(...)\n\t/usr/local/go/src/net/http/server.go:3144\ngithub.com/cockroachdb/cockroach/pkg/workload/cli.startPProfEndPoint.func1(0x8a99a00, 0xc00013a038)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/cli/run.go:341 +0xca fp=0xc000297fd0 sp=0xc000297f58 pc=0x3b1944a\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000297fd8 sp=0xc000297fd0 pc=0x4c5cc1\ncreated by github.com/cockroachdb/cockroach/pkg/workload/cli.startPProfEndPoint\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/cli/run.go:340 +0xbf\n\ngoroutine 82 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000514e18 sp=0xc000514df8 pc=0x48e2e5\nruntime.selectgo(0xc000514f98, 0xc000514f90, 0x0, 0x0, 0x2, 0x1, 0x0, 0x0)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000514f50 sp=0xc000514e18 pc=0x49f557\ndatabase/sql.(*DB).connectionOpener(0xc000c12000, 0x8a999c8, 0xc000c0e000)\n\t/usr/local/go/src/database/sql/sql.go:1133 +0xb5 fp=0xc000514fc8 sp=0xc000514f50 pc=0xfb1ad5\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000514fd0 sp=0xc000514fc8 pc=0x4c5cc1\ncreated by database/sql.OpenDB\n\t/usr/local/go/src/database/sql/sql.go:740 +0x12a\n\ngoroutine 83 [IO wait]:\nruntime.gopark(0x7aa44e8, 0x7ff93c23b630, 0x1b02, 0x5)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000c1b18 sp=0xc0000c1af8 pc=0x48e2e5\nruntime.netpollblock(0x7ff93c23b608, 0x72, 0xd0)\n\t/usr/local/go/src/runtime/netpoll.go:438 +0x9b fp=0xc0000c1b50 sp=0xc0000c1b18 pc=0x48687b\ninternal/poll.runtime_pollWait(0x7ff93c23b608, 0x72, 0x0)\n\t/usr/local/go/src/runtime/netpoll.go:222 +0x55 fp=0xc0000c1b78 sp=0xc0000c1b50 pc=0x4bfe35\ninternal/poll.(*pollDesc).wait(0xc0015b0018, 0x72, 0x0, 0x0, 0x5210cbf)\n\t/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45 fp=0xc0000c1ba8 sp=0xc0000c1b78 pc=0x539dc5\ninternal/poll.(*pollDesc).waitRead(...)\n\t/usr/local/go/src/internal/poll/fd_poll_runtime.go:92\ninternal/poll.(*FD).Accept(0xc0015b0000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)\n\t/usr/local/go/src/internal/poll/fd_unix.go:401 +0x212 fp=0xc0000c1c28 sp=0xc0000c1ba8 pc=0x53cc32\nnet.(*netFD).accept(0xc0015b0000, 0x30, 0x30, 0x7ff93c2ab108)\n\t/usr/local/go/src/net/fd_unix.go:172 +0x45 fp=0xc0000c1cf0 sp=0xc0000c1c28 pc=0x5ee805\nnet.(*TCPListener).accept(0xc0006476b0, 0xc0000c1d68, 0x461658, 0x30)\n\t/usr/local/go/src/net/tcpsock_posix.go:139 +0x32 fp=0xc0000c1d30 sp=0xc0000c1cf0 pc=0x60d7b2\nnet.(*TCPListener).Accept(0xc0006476b0, 0x4ee40a0, 0xc00143c630, 0x4a4b980, 0xb066520)\n\t/usr/local/go/src/net/tcpsock.go:261 +0x65 fp=0xc0000c1d70 sp=0xc0000c1d30 pc=0x60c5a5\nnet/http.(*onceCloseListener).Accept(0xc00143c570, 0xc00013a038, 0x4a4b980, 0xb066520, 0x517c340)\n\t<autogenerated>:1 +0x3c fp=0xc0000c1da8 sp=0xc0000c1d70 pc=0x95d8bc\nnet/http.(*Server).Serve(0xc000738c40, 0x8a73d40, 0xc0006476b0, 0x0, 0x0)\n\t/usr/local/go/src/net/http/server.go:2961 +0x285 fp=0xc0000c1ed0 sp=0xc0000c1da8 pc=0x9355e5\nnet/http.(*Server).ListenAndServe(0xc000738c40, 0xc000738c40, 0x0)\n\t/usr/local/go/src/net/http/server.go:2890 +0xba fp=0xc0000c1f20 sp=0xc0000c1ed0 pc=0x9352fa\nnet/http.ListenAndServe(...)\n\t/usr/local/go/src/net/http/server.go:3144\ngithub.com/cockroachdb/cockroach/pkg/workload/cli.runRun.func1(0xc000c9e180)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/cli/run.go:398 +0x179 fp=0xc0000c1fd8 sp=0xc0000c1f20 pc=0x3b19679\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000c1fe0 sp=0xc0000c1fd8 pc=0x4c5cc1\ncreated by github.com/cockroachdb/cockroach/pkg/workload/cli.runRun\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/workload/cli/run.go:397 +0x4ef\n\ngoroutine 130 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000515e20 sp=0xc000515e00 pc=0x48e2e5\nruntime.selectgo(0xc000515fa8, 0xc000515f98, 0x0, 0x0, 0x2, 0x1, 0x4958f6, 0x7aa4400)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000515f58 sp=0xc000515e20 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc000152070)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc000515fd8 sp=0xc000515f58 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000515fe0 sp=0xc000515fd8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 118 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000c2b620 sp=0xc000c2b600 pc=0x48e2e5\nruntime.selectgo(0xc000c2b7a8, 0xc000c2b798, 0x0, 0x0, 0x2, 0x1, 0x4958f6, 0x7aa4400)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000c2b758 sp=0xc000c2b620 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc0002adc70)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc000c2b7d8 sp=0xc000c2b758 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000c2b7e0 sp=0xc000c2b7d8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 60 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc001712e20 sp=0xc001712e00 pc=0x48e2e5\nruntime.selectgo(0xc001712fa8, 0xc001712f98, 0x0, 0x0, 0x2, 0x1, 0x70, 0x79)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc001712f58 sp=0xc001712e20 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc0001a2930)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc001712fd8 sp=0xc001712f58 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc001712fe0 sp=0xc001712fd8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 56 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000519620 sp=0xc000519600 pc=0x48e2e5\nruntime.selectgo(0xc0005197a8, 0xc000519798, 0x0, 0x0, 0x2, 0xc000cd6901, 0x1000000004958f6, 0x7aa4400)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000519758 sp=0xc000519620 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc0001a2310)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc0005197d8 sp=0xc000519758 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005197e0 sp=0xc0005197d8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 211 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc001713620 sp=0xc001713600 pc=0x48e2e5\nruntime.selectgo(0xc0017137a8, 0xc001713798, 0x0, 0x0, 0x2, 0x1, 0x4958f6, 0x7aa4400)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc001713758 sp=0xc001713620 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc000534070)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc0017137d8 sp=0xc001713758 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0017137e0 sp=0xc0017137d8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 114 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000c28e20 sp=0xc000c28e00 pc=0x48e2e5\nruntime.selectgo(0xc000c28fa8, 0xc000c28f98, 0x0, 0x0, 0x2, 0x1, 0x100000000000000, 0x0)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000c28f58 sp=0xc000c28e20 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc0002ad9d0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc000c28fd8 sp=0xc000c28f58 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000c28fe0 sp=0xc000c28fd8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 11 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000c26e20 sp=0xc000c26e00 pc=0x48e2e5\nruntime.selectgo(0xc000c26fa8, 0xc000c26f98, 0x0, 0x0, 0x2, 0x1, 0x100000000000000, 0x0)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000c26f58 sp=0xc000c26e20 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc00024bea0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc000c26fd8 sp=0xc000c26f58 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000c26fe0 sp=0xc000c26fd8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 162 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000c28620 sp=0xc000c28600 pc=0x48e2e5\nruntime.selectgo(0xc000c287a8, 0xc000c28798, 0x0, 0x0, 0x2, 0x1, 0x100000000000000, 0x0)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000c28758 sp=0xc000c28620 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc00086a070)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc000c287d8 sp=0xc000c28758 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000c287e0 sp=0xc000c287d8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 94 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000c27e20 sp=0xc000c27e00 pc=0x48e2e5\nruntime.selectgo(0xc000c27fa8, 0xc000c27f98, 0x0, 0x0, 0x2, 0x1, 0x4958f6, 0x7aa4400)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000c27f58 sp=0xc000c27e20 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc001006070)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc000c27fd8 sp=0xc000c27f58 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000c27fe0 sp=0xc000c27fd8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 64 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000c2ae20 sp=0xc000c2ae00 pc=0x48e2e5\nruntime.selectgo(0xc000c2afa8, 0xc000c2af98, 0x0, 0x0, 0x2, 0x1, 0x4958f6, 0x7aa4400)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000c2af58 sp=0xc000c2ae20 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc0002ec150)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc000c2afd8 sp=0xc000c2af58 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000c2afe0 sp=0xc000c2afd8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 168 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000c27620 sp=0xc000c27600 pc=0x48e2e5\nruntime.selectgo(0xc000c277a8, 0xc000c27798, 0x0, 0x0, 0x2, 0x1, 0x4958f6, 0x7aa4400)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000c27758 sp=0xc000c27620 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc0001a2380)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc000c277d8 sp=0xc000c27758 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000c277e0 sp=0xc000c277d8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 148 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000517e20 sp=0xc000517e00 pc=0x48e2e5\nruntime.selectgo(0xc000517fa8, 0xc000517f98, 0x0, 0x0, 0x2, 0x1, 0x100000000000000, 0x0)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000517f58 sp=0xc000517e20 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc0001fab60)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc000517fd8 sp=0xc000517f58 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000517fe0 sp=0xc000517fd8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 15 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000519e20 sp=0xc000519e00 pc=0x48e2e5\nruntime.selectgo(0xc000519fa8, 0xc000519f98, 0x0, 0x0, 0x2, 0xc00033b001, 0x6, 0x0)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000519f58 sp=0xc000519e20 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc000308230)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc000519fd8 sp=0xc000519f58 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000519fe0 sp=0xc000519fd8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 73 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000517620 sp=0xc000517600 pc=0x48e2e5\nruntime.selectgo(0xc0005177a8, 0xc000517798, 0x0, 0x0, 0x2, 0x1, 0x100000000000000, 0x0)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000517758 sp=0xc000517620 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc0004ff420)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc0005177d8 sp=0xc000517758 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0005177e0 sp=0xc0005177d8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 642 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc0000b0620 sp=0xc0000b0600 pc=0x48e2e5\nruntime.selectgo(0xc0000b07a8, 0xc0000b0798, 0x0, 0x0, 0x2, 0x1, 0x4958f6, 0x7aa4400)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc0000b0758 sp=0xc0000b0620 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc000532000)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc0000b07d8 sp=0xc0000b0758 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000b07e0 sp=0xc0000b07d8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 133 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc000c2be20 sp=0xc000c2be00 pc=0x48e2e5\nruntime.selectgo(0xc000c2bfa8, 0xc000c2bf98, 0x0, 0x0, 0x2, 0x1, 0x4958f6, 0x7aa4400)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc000c2bf58 sp=0xc000c2be20 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc000152770)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc000c2bfd8 sp=0xc000c2bf58 pc=0x3a95fdb\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000c2bfe0 sp=0xc000c2bfd8 pc=0x4c5cc1\ncreated by github.com/jackc/pgx/v4/pgxpool.ConnectConfig\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:223 +0x27f\n\ngoroutine 226 [select]:\nruntime.gopark(0x7aa4560, 0x0, 0x1809, 0x1)\n\t/usr/local/go/src/runtime/proc.go:336 +0xe5 fp=0xc001810e20 sp=0xc001810e00 pc=0x48e2e5\nruntime.selectgo(0xc001810fa8, 0xc001810f98, 0x0, 0x0, 0x2, 0x1, 0x7f, 0xc00181dc00)\n\t/usr/local/go/src/runtime/select.go:327 +0xef7 fp=0xc001810f58 sp=0xc001810e20 pc=0x49f557\ngithub.com/jackc/pgx/v4/pgxpool.(*Pool).backgroundHealthCheck(0xc000e5e0e0)\n\t/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/jackc/pgx/v4/pgxpool/pool.go:342 +0xbb fp=0xc001810fd8 sp=0xc001810f58 pc=0x3a95fdb\nruntime.goexit()\n\t/us
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Sep 2, 2021
@craig craig bot closed this as completed in #69747 Sep 2, 2021
@craig craig bot closed this as completed in a46e3a2 Sep 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant