Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql/distsql: TestAggregatorAgainstProcessor failed #62141

Closed
cockroach-teamcity opened this issue Mar 17, 2021 · 3 comments · Fixed by #62183
Closed

sql/distsql: TestAggregatorAgainstProcessor failed #62141

cockroach-teamcity opened this issue Mar 17, 2021 · 3 comments · Fixed by #62183
Assignees
@otan @andyyang890
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.

Comments

@cockroach-teamcity
Copy link
Member

(sql/distsql).TestAggregatorAgainstProcessor failed on master@597e4a8c487e3c23d64885563d608a692b59055c:

runtime/pprof.Do(0x503d4c0, 0xc000e90030, 0xc000599480, 0x1, 0x1, 0xc000117fb8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcc
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc00068f200)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:175 +0x89
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:167 +0x1a5

goroutine 12 [chan receive]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x503d4c0, 0xc000a921b0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:176 +0x76
runtime/pprof.Do(0x503d4c0, 0xc000a921b0, 0xc000599480, 0x1, 0x1, 0xc00009b7b8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcc
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc00068f300)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:175 +0x89
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:167 +0x1a5

goroutine 13 [chan receive]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x503d4c0, 0xc000a92270)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:176 +0x76
runtime/pprof.Do(0x503d4c0, 0xc000a92270, 0xc000599480, 0x1, 0x1, 0xc00009bfb8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcc
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc00068f400)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:175 +0x89
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:167 +0x1a5

goroutine 14 [sync.Cond.Wait]:
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:312
sync.runtime_notifyListWait(0xc000374ae0, 0x0)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc000374ad0)
	/usr/local/go/src/sync/cond.go:56 +0x9d
github.com/cockroachdb/pebble/internal/record.(*flusherCond).Wait(...)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/internal/record/log_writer.go:193
github.com/cockroachdb/pebble/internal/record.(*LogWriter).flushLoop(0xc000374a00, 0x503d4c0, 0xc000c9a030)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/internal/record/log_writer.go:396 +0x10b
runtime/pprof.Do(0x503d4c0, 0xc000c9a030, 0xc000598d40, 0x1, 0x1, 0xc000112fb8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcc
github.com/cockroachdb/pebble/internal/record.NewLogWriter.func2(0xc000374a00)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/internal/record/log_writer.go:314 +0x89
created by github.com/cockroachdb/pebble/internal/record.NewLogWriter
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/internal/record/log_writer.go:313 +0x2ea


ERROR: exit status 2

1 runs completed, 1 failures, over 0s
context canceled
More

Parameters:

  • TAGS=
  • GOFLAGS=-parallel=4
make stressrace TESTS=TestAggregatorAgainstProcessor PKG=./pkg/sql/distsql TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Mar 17, 2021
@cockroach-teamcity
Copy link
Member Author

(sql/distsql).TestAggregatorAgainstProcessor failed on master@597e4a8c487e3c23d64885563d608a692b59055c:

runtime/pprof.Do(0x7b13f20, 0xc0006284b0, 0xc000483bc0, 0x1, 0x1, 0xc00049efb8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0x122
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc000c40000)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:175 +0xc5
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:167 +0x345

goroutine 113 [chan receive]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x7b13f20, 0xc000628510)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:176 +0xa9
runtime/pprof.Do(0x7b13f20, 0xc000628510, 0xc000483bc0, 0x1, 0x1, 0xc00049e7b8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0x122
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc000c40100)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:175 +0xc5
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:167 +0x345

goroutine 114 [chan receive]:
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x7b13f20, 0xc00055ef30)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:176 +0xa9
runtime/pprof.Do(0x7b13f20, 0xc00055ef30, 0xc000483bc0, 0x1, 0x1, 0xc00049b7b8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0x122
github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc000c40300)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:175 +0xc5
created by github.com/cockroachdb/pebble.(*tableCacheShard).init
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:167 +0x345

goroutine 115 [sync.Cond.Wait]:
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:312
sync.runtime_notifyListWait(0xc000374ae0, 0xc000000000)
	/usr/local/go/src/runtime/sema.go:513 +0xf8
sync.(*Cond).Wait(0xc000374ad0)
	/usr/local/go/src/sync/cond.go:56 +0x90
github.com/cockroachdb/pebble/internal/record.(*flusherCond).Wait(...)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/internal/record/log_writer.go:193
github.com/cockroachdb/pebble/internal/record.(*LogWriter).flushLoop(0xc000374a00, 0x7b13f20, 0xc00055f110)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/internal/record/log_writer.go:396 +0x146
runtime/pprof.Do(0x7b13f20, 0xc00055f110, 0xc000483480, 0x1, 0x1, 0xc00049afb8)
	/usr/local/go/src/runtime/pprof/runtime.go:40 +0x122
github.com/cockroachdb/pebble/internal/record.NewLogWriter.func2(0xc000374a00)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/internal/record/log_writer.go:314 +0xc5
created by github.com/cockroachdb/pebble/internal/record.NewLogWriter
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/internal/record/log_writer.go:313 +0x51d


ERROR: exit status 2

12 runs completed, 1 failures, over 4m38s
context canceled
More

Parameters:

  • TAGS=
  • GOFLAGS=-race -parallel=4
make stressrace TESTS=TestAggregatorAgainstProcessor PKG=./pkg/sql/distsql TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

See this test on roachdash
powered by pkg/cmd/internal/issues

@yuzefovich
Copy link
Member 

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7f5bb9cff391]

runtime stack:
runtime.throw(0x4428440, 0x2a)
	/usr/local/go/src/runtime/panic.go:1116 +0x72
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:726 +0x4ac

goroutine 43 [syscall]:
runtime.cgocall(0x38b3380, 0xc000a7a4d0, 0x420001)
	/usr/local/go/src/runtime/cgocall.go:133 +0x5b fp=0xc000a7a488 sp=0xc000a7a450 pc=0x41ea5b
github.com/cockroachdb/cockroach/pkg/geo/geos._Cfunc_CR_GEOS_Union(0x7f5bb8e6e000, 0xc000b64000, 0x26ad, 0xc000a45a20, 0x10e, 0xc0008b0c50, 0x0, 0x0)
	_cgo_gotypes.go:902 +0x4d fp=0xc000a7a4d0 sp=0xc000a7a488 pc=0x176c8ad
github.com/cockroachdb/cockroach/pkg/geo/geos.Union.func1(0x7f5bb8e6e000, 0xc000b64000, 0x26ad, 0x26ad, 0xc000a45a20, 0x10e, 0x148, 0xc0008b0c50, 0xc000a7a701, 0xc000a7a5b8)
	/go/src/github.com/cockroachdb/cockroach/pkg/geo/geos/geos.go:533 +0xfd fp=0xc000a7a540 sp=0xc000a7a4d0 pc=0x17766bd
github.com/cockroachdb/cockroach/pkg/geo/geos.Union(0xc000b64000, 0x26ad, 0x26ad, 0xc000a45a20, 0x10e, 0x148, 0x428c10, 0x5187a00, 0x2, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/geo/geos/geos.go:533 +0xd3 fp=0xc000a7a5c8 sp=0xc000a7a540 pc=0x1770313
github.com/cockroachdb/cockroach/pkg/sql/sem/builtins.(*stUnionAgg).Add(0xc000e7e240, 0x503d440, 0xc00019e010, 0x51784c0, 0xc00042eff0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/sem/builtins/aggregate_builtins.go:819 +0x450 fp=0xc000a7a7e8 sp=0xc000a7a5c8 pc=0x1a81450
github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*aggregatorBase).accumulateRowIntoBucket(0xc0001b5900, 0xc00042b9e0, 0x6, 0x6, 0x0, 0x0, 0x0, 0xc0007e77c0, 0x5, 0x5, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/rowexec/aggregator.go:796 +0x402 fp=0xc000a7a950 sp=0xc000a7a7e8 pc=0x29af682
github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*orderedAggregator).accumulateRow(0xc0001b5900, 0xc00042b9e0, 0x6, 0x6, 0x1, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/rowexec/aggregator.go:877 +0xbe fp=0xc000a7a9c0 sp=0xc000a7a950 pc=0x29b019e
github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*orderedAggregator).accumulateRows(0xc0001b5900, 0x0, 0x0, 0xc00019e001, 0xc000a7aab0, 0x450de9)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/rowexec/aggregator.go:510 +0x21b fp=0xc000a7aa58 sp=0xc000a7a9c0 pc=0x29add1b
github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*orderedAggregator).Next(0xc0001b5900, 0x4a37f08, 0xc000544600, 0xc000e69500, 0xc00062d8c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/rowexec/aggregator.go:714 +0x154 fp=0xc000a7aac0 sp=0xc000a7aa58 pc=0x29af194
github.com/cockroachdb/cockroach/pkg/sql/distsql.verifyColOperator(0xc000103080, 0x0, 0x0, 0x0, 0x0, 0xc000a5e7c0, 0x1, 0x1, 0xc000a5e7e0, 0x1, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql/columnar_utils_test.go:202 +0xfbf fp=0xc000a7b4e0 sp=0xc000a7aac0 pc=0x2ac82ff
github.com/cockroachdb/cockroach/pkg/sql/distsql.TestAggregatorAgainstProcessor(0xc000103080)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql/columnar_operators_test.go:308 +0x1425 fp=0xc000a7bf80 sp=0xc000a7b4e0 pc=0x2abe8e5
testing.tRunner(0xc000103080, 0x4a37ff8)
	/usr/local/go/src/testing/testing.go:1123 +0xef fp=0xc000a7bfd0 sp=0xc000a7bf80 pc=0x5471ef
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1374 +0x1 fp=0xc000a7bfd8 sp=0xc000a7bfd0 pc=0x48b741
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1168 +0x2b3

My guess is that #61966 is the culprit.

@yuzefovich yuzefovich assigned otan and andyyang890 and unassigned yuzefovich Mar 17, 2021
@andyyang890 andyyang890 mentioned this issue Mar 18, 2021
Merged
@andyyang890
Copy link
Collaborator

andyyang890 commented Mar 18, 2021 •
edited
Loading

The root cause of this is #62184. The temp fix in #62183 should prevent this test from failing for now.

@craig craig bot closed this as completed in d7a553b Mar 18, 2021
@otan otan mentioned this issue Mar 23, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@otan otan

@andyyang890 andyyang890

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

geogen: disallow EMPTY in multi and geometry collections andyyang890/cockroach
4 participants
@otan @yuzefovich @cockroach-teamcity @andyyang890