Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: v23.1.4: couldn't decode type × in the external hash aggregator #106158

Closed
cockroach-sentry opened this issue Jul 5, 2023 · 1 comment
Closed
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report. T-sql-queries SQL Queries Team

Comments

@cockroach-sentry
Copy link
Collaborator

cockroach-sentry commented Jul 5, 2023

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/4295691194/?referrer=webhooks_plugin

Panic Message:

error.go:89: unexpected error from the vectorized engine: couldn't decode type ×
(1) assertion failure
Wraps: (2) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:89
  | runtime.gopanic
  | 	GOROOT/src/runtime/panic.go:884
  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.InternalError
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:195
  | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*hashAggregator).ExportBuffered
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/hash_aggregator.go:451
  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*bufferExportingOperator).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:344
  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*hashBasedPartitioner).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/hash_based_partitioner.go:425
  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:223
  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*BatchSchemaSubsetEnforcer).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/operator.go:224
  | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*bufferOp).advance
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/buffer.go:48
  | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*caseOp).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/case.go:195
  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*vectorTypeEnforcer).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/operator.go:157
  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecproj.(*defaultCmpProjOp).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecproj/bazel-out/k8-opt/bin/pkg/sql/colexec/colexecproj/default_cmp_proj_op.eg.go:37
  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase.(*simpleProjectOp).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase/simple_project.go:124
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*batchInfoCollector).next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:118
  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*batchInfoCollector).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:126
  | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Materializer).next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:247
  | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Materializer).nextAdapter
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:272
  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92
  | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Materializer).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:278
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*FlowCoordinator).next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:128
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*FlowCoordinator).nextAdapter
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:145
  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
  | 	github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*FlowCoordinator).Next
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:150
  | github.com/cockroachdb/cockroach/pkg/sql/execinfra.Run
  | 	github.com/cockroachdb/cockroach/pkg/sql/execinfra/base.go:192
  | github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBaseNoHelper).Run
  | 	github.com/cockroachdb/cockroach/pkg/sql/execinfra/processorsbase.go:726
  | github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowBase).Run
  | 	github.com/cockroachdb/cockroach/pkg/sql/flowinfra/flow.go:575
  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlow).Run
  | 	github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:310
  | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run
  | 	github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:902
  | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun
  | 	github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1913
  | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll.func3
  | 	github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1645
  | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll
  | 	github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1648
Wraps: (3) unexpected error from the vectorized engine
Wraps: (4) couldn't decode type ×
  | -- cause hidden behind barrier
  | couldn't decode type ×
  | (1) attached stack trace
  |   -- stack trace:
  |   | github.com/cockroachdb/cockroach/pkg/sql/rowenc/valueside.DecodeUntaggedDatum
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/rowenc/valueside/decode.go:242
  |   | github.com/cockroachdb/cockroach/pkg/sql/rowenc/valueside.Decode
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/rowenc/valueside/decode.go:43
  |   | github.com/cockroachdb/cockroach/pkg/col/coldataext.(*datumVec).UnmarshalTo
  |   | 	github.com/cockroachdb/cockroach/pkg/col/coldataext/datum_vec.go:162
  |   | github.com/cockroachdb/cockroach/pkg/col/colserde.(*ArrowBatchConverter).ArrowToBatch
  |   | 	github.com/cockroachdb/cockroach/pkg/col/colserde/arrowbatchconverter.go:477
  |   | github.com/cockroachdb/cockroach/pkg/col/colserde.(*FileDeserializer).GetBatch
  |   | 	github.com/cockroachdb/cockroach/pkg/col/colserde/file.go:260
  |   | github.com/cockroachdb/cockroach/pkg/sql/colcontainer.(*diskQueue).Dequeue
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/colcontainer/diskqueue.go:793
  |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*SpillingQueue).Dequeue
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/spilling_queue.go:368
  |   | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*hashAggregator).ExportBuffered
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/hash_aggregator.go:449
  |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*bufferExportingOperator).Next
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:344
  |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*hashBasedPartitioner).Next
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/hash_based_partitioner.go:425
  |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:223
  |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*BatchSchemaSubsetEnforcer).Next
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/operator.go:224
  |   | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*bufferOp).advance
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/buffer.go:48
  |   | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*caseOp).Next
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/case.go:195
  |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*vectorTypeEnforcer).Next
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/operator.go:157
  |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecproj.(*defaultCmpProjOp).Next
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecproj/bazel-out/k8-opt/bin/pkg/sql/colexec/colexecproj/default_cmp_proj_op.eg.go:37
  |   | github.com/cockroachdb/cockroach/pkg/s...
Stacktrace (expand for inline code snippets):

https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/distsql_running.go#L1647-L1649
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/distsql_running.go#L1644-L1646
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/distsql_running.go#L1912-L1914
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/distsql_running.go#L901-L903
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colflow/vectorized_flow.go#L309-L311
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/flowinfra/flow.go#L574-L576
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/execinfra/processorsbase.go#L725-L727
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/execinfra/base.go#L191-L193
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colflow/flow_coordinator.go#L149-L151
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexecerror/error.go#L91-L93
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colflow/flow_coordinator.go#L144-L146
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colflow/flow_coordinator.go#L127-L129
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/materializer.go#L277-L279
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexecerror/error.go#L91-L93
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/materializer.go#L271-L273
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/materializer.go#L246-L248
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colflow/stats.go#L125-L127
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexecerror/error.go#L91-L93
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colflow/stats.go#L117-L119
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/colexecbase/simple_project.go#L123-L125
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/colexecproj/bazel-out/k8-opt/bin/pkg/sql/colexec/colexecproj/default_cmp_proj_op.eg.go#L36-L38
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/colexecutils/operator.go#L156-L158
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/case.go#L194-L196
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/buffer.go#L47-L49
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/colexecutils/operator.go#L223-L225
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/colexecdisk/disk_spiller.go#L222-L224
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/colexecdisk/hash_based_partitioner.go#L424-L426
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/colexecdisk/disk_spiller.go#L343-L345
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexec/hash_aggregator.go#L450-L452
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexecerror/error.go#L194-L196
GOROOT/src/runtime/panic.go#L883-L885
https://github.com/cockroachdb/cockroach/blob/d8d427b1574281f7dc16da266867279c13cdbfd6/pkg/sql/colexecerror/error.go#L88-L90

pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).PlanAndRunAll at line 1648
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).PlanAndRunAll.func3 at line 1645
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).PlanAndRun at line 1913
pkg/sql/distsql_running.go in pkg/sql.(*DistSQLPlanner).Run at line 902
pkg/sql/colflow/vectorized_flow.go in pkg/sql/colflow.(*vectorizedFlow).Run at line 310
pkg/sql/flowinfra/flow.go in pkg/sql/flowinfra.(*FlowBase).Run at line 575
pkg/sql/execinfra/processorsbase.go in pkg/sql/execinfra.(*ProcessorBaseNoHelper).Run at line 726
pkg/sql/execinfra/base.go in pkg/sql/execinfra.Run at line 192
pkg/sql/colflow/flow_coordinator.go in pkg/sql/colflow.(*FlowCoordinator).Next at line 150
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 92
pkg/sql/colflow/flow_coordinator.go in pkg/sql/colflow.(*FlowCoordinator).nextAdapter at line 145
pkg/sql/colflow/flow_coordinator.go in pkg/sql/colflow.(*FlowCoordinator).next at line 128
pkg/sql/colexec/materializer.go in pkg/sql/colexec.(*Materializer).Next at line 278
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 92
pkg/sql/colexec/materializer.go in pkg/sql/colexec.(*Materializer).nextAdapter at line 272
pkg/sql/colexec/materializer.go in pkg/sql/colexec.(*Materializer).next at line 247
pkg/sql/colflow/stats.go in pkg/sql/colflow.(*batchInfoCollector).Next at line 126
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError at line 92
pkg/sql/colflow/stats.go in pkg/sql/colflow.(*batchInfoCollector).next at line 118
pkg/sql/colexec/colexecbase/simple_project.go in pkg/sql/colexec/colexecbase.(*simpleProjectOp).Next at line 124
pkg/sql/colexec/colexecproj/bazel-out/k8-opt/bin/pkg/sql/colexec/colexecproj/default_cmp_proj_op.eg.go in pkg/sql/colexec/colexecproj.(*defaultCmpProjOp).Next at line 37
pkg/sql/colexec/colexecutils/operator.go in pkg/sql/colexec/colexecutils.(*vectorTypeEnforcer).Next at line 157
pkg/sql/colexec/case.go in pkg/sql/colexec.(*caseOp).Next at line 195
pkg/sql/colexec/buffer.go in pkg/sql/colexec.(*bufferOp).advance at line 48
pkg/sql/colexec/colexecutils/operator.go in pkg/sql/colexec/colexecutils.(*BatchSchemaSubsetEnforcer).Next at line 224
pkg/sql/colexec/colexecdisk/disk_spiller.go in pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next at line 223
pkg/sql/colexec/colexecdisk/hash_based_partitioner.go in pkg/sql/colexec/colexecdisk.(*hashBasedPartitioner).Next at line 425
pkg/sql/colexec/colexecdisk/disk_spiller.go in pkg/sql/colexec/colexecdisk.(*bufferExportingOperator).Next at line 344
pkg/sql/colexec/hash_aggregator.go in pkg/sql/colexec.(*hashAggregator).ExportBuffered at line 451
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.InternalError at line 195
GOROOT/src/runtime/panic.go in runtime.gopanic at line 884
pkg/sql/colexecerror/error.go in pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1 at line 89

Tags

Tag Value
Command mt start-sql
Environment v23.1.4
Go Version go1.19.4
Platform linux amd64
Distribution CCL
Cockroach Release v23.1.3-121-gd8d427b1574
Cockroach SHA d8d427b1574281f7dc16da266867279c13cdbfd6
# of CPUs 32
# of Goroutines 385

Jira issue: CRDB-29431

@cockroach-sentry cockroach-sentry added O-sentry Originated from an in-the-wild panic report. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. v23.1.3-121-gd8d427b1574 labels Jul 5, 2023
@michae2 michae2 added the T-sql-queries SQL Queries Team label Jul 12, 2023
@yuzefovich yuzefovich changed the title Sentry: error.go:89: unexpected error from the vectorized engine: couldn't decode type × (1) assertion failure Wraps: (2) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/p... sql: v23.1.4: couldn't decode type × in the external hash aggregator Jul 12, 2023
@mgartner mgartner self-assigned this Jul 18, 2023
@mgartner mgartner moved this to Triage in SQL Queries Jul 24, 2023
@yuzefovich
Copy link
Member

closing as unactionable (#107732 will include at least the type into the error message)

@github-project-automation github-project-automation bot moved this from Triage to Done in SQL Queries Jul 27, 2023
craig bot pushed a commit that referenced this issue Jul 28, 2023
105552: multitenant: deflake TestConsumption r=yuzefovich a=DrewKimball

This patch adds retries to the `TestConsumption` test for retrieving the total tenant RU consumption in order to ensure that the delta includes the bucket flush that was triggered by the test query, instead of background activity.


Fixes #94286
Fixes #106572

Release note: None

107719: ui: add latency info to explain plan tab r=maryliag a=maryliag

This commit adds columns for latency information
(p50, p90, p99, pMax, pMin) on Explain Plan tab under Statement Details page.

Fixes #105371

<img width="1707" alt="Screenshot 2023-07-27 at 10 59 41 AM" src="https://github.com/cockroachdb/cockroach/assets/1017486/81a2ce3d-655e-47c1-931b-00e34433b3a1">


Release note (ui change): Add columns for p50, p90, p99 percentiles and latency min and max on Explain Plan tab on Statement Details page.

107732: valueside: use redact type name when can't decode r=yuzefovich a=yuzefovich

Informs: #106158
Epic: None

Release note: None

Co-authored-by: Drew Kimball <[email protected]>
Co-authored-by: maryliag <[email protected]>
Co-authored-by: Yahor Yuzefovich <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report. T-sql-queries SQL Queries Team
Projects
Archived in project
Development

No branches or pull requests

5 participants