Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachtest: sqlsmith/setup=seed/setting=no-ddl failed #62846

Closed
cockroach-teamcity opened this issue Mar 31, 2021 · 6 comments
Closed

roachtest: sqlsmith/setup=seed/setting=no-ddl failed #62846

cockroach-teamcity opened this issue Mar 31, 2021 · 6 comments
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot.

Comments

@cockroach-teamcity
Copy link
Member

(roachtest).sqlsmith/setup=seed/setting=no-ddl failed on master@d145e9fc02064a8b6b4179b5af7da5238b192f74:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/sqlsmith/setup=seed/setting=no-ddl/run_1
	cluster.go:1667,context.go:140,cluster.go:1656,test_runner.go:849: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-2831942-1617170857-07-n4cpu4 --oneshot --ignore-empty-nodes: exit status 1 3: 8180
		4: 7205
		1: 8815
		2: dead
		Error: UNCLASSIFIED_PROBLEM: 2: dead
		(1) UNCLASSIFIED_PROBLEM
		Wraps: (2) attached stack trace
		  -- stack trace:
		  | main.glob..func14
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1147
		  | main.wrap.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:271
		  | github.com/spf13/cobra.(*Command).execute
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:830
		  | github.com/spf13/cobra.(*Command).ExecuteC
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:914
		  | github.com/spf13/cobra.(*Command).Execute
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:864
		  | main.main
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1852
		  | runtime.main
		  | 	/usr/local/go/src/runtime/proc.go:204
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (3) 2: dead
		Error types: (1) errors.Unclassified (2) *withstack.withStack (3) *errutil.leafError

More

Artifacts: /sqlsmith/setup=seed/setting=no-ddl

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Mar 31, 2021
@yuzefovich yuzefovich removed the release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. label Mar 31, 2021
@yuzefovich
Copy link
Member

panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x1d34674]

goroutine 20847 [running]:
panic(0x45307c0, 0x7c641b0)
	/usr/local/go/src/runtime/panic.go:1064 +0x545 fp=0xc00338d2a0 sp=0xc00338d1d8 pc=0x48b2c5
github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1(0xc00338dd50)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:58 +0x68b fp=0xc00338d440 sp=0xc00338d2a0 pc=0x295998b
runtime.call32(0x0, 0x52e2020, 0xc002faa1d8, 0x800000008)
	/usr/local/go/src/runtime/asm_amd64.s:540 +0x3e fp=0xc00338d470 sp=0xc00338d440 pc=0x4c2d9e
runtime.reflectcallSave(0xc00338d5b0, 0x52e2020, 0xc002faa1d8, 0x8)
	/usr/local/go/src/runtime/panic.go:881 +0x58 fp=0xc00338d4a0 sp=0xc00338d470 pc=0x48acb8
runtime.runOpenDeferFrame(0xc00141e180, 0xc002faa190, 0xc00338d5f8)
	/usr/local/go/src/runtime/panic.go:855 +0x2cd fp=0xc00338d530 sp=0xc00338d4a0 pc=0x48ab6d
panic(0x45307c0, 0x7c641b0)
	/usr/local/go/src/runtime/panic.go:969 +0x1b9 fp=0xc00338d5f8 sp=0xc00338d530 pc=0x48af39
runtime.panicmem(...)
	/usr/local/go/src/runtime/panic.go:212
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:742 +0x413 fp=0xc00338d628 sp=0xc00338d5f8 pc=0x4a2013
github.com/cockroachdb/cockroach/pkg/sql/sem/builtins.glob..func217(0xc0039b3b00, 0x86ecbf0, 0x0, 0x0, 0x0, 0xc00338d6f0, 0x295ebf2, 0xc002a74880)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/sem/builtins/builtins.go:3834 +0x34 fp=0xc00338d680 sp=0xc00338d628 pc=0x1d34674
github.com/cockroachdb/cockroach/pkg/sql/colexec.(*defaultBuiltinFuncOperator).Next.func1()
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/builtin_funcs.go:83 +0x3ac fp=0xc00338d740 sp=0xc00338d680 pc=0x2d1a0cc
github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).PerformOperation(0xc00382d3e0, 0xc00338d7e0, 0x1, 0x1, 0xc00338d7f0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:299 +0x59 fp=0xc00338d778 sp=0xc00338d740 pc=0x295fed9
github.com/cockroachdb/cockroach/pkg/sql/colexec.(*defaultBuiltinFuncOperator).Next(0xc002357680, 0x5a52620, 0xc00368a658, 0xc00168b5e0, 0x7f68229d7eb8)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/builtin_funcs.go:61 +0x205 fp=0xc00338d850 sp=0xc00338d778 pc=0x2ca7cc5
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*vectorTypeEnforcer).Next(0xc0028ebb00, 0x5a52620, 0xc00368a658, 0x203000, 0x7f6822986e10)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/operator.go:145 +0x4b fp=0xc00338d898 sp=0xc00338d850 pc=0x296564b
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase.constDatumOp.Next(0x5a4d760, 0xc0028ebb00, 0xc00382d650, 0x9, 0x4b7e600, 0xc0028ee980, 0x5a52620, 0xc00368a658, 0x2920515, 0xc001e3d9e8)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase/const.eg.go:595 +0x63 fp=0xc00338d978 sp=0xc00338d898 pc=0x2a95803
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase.(*constDatumOp).Next(0xc00382d680, 0x5a52620, 0xc00368a658, 0xc002f40d40, 0x1)
	<autogenerated>:1 +0xa5 fp=0xc00338da08 sp=0xc00338d978 pc=0x2ab23c5
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase.(*simpleProjectOp).Next(0xc003828190, 0x5a52620, 0xc00368a658, 0x291c7169fb, 0x8693700)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecbase/simple_project.go:124 +0x62 fp=0xc00338dab0 sp=0xc00338da08 pc=0x2a9dd22
github.com/cockroachdb/cockroach/pkg/sql/colflow.panicInjector.Next(0x5a37a60, 0xc003828190, 0xc00382db60, 0x5a52620, 0xc00368a658, 0xc0028ebb80, 0x52e8e98)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/panic_injector.go:63 +0x98 fp=0xc00338daf8 sp=0xc00338dab0 pc=0x32fc0b8
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*panicInjector).Next(0xc000cfba60, 0x5a52620, 0xc00368a658, 0x2ab22de, 0xc0028ebb00)
	<autogenerated>:1 +0x65 fp=0xc00338db40 sp=0xc00338daf8 pc=0x330bf05
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*batchInfoCollector).Next(0xc002f4a9c0, 0x5a52620, 0xc00368a658, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:86 +0x5d fp=0xc00338db90 sp=0xc00338db40 pc=0x32fee5d
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*deselectorOp).Next(0xc0038283c0, 0x5a52620, 0xc00368a658, 0xc001e3dcb0, 0xec23c9)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/deselector.go:68 +0xe7 fp=0xc00338dc60 sp=0xc00338db90 pc=0x2964f47
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc.(*Outbox).sendBatches.func1()
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc/outbox.go:257 +0xc8 fp=0xc00338dd18 sp=0xc00338dc60 pc=0x2d34e88
github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError(0xc001e3dd60, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:91 +0x57 fp=0xc00338dd48 sp=0xc00338dd18 pc=0x2958e37
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc.(*Outbox).sendBatches(0xc002b08be0, 0x5a32c20, 0xc00393e060, 0x7f6822952a60, 0xc00256ffd0, 0xc00256fec0, 0xc002b08b00, 0xc000e15c80, 0x45f465)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc/outbox.go:249 +0xba fp=0xc00338ddb0 sp=0xc00338dd48 pc=0x2d33c7a
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc.(*Outbox).runWithStream(0xc002b08be0, 0x5a32c20, 0xc00393e060, 0x7f6822952a60, 0xc00256ffd0, 0xc00256fec0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc/outbox.go:348 +0xee fp=0xc00338de28 sp=0xc00338ddb0 pc=0x2d3454e
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc.(*Outbox).Run(0xc002b08be0, 0x5a32b60, 0xc002f403c0, 0x5990980, 0xc000c85590, 0x8e82284f00000001, 0x7d58eda0f34559fa, 0x9a89a8fa, 0x2, 0xc00256fec0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc/outbox.go:194 +0x3c9 fp=0xc00338def8 sp=0xc00338de28 pc=0x2d33889
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlowCreator).setupRemoteOutputStream.func1(0x5a32b60, 0xc0028ea4c0, 0xc002e9ee30)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:657 +0x125 fp=0xc00338df90 sp=0xc00338def8 pc=0x3307f05
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlowCreatorHelper).accumulateAsyncComponent.func1.1(0xc00382dbf0, 0x5a32b60, 0xc0028ea4c0, 0xc002e9ee30, 0xc00280c9e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:1323 +0x44 fp=0xc00338dfb8 sp=0xc00338df90 pc=0x330abe4
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1374 +0x1 fp=0xc00338dfc0 sp=0xc00338dfb8 pc=0x4c47c1
created by github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlowCreatorHelper).accumulateAsyncComponent.func1
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:1322 +0x6f

The panic is coming from

"crdb_internal.create_join_token": makeBuiltin(
tree.FunctionProperties{Category: categorySystemInfo},
tree.Overload{
Types: tree.ArgTypes{},
ReturnType: tree.FixedReturnType(types.String),
Fn: func(ctx *tree.EvalContext, args tree.Datums) (tree.Datum, error) {
token, err := ctx.JoinTokenCreator.CreateJoinToken(ctx.Context)
if err != nil {
return nil, err
}
return tree.NewDString(token), nil
},
Info: "Creates a join token for use when adding a new node to a secure cluster.",
Volatility: tree.VolatilityVolatile,
},
),

Looks like ctx.JoinTokenCreator might be unset. cc @itsbilal

A side note to myself: I think we should be catching panics coming from sql/sem/builtins package.

craig bot pushed a commit that referenced this issue Mar 31, 2021
62498: utilccl,kvccl: improve performance when checking enterprise features r=tbg a=erikgrinaker

**utilccl: cache license decoding**

Previously, the `utilccl` package would decode the license from the the
base64-encoded Protobuf representation in settings every time it was
needed, which was sufficient for its uses. However, recently there's
been a need to check whether enterprise features are enabled in hot
paths (e.g. with follower reads as seen in #62447), making the decoding
cost too great.

This patch adds `cluster.Settings.Cache` as a shared cache, and uses it
to cache decoded licenses with a private key type.

**utilccl,kvccl: add IsEnterpriseEnabled for faster license checks**

`utilccl.CheckEnterpriseEnabled()` is used to check whether a valid
enterprise license exists for a given feature. If no valid license is
found, it returns an error with specific details.

However, `kvccl` used this function in follower read hot paths, and
instantiating an error when follower reads are unavailable could have
significant overhead -- see e.g. #62447.

This patch adds `IsEnterpriseEnabled()`, which has the same behavior as
`CheckEnterpriseEnabled()` but returns a boolean instead. This is
significantly faster since we can avoid instantiating a custom error
each time. `kvccl` is also updated to use this in hot paths.

Resolves #62489.

Release note: None

62642: colserde: fix the edge case with nulls handling r=yuzefovich a=yuzefovich

When serializing the data of Bool, Bytes, Int, and Float types when they
don't have any nulls in the vector, we don't explicit specify the null
bitmap. Previously, when deserializing such vectors with no nulls we
would simply call `UnsetNulls` on the `coldata.Nulls` object that is
currently present. However, it is possible that already present nulls
object cannot support the desired batch length. This could lead to index
out of bounds accesses. Note that in the vast majority of cases this
likely doesn't happen in practice because we check `MaybeHasNulls`, and
that would return `false` making us omit the null checking code.

Fixes: #62636.

Release note (bug fix): Previously, CockroachDB could encounter an
internal error in rare circumstances when executing queries via the
vectorized engine that operate on columns of BOOL, BYTES, INT, and FLOAT
types that have a mix of NULL and non-NULL values.

62740: workload: add idle-conns flag for adding idle connections to tpcc r=rafiss a=RichardJCai

workload: add idle-conns flag for adding idle connections to tpcc

Release note: None

#62526

62814: tenantrate: add "test" that reports IOPS estimations r=RaduBerinde a=RaduBerinde

This change adds a "test" facility which takes the description of a
uniform workload (read percentage, read size, write size) and prints
out an estimation of the sustained IOPS and burst IO. This will allow
a better understanding of how changes to the settings or the mechanism
translate into IOPS changes.

Release note: None

62833: kvserver: deflake TestFollowerReadsWithStaleDescriptor r=aayushshah15 a=aayushshah15

A preceding change (#62696) introduced a flakey update to this test.
Prior to that change, this test was using 2 voting replicas but that
change tried to make it use 1 voter and 1 non-voter instead (as a litmus
test for the new syntax added in #62696).

The test rebalances a replica away from a node and ensures that a
historical read sent immediately afterwards gets re-routed to the
leaseholder replica, since the receiving store had its replica
destroyed. However, when we're using a non-voter in this test, that
non-voter may not have learned about this replication change by the time
it receives this historical query and that fails the assertion.

This commit re-organizes the test and fixes the flake.

Release note: None

62862: testutils: add skip.UnderBazelWithIssue r=rickystewart a=stevendanna

This is to skip individual tests under bazel. This seems a bit more
fine-grained than the broken_in_bazel tag in the bazel configuration
but also allows us to avoid skipping tests that work outside of bazel
in our main test suite.

Release note: None

62877: Added CACHE to SEQUENCE syntax diagrams r=ericharmeling a=ericharmeling

Follow-up of #56954.

Release justification: non-production code changes

Release note: None

62889: colexecerror: catch panics from packages in sql/sem folder r=yuzefovich a=yuzefovich

Previously, we would only catch panics from `sql/sem/tree` package.
Recently sqlsmith encountered a crash because of a panic in
`sql/sem/builtins` package, and I believe it is reasonable to catch
panics from that package as well as from `sql/sem/transform`, so we will
now be catching based on `sql/sem` prefix.

Addresses: #62846.

Release note: None

62898: build: install essential build tools in teamcity build agents r=jlinder a=rickystewart

In #62815, we migrated from an alternative way of installing golang, the
`longsleep/golang-backports` deb repo, to the currently recommended
install method found at https://golang.org/doc/install -- namely, we
download a tarball and then just unzip it in the right spot. This
works perfectly, *except* that the deb package had a dependency on build
tools like `gcc` and `make`, and certain build configurations had come
to depend on their global installation (namely, all those that don't use
`builder.sh` to run a build). This resulted in a couple of failures
being reported:

* https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_ExampleORMs/2834741
* https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_UnitTests_Acceptance/2834732

We just install [`build-essential`](https://packages.ubuntu.com/xenial/build-essential)
here, which is the easiest way to get all of that stuff.

Release note: None

Co-authored-by: Erik Grinaker <[email protected]>
Co-authored-by: Yahor Yuzefovich <[email protected]>
Co-authored-by: richardjcai <[email protected]>
Co-authored-by: Radu Berinde <[email protected]>
Co-authored-by: Aayush Shah <[email protected]>
Co-authored-by: Steven Danna <[email protected]>
Co-authored-by: Eric Harmeling <[email protected]>
Co-authored-by: Ricky Stewart <[email protected]>
@cockroach-teamcity
Copy link
Member Author

(roachtest).sqlsmith/setup=seed/setting=no-ddl failed on master@9891b8db5ea2e9339a04fd2731a16658103e969f:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/sqlsmith/setup=seed/setting=no-ddl/run_1
	sqlsmith.go:226,sqlsmith.go:262,test_runner.go:768: error: pq: internal error: unexpected error from the vectorized engine: runtime error: invalid memory address or nil pointer dereference
		stmt:
		SELECT
			crdb_internal.create_join_token()::STRING AS col_28966,
			'-81 years -1 mons -527 days -23:09:37.904059':::INTERVAL AS col_28967,
			NULL AS col_28968,
			(-0.21433685719966888):::FLOAT8 AS col_28969,
			('04:50:58.248504':::TIME::TIME + '1978-06-14':::DATE::DATE)::TIMESTAMP AS col_28970
		FROM
			defaultdb.public.seed@seed__int8__float8__date_idx AS tab_10381
		LIMIT
			40:::INT8;

More

Artifacts: /sqlsmith/setup=seed/setting=no-ddl

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

roachtest.sqlsmith/setup=seed/setting=no-ddl failed with artifacts on master @ 71a023f9212cf1285a4475ed85d5ab6eefa9bdf7:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/sqlsmith/setup=seed/setting=no-ddl/run_1
	sqlsmith.go:226,sqlsmith.go:262,test_runner.go:793: error: pq: internal error: unexpected error from the vectorized engine: runtime error: index out of range [1] with length 1
		stmt:
		SELECT
			json_agg(tab_4303._jsonb)::JSONB AS col_9963
		FROM
			defaultdb.public.seed@seed__int8__float8__date_idx AS tab_4303
		WHERE
			tab_4303._bool
		GROUP BY
			tab_4303._jsonb, tab_4303._enum
		ORDER BY
			tab_4303._enum
		LIMIT
			95:::INT8;
Reproduce

To reproduce, try:

#!/usr/bin/env bash
#!/bin/bash
set -euxo pipefail

# NB: invoke this script with "caffeinate" on OSX and/or linux to
# prevent runs failing due to standby.

sha=$(git rev-parse HEAD)

if [ ! -f roachtest.$sha ]; then
	./build/builder.sh mkrelease amd64-linux-gnu bin/{roach{prod,test},workload}
	mv -f bin.docker_amd64/roachprod roachprod.$sha
	mv -f bin.docker_amd64/workload workload.$sha
	mv -f bin.docker_amd64/roachtest roachtest.$sha
fi

if [ ! -f cockroach.$sha ]; then
	./build/builder.sh mkrelease amd64-linux-gnu
	mv cockroach-linux-2.6.32-gnu-amd64 cockroach.$sha
fi

# NB: consider adding --debug if it is useful to let the clusters
# for failed tests survive.
./roachtest.$sha run "sqlsmith/setup=seed/setting=no-ddl" \
  --port 8080 --count 10 --cpu-quota 500 \
  --roachprod roachprod.${sha} --workload workload.${sha} \
  --cockroach ./cockroach.$sha \
  --artifacts artifacts.$sha | tee roachtest-stress.${sha}

Same failure on other branches

/cc @cockroachdb/sql-queries
Internal log

no author provided

See this test on roachdash

Improve this report!

@yuzefovich
Copy link
Member

dup of #63819

@rytaft
Copy link
Collaborator

rytaft commented Apr 19, 2021

@yuzefovich this latest failure looks unrelated to #63819, since the error is "error: index out of range [1] with length 1" and crdb_internal.create_join_token() is not part of the query. Is there already another issue open for that error?

@yuzefovich
Copy link
Member

The latest one fixed by #63446:

E210410 19:31:23.850417 2649 sql/distsql/server.go:330 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 732  error setting up flow: unexpected error from the vectorized engine: ‹runtime error: index out of range [1] with length 1›
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733  encountered internal error:
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +unexpected error from the vectorized engine: ‹runtime error: index out of range [1] with length 1›
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +(1) assertion failure
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +Wraps: (2) attached stack trace
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  -- stack trace:
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:88
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | runtime.gopanic
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/usr/local/go/src/runtime/panic.go:969
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.InternalError
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:192
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colbuilder.NewColOperator.func1
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/colbuilder/execplan.go:674
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | runtime.gopanic
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/usr/local/go/src/runtime/panic.go:975
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | runtime.goPanicIndex
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/usr/local/go/src/runtime/panic.go:88
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexec.newSorter
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/sort.go:50
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexec.NewSorter
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/sort.go:37
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colbuilder.opResult.createDiskBackedSort
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/colbuilder/execplan.go:448
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colbuilder.opResult.makeDiskBackedSorterConstructor.func1
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/colbuilder/execplan.go:532
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexec.NewExternalHashAggregator
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/external_hash_aggregator.go:103
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colbuilder.NewColOperator.func2
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/colbuilder/execplan.go:925
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexec.NewOneInputDiskSpiller
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/disk_spiller.go:86
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexec/colbuilder.NewColOperator
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexec/colbuilder/execplan.go:913
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlowCreator).setupFlow.func1
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:1158
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:91
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlowCreator).setupFlow
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:1077
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlow).Setup
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:211
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/distsql.(*ServerImpl).setupFlow
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql/server.go:329
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/distsql.(*ServerImpl).SetupLocalSyncFlow
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql/server.go:497
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).setupFlows
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:220
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:325
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:979
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1048
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:915
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:659
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:122
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd.func1
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1527
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1529
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1450
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:484
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync.func1
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +  | 	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:626
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +Wraps: (3) unexpected error from the vectorized engine
E210410 19:31:23.850618 2649 sql/sqltelemetry/report.go:57 ⋮ [n1,client=‹35.229.85.164:60316›,hostnossl,user=root] 733 +Wraps: (4) ‹runtime error: index out of range [1] with length 1›

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot.
Projects
None yet
Development

No branches or pull requests

3 participants