roachtest: tpce/c=5000/nodes=3 failed #61708

cockroach-teamcity · 2021-03-09T13:58:31Z

(roachtest).tpce/c=5000/nodes=3 failed on release-21.1@8c3fa11ca35deca2f38a1e6fb5f0dea4a77285bd:

		  | cd798458a46f: Pull complete
		  | Digest: sha256:1e299df6b79d4630bdb394ed98211f9303292b35aff2555ba5b997f2f889fdea
		  | Status: Downloaded newer image for cockroachdb/tpc-e:latest
		  | Error: Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: None, code: SqlState("XXUUU"), message: "relation \"trade_type\" is offline: importing", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("descriptor.go"), line: Some(530), routine: Some("FilterDescriptorState") }) }
		  | Error: COMMAND_PROBLEM: exit status 1
		  | (1) COMMAND_PROBLEM
		  | Wraps: (2) Node 4. Command with error:
		  |   | ```
		  |   | sudo docker run cockroachdb/tpc-e:latest --customers=5000 --racks=3 --init --hosts=10.128.0.23
		  |   | ```
		  | Wraps: (3) exit status 1
		  | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
		  |
		  | stdout:
		  | Initializing schema...
		  | Importing dataset...
		Wraps: (4) exit status 20
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError

	cluster.go:2688,tpce.go:96,tpce.go:113,test_runner.go:767: monitor failure: monitor task failed: t.Fatal() was called
		(1) attached stack trace
		  -- stack trace:
		  | main.(*monitor).WaitE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2676
		  | main.(*monitor).Wait
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2684
		  | main.registerTPCE.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:96
		  | main.registerTPCE.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:113
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:767
		Wraps: (2) monitor failure
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | main.(*monitor).wait.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2732
		Wraps: (4) monitor task failed
		Wraps: (5) attached stack trace
		  -- stack trace:
		  | main.init
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2646
		  | runtime.doInit
		  | 	/usr/local/go/src/runtime/proc.go:5652
		  | runtime.main
		  | 	/usr/local/go/src/runtime/proc.go:191
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (6) t.Fatal() was called
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError

More

Artifacts: /tpce/c=5000/nodes=3
Related:

roachtest: tpce/c=5000/nodes=3 failed #61065 roachtest: tpce/c=5000/nodes=3 failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

The text was updated successfully, but these errors were encountered:

nvanbenschoten · 2021-03-09T17:56:59Z

Same as #61065.

pbardea · 2021-03-11T18:22:30Z

Some notes from the latest failure for posterity:

639708586412441601	IMPORT	IMPORT INTO tpce.public.trade_type CSV DATA ('gs://cockroach-fixtures/tpce-csv/fixed/TradeType.txt') WITH delimiter = '|'		root	{}	succeeded	NULL	2021-03-09 12:46:07.638762+00:00:00	2021-03-09 12:46:07.759725+00:00:00	2021-03-09 12:46:17.574831+00:00:00	2021-03-09 12:46:17.555058+00:00:00	1	NULL		NULL

It looks like the job eventually succeeded.

Descriptor is PUBLIC in the debug.zip:

{"table": {"auditMode": "DISABLED", "checks": [], "columns": [{"alterColumnTypeInProgress": false, "computeExpr": null, "defaultExpr": null, "hidden": false, "id": 1, "name": "sc_id", "nullable": false, "ownsSequenceIds": [], "pgAttributeNum": 0, "systemColumnKind": "NONE", "type": {"arrayContents": null, "arrayDimensions": [], "arrayElemType": null, "family": "StringFamily", "geoMetadata": null, "intervalDurationField": null, "locale": null, "oid": 1043, "precision": 0, "timePrecisionIsSet": false, "tupleContents": [], "tupleLabels": [], "udtMetadata": null, "visibleType": 7, "width": 2}, "usesSequenceIds": [], "virtual": false}, {"alterColumnTypeInProgress": false, "computeExpr": null, "defaultExpr": null, "hidden": false, "id": 2, "name": "sc_name", "nullable": false, "ownsSequenceIds": [], "pgAttributeNum": 0, "systemColumnKind": "NONE", "type": {"arrayContents": null, "arrayDimensions": [], "arrayElemType": null, "family": "StringFamily", "geoMetadata": null, "intervalDurationField": null, "locale": null, "oid": 1043, "precision": 0, "timePrecisionIsSet": false, "tupleContents": [], "tupleLabels": [], "udtMetadata": null, "visibleType": 7, "width": 30}, "usesSequenceIds": [], "virtual": false}], "createAsOfTime": {"logical": 2, "synthetic": false, "wallTime": "1615120283763984044"}, "createQuery": "", "dependedOnBy": [], "dependsOn": [], "drainingNames": [], "dropJobId": "0", "dropTime": "0", "families": [{"columnIds": [1, 2], "columnNames": ["sc_id", "sc_name"], "defaultColumnId": 2, "id": 0, "name": "primary"}], "formatVersion": 3, "gcMutations": [], "id": 80, "inboundFks": [{"match": "SIMPLE", "name": "fk_in_sc_id_ref_sector", "onDelete": "NO_ACTION", "onUpdate": "NO_ACTION", "originColumnIds": [3], "originTableId": 76, "referencedColumnIds": [1], "referencedTableId": 80, "validity": "Validating"}], "indexes": [{"columnDirections": ["ASC"], "columnIds": [2], "columnNames": ["sc_name"], "compositeColumnIds": [], "createdExplicitly": true, "disabled": false, "encodingType": 0, "extraColumnIds": [1], "foreignKey": {"index": 0, "match": "SIMPLE", "name": "", "onDelete": "NO_ACTION", "onUpdate": "NO_ACTION", "sharedPrefixLen": 0, "table": 0, "validity": "Validated"}, "geoConfig": {"s2Geography": null, "s2Geometry": null}, "id": 2, "interleave": {"ancestors": []}, "interleavedBy": [], "name": "sector_sc_name_key", "partitioning": {"list": [], "numColumns": 0, "numImplicitColumns": 0, "range": []}, "predicate": "", "referencedBy": [], "sharded": {"columnNames": [], "isSharded": false, "name": "", "shardBuckets": 0}, "storeColumnIds": [], "storeColumnNames": [], "type": "FORWARD", "unique": true, "version": 2}, {"columnDirections": ["ASC"], "columnIds": [1], "columnNames": ["sc_id"], "compositeColumnIds": [], "createdExplicitly": true, "disabled": false, "encodingType": 0, "extraColumnIds": [], "foreignKey": {"index": 0, "match": "SIMPLE", "name": "", "onDelete": "NO_ACTION", "onUpdate": "NO_ACTION", "sharedPrefixLen": 0, "table": 0, "validity": "Validated"}, "geoConfig": {"s2Geography": null, "s2Geometry": null}, "id": 3, "interleave": {"ancestors": []}, "interleavedBy": [], "name": "primary_replicated_0", "partitioning": {"list": [], "numColumns": 0, "numImplicitColumns": 0, "range": []}, "predicate": "", "referencedBy": [], "sharded": {"columnNames": [], "isSharded": false, "name": "", "shardBuckets": 0}, "storeColumnIds": [2], "storeColumnNames": ["sc_name"], "type": "FORWARD", "unique": true, "version": 2}, {"columnDirections": ["ASC"], "columnIds": [1], "columnNames": ["sc_id"], "compositeColumnIds": [], "createdExplicitly": true, "disabled": false, "encodingType": 0, "extraColumnIds": [], "foreignKey": {"index": 0, "match": "SIMPLE", "name": "", "onDelete": "NO_ACTION", "onUpdate": "NO_ACTION", "sharedPrefixLen": 0, "table": 0, "validity": "Validated"}, "geoConfig": {"s2Geography": null, "s2Geometry": null}, "id": 4, "interleave": {"ancestors": []}, "interleavedBy": [], "name": "primary_replicated_1", "partitioning": {"list": [], "numColumns": 0, "numImplicitColumns": 0, "range": []}, "predicate": "", "referencedBy": [], "sharded": {"columnNames": [], "isSharded": false, "name": "", "shardBuckets": 0}, "storeColumnIds": [2], "storeColumnNames": ["sc_name"], "type": "FORWARD", "unique": true, "version": 2}, {"columnDirections": ["ASC"], "columnIds": [1], "columnNames": ["sc_id"], "compositeColumnIds": [], "createdExplicitly": true, "disabled": false, "encodingType": 0, "extraColumnIds": [], "foreignKey": {"index": 0, "match": "SIMPLE", "name": "", "onDelete": "NO_ACTION", "onUpdate": "NO_ACTION", "sharedPrefixLen": 0, "table": 0, "validity": "Validated"}, "geoConfig": {"s2Geography": null, "s2Geometry": null}, "id": 5, "interleave": {"ancestors": []}, "interleavedBy": [], "name": "primary_replicated_2", "partitioning": {"list": [], "numColumns": 0, "numImplicitColumns": 0, "range": []}, "predicate": "", "referencedBy": [], "sharded": {"columnNames": [], "isSharded": false, "name": "", "shardBuckets": 0}, "storeColumnIds": [2], "storeColumnNames": ["sc_name"], "type": "FORWARD", "unique": true, "version": 2}, {"columnDirections": ["ASC", "ASC"], "columnIds": [2, 1], "columnNames": ["sc_name", "sc_id"], "compositeColumnIds": [], "createdExplicitly": true, "disabled": false, "encodingType": 0, "extraColumnIds": [], "foreignKey": {"index": 0, "match": "SIMPLE", "name": "", "onDelete": "NO_ACTION", "onUpdate": "NO_ACTION", "sharedPrefixLen": 0, "table": 0, "validity": "Validated"}, "geoConfig": {"s2Geography": null, "s2Geometry": null}, "id": 6, "interleave": {"ancestors": []}, "interleavedBy": [], "name": "sector_sc_name_key_replicated_0", "partitioning": {"list": [], "numColumns": 0, "numImplicitColumns": 0, "range": []}, "predicate": "", "referencedBy": [], "sharded": {"columnNames": [], "isSharded": false, "name": "", "shardBuckets": 0}, "storeColumnIds": [], "storeColumnNames": [], "type": "FORWARD", "unique": true, "version": 2}, {"columnDirections": ["ASC", "ASC"], "columnIds": [2, 1], "columnNames": ["sc_name", "sc_id"], "compositeColumnIds": [], "createdExplicitly": true, "disabled": false, "encodingType": 0, "extraColumnIds": [], "foreignKey": {"index": 0, "match": "SIMPLE", "name": "", "onDelete": "NO_ACTION", "onUpdate": "NO_ACTION", "sharedPrefixLen": 0, "table": 0, "validity": "Validated"}, "geoConfig": {"s2Geography": null, "s2Geometry": null}, "id": 7, "interleave": {"ancestors": []}, "interleavedBy": [], "name": "sector_sc_name_key_replicated_1", "partitioning": {"list": [], "numColumns": 0, "numImplicitColumns": 0, "range": []}, "predicate": "", "referencedBy": [], "sharded": {"columnNames": [], "isSharded": false, "name": "", "shardBuckets": 0}, "storeColumnIds": [], "storeColumnNames": [], "type": "FORWARD", "unique": true, "version": 2}, {"columnDirections": ["ASC", "ASC"], "columnIds": [2, 1], "columnNames": ["sc_name", "sc_id"], "compositeColumnIds": [], "createdExplicitly": true, "disabled": false, "encodingType": 0, "extraColumnIds": [], "foreignKey": {"index": 0, "match": "SIMPLE", "name": "", "onDelete": "NO_ACTION", "onUpdate": "NO_ACTION", "sharedPrefixLen": 0, "table": 0, "validity": "Validated"}, "geoConfig": {"s2Geography": null, "s2Geometry": null}, "id": 8, "interleave": {"ancestors": []}, "interleavedBy": [], "name": "sector_sc_name_key_replicated_2", "partitioning": {"list": [], "numColumns": 0, "numImplicitColumns": 0, "range": []}, "predicate": "", "referencedBy": [], "sharded": {"columnNames": [], "isSharded": false, "name": "", "shardBuckets": 0}, "storeColumnIds": [], "storeColumnNames": [], "type": "FORWARD", "unique": true, "version": 2}], "isMaterializedView": false, "lease": null, "localityConfig": null, "modificationTime": {"logical": 0, "synthetic": false, "wallTime": "0"}, "mutationJobs": [], "mutations": [], "name": "sector", "nextColumnId": 3, "nextFamilyId": 1, "nextIndexId": 9, "nextMutationId": 8, "offlineReason": "", "outboundFks": [], "parentId": 52, "partitionAllBy": false, "primaryIndex": {"columnDirections": ["ASC"], "columnIds": [1], "columnNames": ["sc_id"], "compositeColumnIds": [], "createdExplicitly": false, "disabled": false, "encodingType": 0, "extraColumnIds": [], "foreignKey": {"index": 0, "match": "SIMPLE", "name": "", "onDelete": "NO_ACTION", "onUpdate": "NO_ACTION", "sharedPrefixLen": 0, "table": 0, "validity": "Validated"}, "geoConfig": {"s2Geography": null, "s2Geometry": null}, "id": 1, "interleave": {"ancestors": []}, "interleavedBy": [], "name": "primary", "partitioning": {"list": [], "numColumns": 0, "numImplicitColumns": 0, "range": []}, "predicate": "", "referencedBy": [], "sharded": {"columnNames": [], "isSharded": false, "name": "", "shardBuckets": 0}, "storeColumnIds": [], "storeColumnNames": [], "type": "FORWARD", "unique": true, "version": 2}, "privileges": {"ownerProto": "root", "users": [{"privileges": 2, "userProto": "admin"}, {"privileges": 2, "userProto": "root"}], "version": 1}, "replacementOf": {"id": 0, "time": {"logical": 0, "synthetic": false, "wallTime": "0"}}, "sequenceOpts": null, "state": "PUBLIC", "temporary": false, "unexposedParentSchemaId": 29, "uniqueWithoutIndexConstraints": [], "version": 26, "viewQuery": ""}}

cockroach-teamcity · 2021-03-12T23:38:45Z

(roachtest).tpce/c=5000/nodes=3 failed on release-21.1@e5aebce55e34e66e8ce0cdbd81b9d9653243df81:

		  | cd798458a46f: Pull complete
		  | Digest: sha256:1e299df6b79d4630bdb394ed98211f9303292b35aff2555ba5b997f2f889fdea
		  | Status: Downloaded newer image for cockroachdb/tpc-e:latest
		  | Error: Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: None, code: SqlState("XXUUU"), message: "another operation is currently operating on the table", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("import_stmt.go"), line: Some(1233), routine: Some("prepareExistingTableDescForIngestion") }) }
		  | Error: COMMAND_PROBLEM: exit status 1
		  | (1) COMMAND_PROBLEM
		  | Wraps: (2) Node 4. Command with error:
		  |   | ```
		  |   | sudo docker run cockroachdb/tpc-e:latest --customers=5000 --racks=3 --init --hosts=10.128.0.205
		  |   | ```
		  | Wraps: (3) exit status 1
		  | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
		  |
		  | stdout:
		  | Initializing schema...
		  | Importing dataset...
		Wraps: (4) exit status 20
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError

	cluster.go:2688,tpce.go:96,tpce.go:113,test_runner.go:767: monitor failure: monitor task failed: t.Fatal() was called
		(1) attached stack trace
		  -- stack trace:
		  | main.(*monitor).WaitE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2676
		  | main.(*monitor).Wait
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2684
		  | main.registerTPCE.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:96
		  | main.registerTPCE.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:113
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:767
		Wraps: (2) monitor failure
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | main.(*monitor).wait.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2732
		Wraps: (4) monitor task failed
		Wraps: (5) attached stack trace
		  -- stack trace:
		  | main.init
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2646
		  | runtime.doInit
		  | 	/usr/local/go/src/runtime/proc.go:5652
		  | runtime.main
		  | 	/usr/local/go/src/runtime/proc.go:191
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (6) t.Fatal() was called
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError

More

Artifacts: /tpce/c=5000/nodes=3
Related:

roachtest: tpce/c=5000/nodes=3 failed #61065 roachtest: tpce/c=5000/nodes=3 failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

cockroach-teamcity · 2021-03-16T12:37:52Z

(roachtest).tpce/c=5000/nodes=3 failed on release-21.1@4a47e0e305cbdabf963f896c1cb571a28b34e63d:

		  | cd798458a46f: Pull complete
		  | Digest: sha256:1e299df6b79d4630bdb394ed98211f9303292b35aff2555ba5b997f2f889fdea
		  | Status: Downloaded newer image for cockroachdb/tpc-e:latest
		  | Error: Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: None, code: SqlState("XXUUU"), message: "relation \"trade_type\" is offline: importing", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("descriptor.go"), line: Some(579), routine: Some("FilterDescriptorState") }) }
		  | Error: COMMAND_PROBLEM: exit status 1
		  | (1) COMMAND_PROBLEM
		  | Wraps: (2) Node 4. Command with error:
		  |   | ```
		  |   | sudo docker run cockroachdb/tpc-e:latest --customers=5000 --racks=3 --init --hosts=10.128.1.13
		  |   | ```
		  | Wraps: (3) exit status 1
		  | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
		  |
		  | stdout:
		  | Initializing schema...
		  | Importing dataset...
		Wraps: (4) exit status 20
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError

	cluster.go:2688,tpce.go:96,tpce.go:113,test_runner.go:767: monitor failure: monitor task failed: t.Fatal() was called
		(1) attached stack trace
		  -- stack trace:
		  | main.(*monitor).WaitE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2676
		  | main.(*monitor).Wait
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2684
		  | main.registerTPCE.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:96
		  | main.registerTPCE.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:113
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:767
		Wraps: (2) monitor failure
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | main.(*monitor).wait.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2732
		Wraps: (4) monitor task failed
		Wraps: (5) attached stack trace
		  -- stack trace:
		  | main.init
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2646
		  | runtime.doInit
		  | 	/usr/local/go/src/runtime/proc.go:5652
		  | runtime.main
		  | 	/usr/local/go/src/runtime/proc.go:191
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (6) t.Fatal() was called
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError

More

Artifacts: /tpce/c=5000/nodes=3
Related:

roachtest: tpce/c=5000/nodes=3 failed #61065 roachtest: tpce/c=5000/nodes=3 failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

cockroach-teamcity · 2021-03-17T12:59:19Z

(roachtest).tpce/c=5000/nodes=3 failed on release-21.1@ba54410af79c0481e1e8f8fc45a9b132ee09a2b4:

		  | cd798458a46f: Pull complete
		  | Digest: sha256:1e299df6b79d4630bdb394ed98211f9303292b35aff2555ba5b997f2f889fdea
		  | Status: Downloaded newer image for cockroachdb/tpc-e:latest
		  | Error: Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: None, code: SqlState("XXUUU"), message: "relation \"charge\" is offline: importing", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("descriptor.go"), line: Some(579), routine: Some("FilterDescriptorState") }) }
		  | Error: COMMAND_PROBLEM: exit status 1
		  | (1) COMMAND_PROBLEM
		  | Wraps: (2) Node 4. Command with error:
		  |   | ```
		  |   | sudo docker run cockroachdb/tpc-e:latest --customers=5000 --racks=3 --init --hosts=10.128.0.96
		  |   | ```
		  | Wraps: (3) exit status 1
		  | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
		  |
		  | stdout:
		  | Initializing schema...
		  | Importing dataset...
		Wraps: (4) exit status 20
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError

	cluster.go:2688,tpce.go:96,tpce.go:113,test_runner.go:767: monitor failure: monitor task failed: t.Fatal() was called
		(1) attached stack trace
		  -- stack trace:
		  | main.(*monitor).WaitE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2676
		  | main.(*monitor).Wait
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2684
		  | main.registerTPCE.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:96
		  | main.registerTPCE.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:113
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:767
		Wraps: (2) monitor failure
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | main.(*monitor).wait.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2732
		Wraps: (4) monitor task failed
		Wraps: (5) attached stack trace
		  -- stack trace:
		  | main.init
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2646
		  | runtime.doInit
		  | 	/usr/local/go/src/runtime/proc.go:5652
		  | runtime.main
		  | 	/usr/local/go/src/runtime/proc.go:191
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (6) t.Fatal() was called
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError

More

Artifacts: /tpce/c=5000/nodes=3
Related:

roachtest: tpce/c=5000/nodes=3 failed #61065 roachtest: tpce/c=5000/nodes=3 failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

cockroach-teamcity · 2021-03-18T12:37:17Z

(roachtest).tpce/c=5000/nodes=3 failed on release-21.1@0ee391c3b96ba8695839ea60c45069bc771229b6:

		  | cd798458a46f: Pull complete
		  | Digest: sha256:1e299df6b79d4630bdb394ed98211f9303292b35aff2555ba5b997f2f889fdea
		  | Status: Downloaded newer image for cockroachdb/tpc-e:latest
		  | Error: Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: None, code: SqlState("XXUUU"), message: "another operation is currently operating on the table", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("import_stmt.go"), line: Some(1233), routine: Some("prepareExistingTableDescForIngestion") }) }
		  | Error: COMMAND_PROBLEM: exit status 1
		  | (1) COMMAND_PROBLEM
		  | Wraps: (2) Node 4. Command with error:
		  |   | ```
		  |   | sudo docker run cockroachdb/tpc-e:latest --customers=5000 --racks=3 --init --hosts=10.128.0.192
		  |   | ```
		  | Wraps: (3) exit status 1
		  | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
		  |
		  | stdout:
		  | Initializing schema...
		  | Importing dataset...
		Wraps: (4) exit status 20
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError

	cluster.go:2688,tpce.go:96,tpce.go:113,test_runner.go:767: monitor failure: monitor task failed: t.Fatal() was called
		(1) attached stack trace
		  -- stack trace:
		  | main.(*monitor).WaitE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2676
		  | main.(*monitor).Wait
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2684
		  | main.registerTPCE.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:96
		  | main.registerTPCE.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:113
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:767
		Wraps: (2) monitor failure
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | main.(*monitor).wait.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2732
		Wraps: (4) monitor task failed
		Wraps: (5) attached stack trace
		  -- stack trace:
		  | main.init
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2646
		  | runtime.doInit
		  | 	/usr/local/go/src/runtime/proc.go:5652
		  | runtime.main
		  | 	/usr/local/go/src/runtime/proc.go:191
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (6) t.Fatal() was called
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError

More

Artifacts: /tpce/c=5000/nodes=3
Related:

roachtest: tpce/c=5000/nodes=3 failed #61065 roachtest: tpce/c=5000/nodes=3 failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

cockroach-teamcity · 2021-03-19T12:46:13Z

(roachtest).tpce/c=5000/nodes=3 failed on release-21.1@cba235b11752e2f1e1c6dfee7ac55c514f4ea930:

		  | cd798458a46f: Pull complete
		  | Digest: sha256:1e299df6b79d4630bdb394ed98211f9303292b35aff2555ba5b997f2f889fdea
		  | Status: Downloaded newer image for cockroachdb/tpc-e:latest
		  | Error: Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: None, code: SqlState("XXUUU"), message: "another operation is currently operating on the table", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("import_stmt.go"), line: Some(1233), routine: Some("prepareExistingTableDescForIngestion") }) }
		  | Error: COMMAND_PROBLEM: exit status 1
		  | (1) COMMAND_PROBLEM
		  | Wraps: (2) Node 4. Command with error:
		  |   | ```
		  |   | sudo docker run cockroachdb/tpc-e:latest --customers=5000 --racks=3 --init --hosts=10.128.0.80
		  |   | ```
		  | Wraps: (3) exit status 1
		  | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
		  |
		  | stdout:
		  | Initializing schema...
		  | Importing dataset...
		Wraps: (4) exit status 20
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError

	cluster.go:2688,tpce.go:96,tpce.go:113,test_runner.go:767: monitor failure: monitor task failed: t.Fatal() was called
		(1) attached stack trace
		  -- stack trace:
		  | main.(*monitor).WaitE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2676
		  | main.(*monitor).Wait
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2684
		  | main.registerTPCE.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:96
		  | main.registerTPCE.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:113
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:767
		Wraps: (2) monitor failure
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | main.(*monitor).wait.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2732
		Wraps: (4) monitor task failed
		Wraps: (5) attached stack trace
		  -- stack trace:
		  | main.init
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2646
		  | runtime.doInit
		  | 	/usr/local/go/src/runtime/proc.go:5652
		  | runtime.main
		  | 	/usr/local/go/src/runtime/proc.go:191
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (6) t.Fatal() was called
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError

More

Artifacts: /tpce/c=5000/nodes=3
Related:

roachtest: tpce/c=5000/nodes=3 failed #61065 roachtest: tpce/c=5000/nodes=3 failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

cockroach-teamcity · 2021-03-22T12:36:13Z

(roachtest).tpce/c=5000/nodes=3 failed on release-21.1@2bdb62260a178e5bb63cf15f704944c5384f4347:

		  | cd798458a46f: Pull complete
		  | Digest: sha256:1e299df6b79d4630bdb394ed98211f9303292b35aff2555ba5b997f2f889fdea
		  | Status: Downloaded newer image for cockroachdb/tpc-e:latest
		  | Error: Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: None, code: SqlState("XXUUU"), message: "another operation is currently operating on the table", detail: None, hint: None, position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("import_stmt.go"), line: Some(1233), routine: Some("prepareExistingTableDescForIngestion") }) }
		  | Error: COMMAND_PROBLEM: exit status 1
		  | (1) COMMAND_PROBLEM
		  | Wraps: (2) Node 4. Command with error:
		  |   | ```
		  |   | sudo docker run cockroachdb/tpc-e:latest --customers=5000 --racks=3 --init --hosts=10.128.0.64
		  |   | ```
		  | Wraps: (3) exit status 1
		  | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
		  |
		  | stdout:
		  | Initializing schema...
		  | Importing dataset...
		Wraps: (4) exit status 20
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError

	cluster.go:2688,tpce.go:96,tpce.go:113,test_runner.go:767: monitor failure: monitor task failed: t.Fatal() was called
		(1) attached stack trace
		  -- stack trace:
		  | main.(*monitor).WaitE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2676
		  | main.(*monitor).Wait
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2684
		  | main.registerTPCE.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:96
		  | main.registerTPCE.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tpce.go:113
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:767
		Wraps: (2) monitor failure
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | main.(*monitor).wait.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2732
		Wraps: (4) monitor task failed
		Wraps: (5) attached stack trace
		  -- stack trace:
		  | main.init
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2646
		  | runtime.doInit
		  | 	/usr/local/go/src/runtime/proc.go:5652
		  | runtime.main
		  | 	/usr/local/go/src/runtime/proc.go:191
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1374
		Wraps: (6) t.Fatal() was called
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError

More

Artifacts: /tpce/c=5000/nodes=3
Related:

roachtest: tpce/c=5000/nodes=3 failed #61065 roachtest: tpce/c=5000/nodes=3 failed C-test-failure O-roachtest O-robot branch-master

See this test on roachdash
_{powered by pkg/cmd/internal/issues}

pbardea · 2021-03-23T12:15:14Z

Should be fixed by #62396.

cockroach-teamcity added branch-release-21.1 C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Mar 9, 2021

nvanbenschoten assigned pbardea Mar 10, 2021

pbardea added the GA-blocker label Mar 10, 2021

pbardea removed the release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. label Mar 11, 2021

cockroach-teamcity mentioned this issue Mar 12, 2021

roachtest: tpce/c=5000/nodes=3 failed #61065

Closed

pbardea mentioned this issue Mar 22, 2021

release-21.1: importccl: commit planner transaction in planHook #62396

Merged

pbardea closed this as completed Mar 23, 2021

tbg mentioned this issue Sep 20, 2021

roachtest: tpccbench/nodes=9/cpu=4/multi-region failed #70410

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

roachtest: tpce/c=5000/nodes=3 failed #61708

roachtest: tpce/c=5000/nodes=3 failed #61708

cockroach-teamcity commented Mar 9, 2021

nvanbenschoten commented Mar 9, 2021

pbardea commented Mar 11, 2021

cockroach-teamcity commented Mar 12, 2021

cockroach-teamcity commented Mar 16, 2021

cockroach-teamcity commented Mar 17, 2021

cockroach-teamcity commented Mar 18, 2021

cockroach-teamcity commented Mar 19, 2021

cockroach-teamcity commented Mar 22, 2021

pbardea commented Mar 23, 2021

roachtest: tpce/c=5000/nodes=3 failed #61708

roachtest: tpce/c=5000/nodes=3 failed #61708

Comments

cockroach-teamcity commented Mar 9, 2021

nvanbenschoten commented Mar 9, 2021

pbardea commented Mar 11, 2021

cockroach-teamcity commented Mar 12, 2021

cockroach-teamcity commented Mar 16, 2021

cockroach-teamcity commented Mar 17, 2021

cockroach-teamcity commented Mar 18, 2021

cockroach-teamcity commented Mar 19, 2021

cockroach-teamcity commented Mar 22, 2021

pbardea commented Mar 23, 2021