roachtest: tpcc/mixed-headroom/n5cpu16 failed

[cockroach-teamcity]

roachtest.tpcc/mixed-headroom/n5cpu16 failed with artifacts on master @ 2c0129ce2507e708382cda830d1e47817a299c2e:

test artifacts and logs in: /artifacts/tpcc/mixed-headroom/n5cpu16/run_1
(test_impl.go:291).Fatal: monitor failure: monitor task failed: output in run_151318.812389790_n1_v2220beta4cockroach_workload_fixtures_import_bank: v22.2.0-beta.4/cockroach workload fixtures import bank --payload-bytes=10240 --rows=32552083 --seed=4 --db=bigbank returned: COMMAND_PROBLEM: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=true , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

• roachtest: tpcc/mixed-headroom/n5cpu16 failed [OOM during import while running 21.2] #74892 roachtest: tpcc/mixed-headroom/n5cpu16 failed [OOM during import while running 21.2] [C-test-failure O-roachtest O-robot T-disaster-recovery branch-release-21.2]

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-21223

[cockroach-teamcity]

roachtest.tpcc/mixed-headroom/n5cpu16 failed with artifacts on master @ c9094fa1a1ef8d44a2d5299d1d7380b34a06241b:

test artifacts and logs in: /artifacts/tpcc/mixed-headroom/n5cpu16/run_1
(test_impl.go:291).Fatal: monitor failure: monitor task failed: output in run_143247.567885604_n1_v2220beta4cockroach_workload_fixtures_import_bank: v22.2.0-beta.4/cockroach workload fixtures import bank --payload-bytes=10240 --rows=32552083 --seed=4 --db=bigbank returned: COMMAND_PROBLEM: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=true , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

• roachtest: tpcc/mixed-headroom/n5cpu16 failed [OOM during import while running 21.2] #74892 roachtest: tpcc/mixed-headroom/n5cpu16 failed [OOM during import while running 21.2] [C-test-failure O-roachtest O-robot T-disaster-recovery branch-release-21.2]

This test on roachdash | Improve this report!

[srosenberg]

(1) v22.2.0-beta.4/cockroach workload fixtures import bank --payload-bytes=10240 --rows=32552083 --seed=4 --db=bigbank returned
  | stderr:
  | I221106 14:32:48.680603 1 ccl/workloadccl/fixture.go:318  [-] 1  starting import of 1 tables
  | Error: importing fixture: importing table bank: pq: pausing due to error; use RESUME JOB to try to proceed once the issue is resolved, or CANCEL JOB to rollback: store 1 has insufficient remaining capacity to ingest data (remaining: 18 GiB / 5.0%, min required: 5.0%)
  |
  | stdout:
Wraps: (2) COMMAND_PROBLEM
Wraps: (3) Node 1. Command with error:
  | ```
  | v22.2.0-beta.4/cockroach workload fixtures import bank --payload-bytes=10240 --rows=32552083 --seed=4 --db=bigbank
  | ```
Wraps: (4) exit status 1

[srosenberg]

[srosenberg]

Disk capacity plot from the previous successful run, during the bank import step,

[cockroach-teamcity]

roachtest.tpcc/mixed-headroom/n5cpu16 failed with artifacts on master @ 4c828ca5df8ea70b92ba58c6cb4868303cee1262:

test artifacts and logs in: /artifacts/tpcc/mixed-headroom/n5cpu16/run_1
(test_impl.go:291).Fatal: monitor failure: monitor task failed: output in run_160818.142739663_n1_v2220beta4cockroach_workload_fixtures_import_bank: v22.2.0-beta.4/cockroach workload fixtures import bank --payload-bytes=10240 --rows=32552083 --seed=4 --db=bigbank returned: COMMAND_PROBLEM: exit status 1

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=16 , ROACHTEST_encrypted=true , ROACHTEST_fs=zfs , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

• roachtest: tpcc/mixed-headroom/n5cpu16 failed [OOM during import while running 21.2] #74892 roachtest: tpcc/mixed-headroom/n5cpu16 failed [OOM during import while running 21.2] [C-test-failure O-roachtest O-robot T-disaster-recovery branch-release-21.2]

This test on roachdash | Improve this report!

[renatolabs]

FWIW, if the failures here were indeed related to the ReturnSST error, they might still happen, so we should consider leaving this issue open until the fix is included in the release used by roachtest.

[srosenberg]

FWIW, if the failures here were indeed related to the ReturnSST error, they might still happen, so we should consider leaving this issue open until the fix is included in the release used by roachtest.

If it does regress, a fresh issue will be created; we could also reopen this one for context. Otherwise, there isn't any pending follow-up for us afaik.