e2e_shadow_indexing_test: bump timeout for workload #11828
Conversation
|
/ci-repeat 4 |
|
/ci-repeat 8 |
|
/ci-repeat 7 |
|
CI failures: #11944 |
|
/cdt tests/rptest/tests/e2e_shadow_indexing_test.py::ShadowIndexingManyPartitionsTest.test_many_partitions_shutdown dt-repeat=10 dt-log-level=debug |
andrwng
force-pushed
the
many-partitions-shutdown-bump
branch
from
eb00e60
to
67d255c
Compare
|
/cdt tests/rptest/tests/e2e_shadow_indexing_test.py::ShadowIndexingManyPartitionsTest.test_many_partitions_shutdown dt-repeat=10 dt-log-level=debug |
|
Is this close to done, @andrwng ? These issues are failing quite a bit. |
The fixes have so far whittled the failures down to just the shutdown hang, which seems fairly reliable in CDT. Adding more debug logging to debug; hopefully will get to the root cause today. |
|
/cdt tests/rptest/tests/e2e_shadow_indexing_test.py::ShadowIndexingManyPartitionsTest.test_many_partitions_shutdown dt-repeat=3 dt-log-level=trace |
|
/ci-repeat |
|
/cdt tests/rptest/tests/e2e_shadow_indexing_test.py::ShadowIndexingManyPartitionsTest.test_many_partitions_shutdown dt-repeat=3 dt-log-level=trace |
1 similar comment
|
/cdt tests/rptest/tests/e2e_shadow_indexing_test.py::ShadowIndexingManyPartitionsTest.test_many_partitions_shutdown dt-repeat=3 dt-log-level=trace |
|
The latest runs show the Kafker server's |
|
/ci-repeat |
Empirically, segments may be uploaded ~1/partition/sec, particularly on dockeriszed environments. Thus, the current timeout for the number of segments we want to generate is flaky.
andrwng
force-pushed
the
many-partitions-shutdown-bump
branch
2 times, most recently
from
e0e1065
to
810069e
Compare
|
/cdt tests/rptest/tests/e2e_shadow_indexing_test.py::ShadowIndexingManyPartitionsTest.test_many_partitions_shutdown dt-repeat=30 |
This makes the final cloud storage scrub timeout configurable for tests that expect to write a lot of data.
In CDT this test could end up generating way more data than in dockerized tests, resulting in the consume workload taking much longer than expected. This patch reduces the workload to ensure a more reasonable runtime.
andrwng
force-pushed
the
many-partitions-shutdown-bump
branch
from
810069e
to
60a0614
Compare
|
Failure is #12104 |
Empirically, segments may be uploaded ~1/partition/sec, particularly on dockeriszed environments. Thus, the current timeout for the number of segments we want to generate is flaky.
Fixes #11268
Also fixes slow scrubbing, from when there are too many segments to analyze in 30s.
Fixes #11698
Remaining flakiness appears to be fixed by #12756
Backports Required
Release Notes