jepsen: More nemeses, flexible configuration #19697
Labels
A-testing
Testing tools and infrastructure
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
no-issue-activity
X-stale
Comments
17 tasks
bdarnell
added a commit
to bdarnell/cockroach
that referenced
this issue
Nov 8, 2017
Enable more nemeses: subcritical-skews, strobe-skews, split Enable more tests: bank-mulittable, g2 Updates cockroachdb#19697
bdarnell
added a commit
to bdarnell/cockroach
that referenced
this issue
Nov 9, 2017
Enable more nemeses: subcritical-skews, split Enable more tests: bank-mulittable, g2 Disable the comments test, which is expected to fail (in practice, it only fails with one of the skew nemeses). Decrease running time per configuration from 6m to 5m to slightly offset the increased number of configurations. Running the full matrix produces enough logs to fill up the workers' disks without this change. Updates cockroachdb#19697
bdarnell
added
C-enhancement
|
We have marked this issue as stale because it has been inactive for |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Our jepsen tests support some nemesis configurations that we are not currently using:
split, and a family of-skewsconfigurations. We should enable both thesplitnemesis and at least one of theskewsnemeses.We currently run each (test, nemesis) combination for six minutes, so adding a new nemesis adds a significant amount of runtime to the test suite. The failure noted in #17815 occurs once per several hours of runtime, so increasing the test duration would improve our ability to find failures like this. However, adding configurations or increasing runtime has a cost in machine resources. We should figure out how we want to handle this, which may include running some test+nemesis configurations longer than others. For example, the
setsandsequentialtests seem to be our most sensitive ones; the other tests have rarely noticed an issue that wasn't also caught by one of those too, so we should give them a larger fraction of our test runtime budget.The text was updated successfully, but these errors were encountered: