Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachtest: refactor ClusterSpec to allow easier reporting of configs #81846

Closed
renatolabs opened this issue May 25, 2022 · 0 comments · Fixed by #87304
Closed

roachtest: refactor ClusterSpec to allow easier reporting of configs #81846

renatolabs opened this issue May 25, 2022 · 0 comments · Fixed by #87304
Labels
A-testing Testing tools and infrastructure C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@renatolabs
Copy link
Contributor

renatolabs commented May 25, 2022

ClusterSpec[1] is the struct that defines the infrastructure where a roachtest will run. Some fields in that struct are defined via a command line flag (e.g., cloud) and never changed again, while others may or may not be specified by tests and are resolved later when left blank (e.g., InstanceType, whether to use ZFS, encryption-at-rest, etc).

When a value is not specified and is resolved later, it is often not easy to find the value used. For instance, tests can opt-in to randomly use ZFS (RandomlyUseZfs). When enabled, there's a 20% chance of running with ZFS [2]; however, there's no easy way to report whether ZFS was used in a test failure because that value is not exported in a way that is easy to consume by other callers (most notably, when reporting parameters for a test failure [3]). This same happens for instance/machine type [4] and encryption-at-rest [5].

Ideally, we want to be able to have all these computed values easily accessible to higher layers in the stack when we report test failures.

Related issues and pull requests: #80799, #81845.

[1] https://github.com/cockroachdb/cockroach/blob/467a86fced31ef7c011f3bb8ab47499c6a6f497b/pkg/cmd/roachtest/spec/cluster_spec.go#L39-L62?plain=1
[2] https://github.com/cockroachdb/cockroach/blob/467a86fced31ef7c011f3bb8ab47499c6a6f497b/pkg/cmd/roachtest/spec/cluster_spec.go#L225-L230?plain=1
[3] https://github.com/cockroachdb/cockroach/blob/467a86fced31ef7c011f3bb8ab47499c6a6f497b/pkg/cmd/internal/issues/formatter_unit.go#L76?plain=1
[4] https://github.com/cockroachdb/cockroach/blob/467a86fced31ef7c011f3bb8ab47499c6a6f497b/pkg/cmd/roachtest/spec/cluster_spec.go#L192-L200?plain=1
[5] https://github.com/cockroachdb/cockroach/blob/467a86fced31ef7c011f3bb8ab47499c6a6f497b/pkg/cmd/roachtest/test_runner.go#L606-L616?plain=1

Jira issue: CRDB-16093

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testing Testing tools and infrastructure C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant