Sonobuoy must handle conformance+live cluster differently than full conformance #877
Assignees
Labels
Comments
|
cc @timothysc FYI |
This was referenced Sep 13, 2019
johnSchnake
added a commit
that referenced
this issue
Sep 17, 2019
) With K8s v1.16, disruptive tests may not be part of the conformance suite. As a result we need to ensure that user workloads are protected by default while still allowing for CNCF certification test runs. In this PR we: - rename the default `conformance` mode to be `non-disruptive-conformance` and modified the skip list appropriately. - added a `certified-conformance` mode which does not provide a skip list and will run even the disruptive tests - tweaked the naming to use lower case values by default to be more consistent with our other flags/naming Fixes #877 Fixes #875 Signed-off-by: John Schnake <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The CNCF conformance instructions do tell people to run
sonobuoy runto get results for certification.Historically sonobuoy ran the conformance tag but skipped alpha, disruptive, flakey, etc. That was all fine since they should have been exclusive. Now that those overlap some (if kubernetes/kubernetes#82664 is updated based on the peer feedback), we'll have to accommodate both flows:
sonobuoy runand get potentially destructive tests? These wouldn't be completely safe to run in a prod cluster.Sonobuoy will make this pretty simple since we have "modes" for the e2e plugin. Currently, we have the Conformance and Quick modes.
The new mode naming will need some thought:
And we'll need to decide which is the default
The text was updated successfully, but these errors were encountered: