Default to using the golang-based conformance runner #916
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In Kubernetes 1.16.0 a problem in the bash runner of the conformance tests caused the plugin to fail to report results whenever a test failure occured. This problem does not impact the golang-based runner which was introduced also in Kubernetes 1.16.0. By default we would like to use this feature as a result. It is a new feature but has been shown so far to be functional and an improvement over the bash-based runner (e.g. like this issue in general). Users of Sonobuoy v0.16.0 should update to a version which utilizes this env var by default, but they can also specify --plugin-env e2e.E2E_USE_GO_RUNNER=true to workaround the issue. If users of new versions of Sonobuoy wish to NOT use the go runner they can also set --plugin-env e2e.E2E_USE_GO_RUNNER to unset the value. Fixes #910 Signed-off-by: John Schnake <[email protected]>
Codecov Report
@@ Coverage Diff @@
## master #916 +/- ##
==========================================
+ Coverage 47.42% 47.47% +0.04%
==========================================
Files 76 76
Lines 5155 5156 +1
==========================================
+ Hits 2445 2448 +3
+ Misses 2560 2559 -1
+ Partials 150 149 -1
Continue to review full report at Codecov.
|
zubron
approved these changes
Sep 28, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
In Kubernetes 1.16.0 a problem in the bash runner of the conformance
tests caused the plugin to fail to report results whenever a test
failure occured.
This problem does not impact the golang-based runner which was
introduced also in Kubernetes 1.16.0. By default we would like to use
this feature as a result. It is a new feature but has been shown
so far to be functional and an improvement over the bash-based
runner (e.g. like this issue in general).
Users of Sonobuoy v0.16.0 should update to a version which utilizes
this env var by default, but they can also specify
--plugin-env e2e.E2E_USE_GO_RUNNER=true to workaround the issue.
If users of new versions of Sonobuoy wish to NOT use the go runner
they can also set --plugin-env e2e.E2E_USE_GO_RUNNER to unset the value.
Which issue(s) this PR fixes
Fixes #910
Special notes for your reviewer:
You can test this on older clusters but just use
--kube-conformance-image-version=v1.16.0If you want to repro, use current Sonobuoy master with this command:
That command will provide a bad regexp to the tests and they will immediately fail. On master, the container fails and 5 min later Sonobuoy will acknowledge that the container failed and stop waiting for the results to come in.
If you try the same command with this branch or by adding
--plugin-env e2e.E2E_USE_GO_RUNNER=trueyou will see that the plugin completes but has unknown results since it submitted results but does not provide any xml since it never even got to test invocation.Release note: