testsuite: revert travis-ci to vm builds and fix many broken &&-chains in tests #1167
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
For flux version to be properly detected, we need to at least fetch tags and possibly also unshallow the git clone. Do this before running configure in travis-ci.
Fix broken `&&`-chains in some tests
Fix some broken &&-chains in some sharness tests. In some cases, the `&&` chain was broken presumably because a call was expected to fail, e.g. in the valid/invalid rank flux-module load/unload tests. These cases were fixed by either adding `!` or using `test_must_fail`.
Where appropriate, fix broken &&-chain in tests. The chain-lint check kludgery in sharness doesn't like when processes are sent into background within the test, so add a flux_exec_bg() shell function to wrap sending flux-exec into that background.
Fix broken &&-chain in tests so that --chain-lint passes on this test.
Add missing `&&`-chain in tests to make --chain-lint happy.
The check for flux-version output wasn't doing anything becuase of leftover debugging in the test. In fact, the test itself had been failing because of the use of extended regex pattern. Remove the debugging from the test and fix the regex pattern.
Fix the broken &&-chain in the heaptrace tests by pulling out the ||-based output checks into their own shell function.
Fix broken &&-chains in test so that --chain-lint doesn't complain.
Fix broken &&-chains in tests so that --chain-lint doesn't complain.
Fix broken &&-chain in tests so --chain-lint won't complain
Fix broken &&-chains in tests so that --chain-lint stops complaining
Fix a few broken &&-chains in tests so that --chain-lint doesn't complain.
Fix broken &&-chain in tests so --chain-lint doesn't complain. Some tests were checking for failed commands with `command; test $? -ne 0 && ...` these were replaced with `test_must_fail command && ...`
Fix broken &&-chains in tests so --chain-lint doesn't complain.
Fix broken &&-chains in tests so that --chain-lint doesn't complain.
Fix a broken &&-chain in a test to make --chain-lint happy.
Fix a broken &&-chain in a test so that --chain-lint doesn't complain.
Timeout the test_under_flux test after 5s in case the test hangs for unknown reason.
Since t0001-basic.t doesn't run under test_under_flux, it doesn't have a test timeout. To ensure invidual tests using flux-start do not hang the testsuite, add -o -Sinit.rc2_timeout option to each invocation of flux-start.
Set `sudo: true` in .travis.yml to revert to VM based builds instead of container/docker builds on Travis-CI. This is done because our build appears to require more resources than are given to the container-based Travis builds.
Add chain-lint check to the gcc-4.9 build in travis. This way, broken &&-chains will be detected before merging.
Of course after I stated that one test failed with |
Codecov Report
@@ Coverage Diff @@
## master #1167 +/- ##
==========================================
- Coverage 77.7% 77.65% -0.05%
==========================================
Files 158 158
Lines 29075 29041 -34
==========================================
- Hits 22592 22552 -40
- Misses 6483 6489 +6
|
|
These are good fixes, even if we don't have a 100 pct fix for the spurious travis errors. I'm for merging this if you're ready. |
|
Yes, I think it is ready. Pretty straightforward really. |
|
Thanks! |
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR reverts Travis-CI to use the vm builder instead of container builders by replacing
sudo: falsewithsudo: truein.travis.yml. In my testing, this change completely resolves the intermittent (but high frequency) failures under travis such as hangs thewrite errorproblem, and really slow builds (#1145, #1141, etc.)WIth that change in place, I was able to rebase many (minor) testsuite fixes I had on a branch, which fix up all the broken
&&-chains in the testsuite, as well as add timeouts to some tests which have the potential to hang.I think because many people have moved to the container build infrastructure, the VM build queue is not quite as long as it was getting before, so I don't think this change will slow down the turnaround time on Travis reporting.
This PR also uses one of the builders (currently the
gcc-4.9build) to run the suite with--chain-lintso we catch any broken&&-chains that might creep back in. (Many of these are harmless, however some broken&&chains could cause a false positive in a test, so best to keep the testsuite suite in this regard)