Add test-wrapper option to wrap test execution #5995
Conversation
Sometimes we need to run the test ins some prebuild context, for these cases we now allow a script/application to be passed as `--test-wrapper`, which will obtain the execution arguments and be responsible for running the test in the environment. A test-wrapper could for example do the following to execute a test through wine: ``` set -euo pipefail WINEDLLOVERRIDES="winemac.drv=d" \ WINEDEBUG=-all+error \ LC_ALL=en_US.UTF-8 \ WINEPREFIX=$(mktemp -d) \ /path/to/bin/wine64 $@* ```
|
Looks fine in principle, but we should think about what happens when Also needs a changelog note and documentation, you can add the latter here: |
| @@ -294,6 +294,7 @@ data ElaboratedConfiguredPackage | |||
| elabTestHumanLog :: Maybe PathTemplate, | |||
| elabTestShowDetails :: Maybe TestShowDetails, | |||
| elabTestKeepTix :: Bool, | |||
| elabTestWrapper :: Maybe FilePath | |||
There was a problem hiding this comment.
Missing trailing comma means this doesn't compile.
| @@ -1884,6 +1884,7 @@ elaborateInstallPlan verbosity platform compiler compilerprogdb pkgConfigDB | |||
| elabTestHumanLog = perPkgOptionMaybe pkgid packageConfigTestHumanLog | |||
| elabTestShowDetails = perPkgOptionMaybe pkgid packageConfigTestShowDetails | |||
| elabTestKeepTix = perPkgOptionFlag pkgid False packageConfigTestKeepTix | |||
| elabTestWrapper = perPkgOptionFlag pkgid False packageConfigTestWrapper | |||
There was a problem hiding this comment.
Did you mean perPkgOptionMaybe pkgid packageConfigTestWrapper?
| "Run test through a wrapper." | ||
| testWrapper (\v flags -> flags { testWrapper = v }) | ||
| (reqArg' "FILE" (toFlag :: FilePath -> Flag FilePath) | ||
| (flagToList :: Flag FilePath -> [FilePath])) |
There was a problem hiding this comment.
You need to add this to at least Distribution.Cabal.Setup.testOptions if you want it to actually make it through to a user-facing option.
This is actually being used with the low level plumbing of the |
What would be the version number used in the code for |
|
Using 3.0.0.0 is fine. |
|
@JoshMeredith There are some CI failures still unfortunately. |
|
Looks like some QuickCheck tests are now failing. |
|
Are the current changes good to merge? |
|
@JoshMeredith Yes, feel free to push the big green button! |
Sometimes we need to run the test ins some prebuild context, for
these cases we now allow a script/application to be passed as
--test-wrapper, which will obtain the execution arguments andbe responsible for running the test in the environment.
A test-wrapper could for example do the following to execute a
test through wine:
Please include the following checklist in your PR:
[ci skip]is used to avoid triggering the build bots.Please also shortly describe how you tested your change. Bonus points for added tests!