-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Integration tests! #218
Comments
Just adding some notes on running the current integration test locally. Conda list output
run the test using openff-bespoke executor run --smiles 'CC' \
--workflow 'default' \
--default-qc-spec xtb gfn2xtb none \
--target-torsion '[C:1]-[C:2]' --directory bespokefit-1-3 --qc-compute-n-cores 1 The test works with the following output
Looking at the bottom of the forcebalance output I notice it runs 2 iterations (0 and 1) and completes in ~11 seconds as expected. I then created the test environment locally the Conda list output no-openeye
I ran the same command as above and changed bespokefit to run the optimisation in a local folder and not clean it up, the test fails so reproduces the CI. I noticed that this also runs 2 iterations but takes almost 60 seconds. |
Let's add an integration test suite to BespokeFit.
BespokeFit has good unit-test coverage, but much of its behavior is non-local. The 0.2.0 release was broken because of a bug in ForceBalance which was not detected by unit tests. The goal of this test suite should be to provide a place to test for non-local interactions in the codebase, to catch bugs before they are released, and to ensure that deployed versions of BespokeFit are still working.
The integration test suite should test at least the following behavior:
torsion_smirks
) without raising an error.optimise.err
is correctly reported (Better track ForceBalance Failures #217, Optimization failed with openff-bespokefit 0.1.1 (ForceBalance 1.9.4) #216, Support OpenFF Toolkit v0.11+ #198 (comment))This integration suite should not be run in PRs (it'll be too slow), but it should be executed:
main
)This integration suite should pull in dependencies from Conda as much as possible to match the recipe for the next release.
This integration suite should NOT be derived from the documentation - while code in the documentation should be tested, that is not the purpose of this suite. The documentation should be free to change without considering the impact on test coverage, and it should be possible to add to the test suite without having to invent new documentation.
The text was updated successfully, but these errors were encountered: