Sample data test infrastructure

[j-emberton]

Have added some basic infrastructure to get up and running with domain specific integration level tests:

Added matlab engine pytest fixture that can be configured to run matlab code from the original or refactored code repo.

Added fixture to read canonical test case data into temporary dirs for testing

Added /test_artifacts/ dir within /tests/ to be a home for reference test data.

Added the input and output files associated with the US_ARc_sample_data.

The idea here is that any individual test can demand a fixture which sets up a specific set of reference data for that test

This should form the backbone of testing against domain data unit and regression tests going forward

[j-emberton]

@dorchard @isaacaka @tztsai

I've setup the matlab engine envisaging that we have two parallel sets of matlab code - 'ustar_cp' and 'ustar_cp_refactor_wip'. The latter would initially be non-refactored matlab code, but over time would be incrementally refactored as we work on it.

This allows us to work from root to leaf as discussed yesterday with the lower level dependencies remaining intact while we refactor the higher level functions.

I'm hoping with this approach Matlab doesn't get confused with duplicate function names on its path - I haven't tested it yet.

This specific issue wouldn't affect the bulk of the implementation in this PR however.

If you are ok for us to test this approach I'll create the 'ustar_cp_refactor_wip' dir and migrate the matlab code into it.

[dorchard]

A few comments around documentation and a minor tweak but otherwise looks good. I think we probably also need a requirements.txt as well for the tests, or to extend the top-level one, to include matlabengine and shutil?

[isaacaka]

Looks good, left a comment on using shutil.copytree()

[j-emberton]

Looks good, left a comment on using shutil.copytree()

Good shout

[j-emberton]

A few comments around documentation and a minor tweak but otherwise looks good. I think we probably also need a requirements.txt as well for the tests, or to extend the top-level one, to include matlabengine and shutil?

I've extended the top level one for the time being, though matlab engine is only needed transiently. I think shutil is part of python standard lib so no need for an entry in requirements.txt.

[j-emberton]

OK, made the changes you guys suggested and created the duplicate matlab dir. Good to go I think.

[tztsai]

In PR #19 you also created a conftest.py file, which one has a newer version?

[dorchard]

Can we have a README.md in the tests directory that explains the requirements and workflow for running the tests? Also which version of Python should these be using wrt. the requirements? @j-emberton

[j-emberton]

Can we have a README.md in the tests directory that explains the requirements and workflow for running the tests? Also which version of Python should these be using wrt. the requirements? @j-emberton

Yes. But it might fit better in the launch.m PR where I've actually setup the test env and have running tests. Otherwise the readme will refer to stuff that doesn't exist yet.

[j-emberton]

In PR #19 you also created a conftest.py file, which one has a newer version?

You're right. This one is the latest one.

[dorchard]

(Note should work with Python 3.11)