Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

write output files from Python tests to temporary system directory rather than source subdirectory #1121

Merged
merged 7 commits into from
Feb 14, 2020

Conversation

oskooi
Copy link
Collaborator

@oskooi oskooi commented Feb 11, 2020

Several of the Python tests create output files due to e.g. output_epsilon, save_flux, Animate2D, etc. which are placed in the python/tests subdirectory during make check. As a security measure, some operating systems prevent the test suite from writing any kind of file to disk in non-temporary system directories. This PR modifies the output directory of any test which creates a file to a temporary directory determined by the OS at runtime via tempfile.mkdtemp.

Additionally, all instances of os.path.realpath are replaced with os.path.abspath since on certain systems expanding ".." using realpath is not allowed.

TODOs:

  1. since MPB does not contain the equivalent of Meep's use_output_directory parameter, the only test that is left out is python/tests/mpb.cpp.

  2. modify the C++ tests in tests which also generate output files to write them to a temporary directory.

note: the tests on Travis are failing due to the following error:

Traceback (most recent call last):
5280  File "../../../python/tests/holey_wvg_cavity.py", line 147, in <module>
5281    temp_dir = mp.comm.bcast(temp_dir, root=0)
5282AttributeError: module 'meep' has no attribute 'comm'

@stevengj
Copy link
Collaborator

Would be nice to make this easier to access from C++ and reduce amount of repeated code.

Maybe just update the make_output_directory function in C++ to add a no-argument version that creates a temporary directory?

@stevengj
Copy link
Collaborator

Note that the analogous function from C is mkdtemp

@stevengj
Copy link
Collaborator

Does mp.make_output_directory('foo', 'bar') work now from Python? Just wondering if the typemaps work here.

@oskooi
Copy link
Collaborator Author

oskooi commented Feb 12, 2020

Following #1122, Python's tempfile.mkdtemp has been replaced with meep.make_output_directory. The C++ tests will be updated in a separate PR.

@stevengj stevengj merged commit 70c9572 into NanoComp:master Feb 14, 2020
@oskooi oskooi deleted the python_tests_output_tempdir branch March 25, 2020 21:42
bencbartlett pushed a commit to bencbartlett/meep that referenced this pull request Sep 9, 2021
…ther than source subdirectory (NanoComp#1121)

* write output files from Python tests to temporary system directories

* fixes

* only use comm.bcast if running under MPI

* abspath fixes/tweaks

* add documentation

* include temp dir in filename_prefix for mpb.cpp

* replace tempfile.mkdtemp with meep.make_output_directory
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants