Improve folder structure of hdf5 / pouts / .dat #166

TaigoFr · 2021-03-13T00:55:13Z

This PR improves the organization of the folder structure of all the output to the filesystem in several ways.

often people have to specify the full path of checkpoint files and plotfiles and restart files in the respective parameters repeateadly (e.g. chk_prefix = "/scratch/mypath/BinaryBH_" and similarly for the other parameters). Now these can be left with just the prefix (i.e. chk_prefix = BinaryBH_) and one new parameter was introduced: output_path , the base path prepending all outputs. It defaults to "", so it doesn't break any old code. If non-empty, output_path must exist already before the start of the job.
hdf5 files, pout files and .dat files (puncture files and extraction files) can now be put in their own specific folder using hdf5_subpath, pout_subpath and data_subpath (output_path prepends all of them). Again, they default to "", so they don't break any code. Unlike output_path, any subpath doesn't need to be created before hand, this PR takes care of creating them (even if they contain multiple levels of uncreated directories which need to be created recursively).
the name of the pout files can also be changed now to whatever prefix one wants, using the parameter pout_prefix. This might be useful not to override old pout files.
Extraction data typically has a lot of files when write_extraction is on. These files can be included in another directory by specifying extraction_subpath (which defaults to data_subpath)
pout verbosity can be reduced with print_progress_only_to_rank_0, which will print the number of boxes per rank on ranks other than rank 0 only at regrid time steps. For large number of nodes this improves performance.

Example:
Adding to the BinaryBH Example params_very_cheap.txt the following:

output_path = "results" # must exist!
hdf5_subpath = "hdf5"
pout_subpath = "pout"
data_subpath = "data"
pout_prefix = "poutChanged"

would put all the files in a folder results/ with files in subdirectories: hdf5's in results/hdf5, pout's in results/pout and .dats files in results/data. It would also change the name of the pout files to poutChanged.

Let me know if you think there is a better naming for any parameter.

Fully backwards compatible. The base folder where all files are outputted is now 'output_path', defaulted to "". This folder must exist. HDF5's are written to 'output_path + hdf5_subpath'. pout files are written to 'output_path + pout_subpath' and can be renamed to 'pout_prefix'. .dat files (punctures, extraction files) are written to 'output_path + extraction_subpath'. All subpaths are defaulted to "". They are created at runtime, so need not exist before.

mirenradia

These changes look great, Tiago! Thanks for doing this 😄 I have tested that it works for me. On my laptop it wasn't necessary for the output_path to exist before running but perhaps this is architecture/compiler/OS dependent.

I have some [relatively minor] comments.

Source/utils/GRParmParse.hpp

mirenradia · 2021-03-13T19:43:43Z

Source/GRChomboCore/ChomboParameters.hpp

+                        GRParmParse::folder_exists(output_path),
+                        "should be a valid folder");
+        check_parameter("pout_path", pout_path,
+                        GRParmParse::folder_exists(pout_path),
+                        "should be a valid folder");


In some respects, this check is pointless since either the path will have been created by the system (as on my laptop) or the code will have failed because the path doesn't exist and it tried to write some default parameter messages to the pout file. Maybe this check needs to be written to std::cout instead in read_filesystem_params and not use the usual check_parameter functions.

Actually the folder was created on your laptop because even the mkdir_recursive for pout's created it, since it's recursive!!! But had you not used a pout/hdf5/.dat subpath, then it would not have been created. So what we mean by "must exist" is probably that we do not take direct responsibility in creating it ourselves.
I will remove the pout check -> indeed we create it ourselves, so it will exist if nothing goes wrong. The output_path check will give an error if the directory doesn't exist and you didn't specify any pout subpath (and hence output_path was not created by mkdir_recursive)

Source/utils/GRParmParse.hpp

Source/GRChomboCore/ChomboParameters.hpp

Source/AMRInterpolator/SurfaceExtraction.impl.hpp

Source/BlackHoles/PunctureTracker.cpp

Source/utils/GRParmParse.hpp

mirenradia · 2021-03-15T15:43:44Z

I've just thought it might be useful to put the mode integral files into a subdirectory of the extraction_subpath directory? When write_extraction == true, there are a lot of Weyl4ExtractionOut_ files and it would be nice to separate them. On the other hand, I think most users don't set write_extraction = true and the default is false.

To print the number of boxes on all ranks on regrids, this is done in postRegrid. postRegrid is not called at t=0 (or at a restart), so GRAMRLevel::advance prints if it's rank 0 OR if it's t=restart_time

Source/GRChomboCore/GRAMRLevel.cpp

Also fixed bug (constraint norms should be sent to appropriate directory at 'data_path')

mirenradia

I ❤ the nicely formatted parameter files. Unfortunately, I still found a few comments to make.

Examples/KerrBH/params.txt

Examples/KerrBH/params_cheap.txt

Examples/ScalarField/params.txt

Source/GRChomboCore/ChomboParameters.hpp

Source/GRChomboCore/SimulationParametersBase.hpp

Source/utils/FilesystemTools.hpp

Source/AMRInterpolator/SurfaceExtraction.impl.hpp

mirenradia · 2021-03-20T17:12:41Z

I have tested all the parameter files with the -check_params flag and there were no issues.

mirenradia

I'm happy for this to be merged.

…prove_folder_structure Improve folder structure of hdf5 / pouts / .dat

TaigoFr requested review from KAClough and mirenradia March 13, 2021 00:55

TaigoFr changed the title ~~Improve folder structure of hdf5 / pouts / .dat~~ Improve folder structure of hdf5 / pouts / .dat Mar 13, 2021

TaigoFr force-pushed the enhancement/improve_folder_structure branch from 1b7eb97 to dbc0fce Compare March 13, 2021 01:05

mirenradia requested changes Mar 13, 2021

View reviewed changes

Fixes for Miren's comments

956bcdb

TaigoFr force-pushed the enhancement/improve_folder_structure branch from 974b089 to bcf138a Compare March 17, 2021 22:12

Add option to pout() GRAMRLevel::advance only to rank 0

f772ac2

To print the number of boxes on all ranks on regrids, this is done in postRegrid. postRegrid is not called at t=0 (or at a restart), so GRAMRLevel::advance prints if it's rank 0 OR if it's t=restart_time

TaigoFr force-pushed the enhancement/improve_folder_structure branch from bcf138a to f772ac2 Compare March 17, 2021 22:17

mirenradia assigned TaigoFr Mar 17, 2021

mirenradia added the enhancement Modification of existing feature/general improvement label Mar 17, 2021

Fixes for serial and debug mode

c582e34

mirenradia reviewed Mar 18, 2021

View reviewed changes

Source/GRChomboCore/GRAMRLevel.cpp Outdated Show resolved Hide resolved

TaigoFr added 3 commits March 18, 2021 11:42

Tidy up parameters files

a9dd403

Move 'activate_extraction' to SimulationParametersBase

2e01962

Also fixed bug (constraint norms should be sent to appropriate directory at 'data_path')

Edit minor details in BBH params

3c57474

mirenradia self-requested a review March 20, 2021 16:12

mirenradia requested changes Mar 20, 2021

View reviewed changes

Fixes for Miren's comments

ad3f1f8

mirenradia self-requested a review March 22, 2021 22:54

mirenradia approved these changes Mar 22, 2021

View reviewed changes

mirenradia merged commit a8135ce into main Mar 26, 2021

mirenradia deleted the enhancement/improve_folder_structure branch March 26, 2021 11:22

mirenradia mentioned this pull request Apr 3, 2021

restart_file check does include hdf5_subpath #172

Closed

mirenradia mentioned this pull request May 5, 2021

Update example for diagnostic use and to give explicit example GRTLCollaboration/Postprocessing_tools#14

Open

mirenradia added a commit to mirenradia/GRChombo-public that referenced this pull request Feb 3, 2023

Merge pull request GRTLCollaboration#166 from GRChombo/enhancement/im…

dac1a58

…prove_folder_structure Improve folder structure of hdf5 / pouts / .dat

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve folder structure of hdf5 / pouts / .dat #166

Improve folder structure of hdf5 / pouts / .dat #166

TaigoFr commented Mar 13, 2021 •

edited

Loading

mirenradia left a comment

mirenradia Mar 13, 2021

TaigoFr Mar 17, 2021

mirenradia commented Mar 15, 2021

mirenradia left a comment •

edited

Loading

mirenradia commented Mar 20, 2021

mirenradia left a comment

Improve folder structure of hdf5 / pouts / .dat #166

Improve folder structure of hdf5 / pouts / .dat #166

Conversation

TaigoFr commented Mar 13, 2021 • edited Loading

mirenradia left a comment

Choose a reason for hiding this comment

mirenradia Mar 13, 2021

Choose a reason for hiding this comment

TaigoFr Mar 17, 2021

Choose a reason for hiding this comment

mirenradia commented Mar 15, 2021

mirenradia left a comment • edited Loading

Choose a reason for hiding this comment

mirenradia commented Mar 20, 2021

mirenradia left a comment

Choose a reason for hiding this comment

TaigoFr commented Mar 13, 2021 •

edited

Loading

mirenradia left a comment •

edited

Loading