Skip to content

Commit

Permalink
Test fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
forsyth2 committed Nov 26, 2024
1 parent 6561a45 commit 65b2f09
Show file tree
Hide file tree
Showing 45 changed files with 397 additions and 321 deletions.
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,8 @@ simulations by automating commonly performed tasks.

See [documentation](https://e3sm-project.github.io/zppy) for more details.

- Run `python -m unittest discover -s tests -p "test_*.py"` to run all tests.
- Run `python -m unittest tests/test_*.py` to run only unit tests.
- Run `python -m unittest tests/integration/test_*.py` to run only integration tests (on server).
- Run `pytest tests/test_*.py` to run only unit tests.
- Run `pytest tests/integration/test_*.py` to run only integration tests (on server).

## License

Expand Down
2 changes: 1 addition & 1 deletion docs/source/dev_guide/new_glb_plot.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ Note that unlike E3SM Diags and MPAS-Analysis which are packages called by ``zpp
- In `coupled_global.py <https://github.com/E3SM-Project/zppy/blob/main/zppy/templates/coupled_global.py>`_: Add variables under "# Variables to extract". Add the code for your new plot under "Plotting functions"; the name of your function should start with ``plot_``. In ``PLOT_DICT``, add ``"function name without plot_" : function name``.
- If this plot should be added to the defaults, then complete this step: for ``plot_names`` under ``[global_time_series]`` in `default.ini <https://github.com/E3SM-Project/zppy/blob/main/zppy/templates/default.ini>`_, add the function name without plot to the string.
- Optionally, after making your edits, you can run ``pre-commit run --all-files`` to clean up your code and check for any obvious errors.
- Run the `integration tests <https://e3sm-project.github.io/zppy/_build/html/main/dev_guide/testing.html#integration-tests>`_ and examine the differences from the expected files. If they match what you expect, update the expected files following "Commands to run to replace outdated expected files" on the machine-specific directions. (The commands to run all the integration tests are ``pip install .`` followed by ``python -u -m unittest tests/integration/test_*.py``).
- Run the `integration tests <https://e3sm-project.github.io/zppy/_build/html/main/dev_guide/testing.html#integration-tests>`_ and examine the differences from the expected files. If they match what you expect, update the expected files following "Commands to run to replace outdated expected files" on the machine-specific directions. (The commands to run all the integration tests are ``pip install .`` followed by ``pytest tests/integration/test_*.py``).
4 changes: 2 additions & 2 deletions docs/source/dev_guide/release_testing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Testing directions for making a release
* Compy: ``source /share/apps/E3SM/conda_envs/load_latest_e3sm_unified_compy.sh``
* Perlmutter: ``source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh``

7. Run the unit tests with ``python -u -m unittest tests/test_*.py``.
7. Run the unit tests with ``pytest tests/test_*.py``.

a. test dev (run before making a new zppy RC):

Expand All @@ -107,7 +107,7 @@ Testing directions for making a release
* `Compy <https://github.com/E3SM-Project/zppy/blob/main/tests/integration/generated/directions_compy.md>`_
* `Perlmutter <https://github.com/E3SM-Project/zppy/blob/main/tests/integration/generated/directions_pm-cpu.md>`_

9. Run the integration tests with ``python -u -m unittest tests/integration/test_*.py``. Note that ``test_complete_run.py`` takes approximately 75 minutes to run on Compy.
9. Run the integration tests with ``pytest tests/integration/test_*.py``. Note that ``test_complete_run.py`` takes approximately 75 minutes to run on Compy.

a. test dev (run before making a new zppy RC):

Expand Down
4 changes: 2 additions & 2 deletions docs/source/dev_guide/testing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Run all unit tests by doing the following:
.. code::
pip install . # Install your changes (`python -m pip install .` also works)
python -u -m unittest tests/test_*.py # Run all unit tests
pytest tests/test_*.py # Run all unit tests
Integration tests
=================
Expand All @@ -24,7 +24,7 @@ Run all integration tests by doing the following:
pip install . # Install your changes (`python -m pip install .` also works)
python -u -m unittest tests/integration/test_*.py # Run all integration tests
pytest tests/integration/test_*.py # Run all integration tests
Automated tests
===============
Expand Down
4 changes: 2 additions & 2 deletions docs/source/dev_guide/tutorial_testing_e3sm_unified.rst
Original file line number Diff line number Diff line change
Expand Up @@ -368,5 +368,5 @@ Run the unit tests and integration tests on Chrysalis:
git fetch upstream
git rebase upstream/main
<ENVIRONMENT_COMMANDS> # Load the appropriate environment
python -u -m unittest tests/test_*.py # Run unit tests
python -u -m unittest tests/integration/test_*.py # Run integration tests
pytest tests/test_*.py # Run unit tests
pytest tests/integration/test_*.py # Run integration tests
2 changes: 1 addition & 1 deletion tests/integration/generated/directions_chrysalis.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Changing `UNIQUE_ID` allows you to simultaneously run jobs
launched from `zppy` on different branches,
without worrying about overwriting results.

NOTE: Actually running the tests (e.g., `python -u -m unittest tests/integration/test_*.py`)
NOTE: Actually running the tests (e.g., `pytest tests/integration/test_*.py`)
can not be done from two different branches simultaneously
(since files in the `zppy` directory rather than an external directory get changed).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ years = "1985:1995:5"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ walltime = "00:30:00"

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
[default]
case = "v3.LR.historical_0051"
constraint = ""
dry_run = "False"
environment_commands = ""
fail_on_dependency_skip = True
guess_path_parameters = False
guess_section_parameters = False
input = /lcrc/group/e3sm2/ac.wlin/E3SMv3/v3.LR.historical_0051
input_subdir = archive/atm/hist
mapping_file = "map_ne30pg2_to_cmip6_180x360_aave.20200201.nc"
output = "/lcrc/group/e3sm/ac.forsyth2/zppy_min_case_global_time_series_comprehensive_v3_setup_only_output/unique_id/v3.LR.historical_0051"
partition = "debug"
qos = "regular"
www = "/lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_min_case_global_time_series_comprehensive_v3_setup_only_www/unique_id"
years = "1985:1989:2",

[ts]
active = True
e3sm_to_cmip_environment_commands = ""
walltime = "00:30:00"

[[ atm_monthly_glb ]]
# Note global average won't work for 3D variables.
frequency = "monthly"
input_files = "eam.h0"
input_subdir = "archive/atm/hist"
mapping_file = "glb"
years = "1985:1995:5",

[[ lnd_monthly_glb ]]
frequency = "monthly"
input_files = "elm.h0"
input_subdir = "archive/lnd/hist"
mapping_file = "glb"
vars = "FSH,RH2M,LAISHA,LAISUN,QINTR,QOVER,QRUNOFF,QSOIL,QVEGE,QVEGT,SOILWATER_10CM,TSA,H2OSNO,TOTLITC,CWDC,SOIL1C,SOIL2C,SOIL3C,SOIL4C,WOOD_HARVESTC,TOTVEGC,NBP,GPP,AR,HR"
years = "1985:1995:5",

[mpas_analysis]
active = True
anomalyRefYear = 1985
climo_years = "1985-1989", "1990-1995",
enso_years = "1985-1989", "1990-1995",
mesh = "IcoswISC30E3r5"
parallelTaskCount = 6
partition = "compute"
qos = "regular"
shortTermArchive = True
ts_years = "1985-1989", "1985-1995",
walltime = "00:30:00"

# (This cfg is the setup portion only)
# [global_time_series]
# active = True
# climo_years = "1985-1989", "1990-1995",
# environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate zi_dev_weekly_20241122"
# experiment_name = "v3.LR.historical_0051"
# figstr = "v3.LR.historical_0051"
# moc_file=mocTimeSeries_1985-1995.nc
# plots_lnd = "FSH,RH2M,LAISHA,LAISUN,QINTR,QOVER,QRUNOFF,QSOIL,QVEGE,QVEGT,SOILWATER_10CM,TSA,H2OSNO,TOTLITC,CWDC,SOIL1C,SOIL2C,SOIL3C,SOIL4C,WOOD_HARVESTC,TOTVEGC,NBP,GPP,AR,HR"
# ts_num_years = 5
# ts_years = "1985-1989", "1985-1995",
# walltime = "00:30:00"
# years = "1985-1995",
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ years = "1985:1995:5",

[global_time_series]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate zi_dev_weekly_20241122"
experiment_name = "v3.LR.historical_0051"
figstr = "v3.LR.historical_0051"
plots_original=""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ walltime = "00:30:00"
[global_time_series]
active = True
climo_years = "1985-1989", "1990-1995",
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate zi_dev_weekly_20241122"
experiment_name = "v3.LR.historical_0051"
figstr = "v3.LR.historical_0051"
moc_file=mocTimeSeries_1985-1995.nc
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ walltime = "00:30:00"

[global_time_series]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate zi_dev_weekly_20241122"
experiment_name = "v3.LR.historical_0051"
figstr = "v3.LR.historical_0051"
plots_original="net_toa_flux_restom,global_surface_air_temperature,toa_radiation,net_atm_energy_imbalance,net_atm_water_imbalance"
Expand Down
4 changes: 2 additions & 2 deletions tests/integration/generated/test_weekly_bundles_chrysalis.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ years = "1985:1989:2",

[e3sm_diags]
active = True
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
ref_final_yr = 1989
ref_start_yr = 1985
Expand Down Expand Up @@ -141,7 +141,7 @@ years = "1985:1989:2",
[global_time_series]
active = True
bundle = "bundle2"
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate zi_dev_weekly_20241122"
experiment_name = "v3.LR.historical_0051"
figstr = "v3.LR.historical_0051"
plots_original="net_toa_flux_restom,global_surface_air_temperature,toa_radiation,net_atm_energy_imbalance,net_atm_water_imbalance"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ walltime = "00:30:00"
active = True
climo_diurnal_frequency = "diurnal_8xdaily"
climo_diurnal_subsection = "atm_monthly_diurnal_8xdaily_180x360_aave"
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down Expand Up @@ -168,7 +168,7 @@ walltime = "00:30:00"
[global_time_series]
active = True
climo_years ="1980-1984", "1985-1990",
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate zi_dev_weekly_20241122"
experiment_name = "v2.LR.historical_0201"
figstr = "v2.LR.historical_0201"
moc_file=mocTimeSeries_1980-1990.nc
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ active = True
climo_diurnal_frequency = "diurnal_8xdaily"
climo_diurnal_subsection = "atm_monthly_diurnal_8xdaily_180x360_aave"
climo_subsection = "atm_monthly_180x360_aave"
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate e3sm_diags_from_20241029"
grid = '180x360_aave'
multiprocessing = True
num_workers = 8
Expand Down Expand Up @@ -196,7 +196,7 @@ walltime = "00:30:00"
[global_time_series]
active = True
climo_years = "1985-1989", "1990-1995",
environment_commands = "source <INSERT PATH TO CONDA>/conda.sh; conda activate <INSERT ENV NAME>"
environment_commands = "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate zi_dev_weekly_20241122"
experiment_name = "v3.LR.historical_0051"
figstr = "v3.LR.historical_0051"
moc_file=mocTimeSeries_1985-1995.nc
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ rm -rf /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bash_files
# You can just move (i.e., not copy) the output since re-running this test will re-generate the output.
mv test_bash_generation_output/post/scripts /lcrc/group/e3sm/public_html/zppy_test_resources/expected_bash_files
# Rerun test
python -u -m unittest tests/integration/test_bash_generation.py
pytest tests/integration/test_bash_generation.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ do
done

# Rerun test
python -m unittest tests/integration/test_campaign.py
pytest tests/integration/test_campaign.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ mkdir -p /lcrc/group/e3sm/public_html/zppy_test_resources/test_defaults_expected
# You can just move (i.e., not copy) the output since re-running this test will re-generate the output.
mv test_defaults_output/post/scripts/*.settings /lcrc/group/e3sm/public_html/zppy_test_resources/test_defaults_expected_files
# Rerun test
python -u -m unittest tests/integration/test_defaults.py
pytest tests/integration/test_defaults.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ do
done

# Rerun test
python -u -m unittest tests/integration/test_weekly.py
pytest tests/integration/test_weekly.py
2 changes: 1 addition & 1 deletion tests/integration/template_directions.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Changing `UNIQUE_ID` allows you to simultaneously run jobs
launched from `zppy` on different branches,
without worrying about overwriting results.

NOTE: Actually running the tests (e.g., `python -u -m unittest tests/integration/test_*.py`)
NOTE: Actually running the tests (e.g., `pytest tests/integration/test_*.py`)
can not be done from two different branches simultaneously
(since files in the `zppy` directory rather than an external directory get changed).

Expand Down
Loading

0 comments on commit 65b2f09

Please sign in to comment.