Improve error messages when data is missing

[jvegreg]

Description

This pull request improves the error message shown when data is missing. In particular, ESMValCore is now able to show all missing data in one run

• Closes/Fixes: A first step to address Data alert tool to run before running the recipe ESMValTool#1365
• Link to documentation:

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

• PR has a descriptive title for the changelog
• Code follows the style guide
• Documentation is available for new functionality
• YAML files pass pre-commit or yamllint checks
• Circle/CI tests pass
• Codacy code quality checks pass
• Documentation builds successfully on readthedocs
• Unit tests are available

---

[jvegreg]

A (very small) sample of the output you get when trying to run ''recipe_perfmetrics_CMIP5.yml" in a laptop with virtually no data:

...

- Missing data for CMIP5_NorESM1-M: od550aer
- Missing data for CMIP5_NorESM1-ME: od550aer
- Missing data for OBS_ESACCI-AEROSOL: od550aer
- Missing data for OBS_MODIS: od550aer
2020-12-21 11:57:32,628 UTC [31263] ERROR   Missing data for preprocessor od870aer/od870aer:
- Missing data for CMIP5_ACCESS1-0: od870aer
- Missing data for CMIP5_ACCESS1-3: od870aer
- Missing data for CMIP5_CSIRO-Mk3-6-0: od870aer
- Missing data for CMIP5_GFDL-CM3: od870aer
- Missing data for CMIP5_GFDL-ESM2G: od870aer
- Missing data for CMIP5_GFDL-ESM2M: od870aer
- Missing data for CMIP5_HadGEM2-CC: od870aer
- Missing data for CMIP5_HadGEM2-ES: od870aer
- Missing data for CMIP5_MIROC-ESM-CHEM: od870aer
- Missing data for CMIP5_MIROC-ESM: od870aer
- Missing data for CMIP5_MIROC5: od870aer
- Missing data for CMIP5_MRI-CGCM3: od870aer
- Missing data for OBS: od870aer
2020-12-21 11:57:32,628 UTC [31263] ERROR   Missing data for preprocessor abs550aer/abs550aer:
- Missing data for CMIP5_CSIRO-Mk3-6-0: abs550aer
- Missing data for CMIP5_GFDL-CM3: abs550aer
- Missing data for CMIP5_GFDL-ESM2G: abs550aer
- Missing data for CMIP5_GFDL-ESM2M: abs550aer
- Missing data for CMIP5_GISS-E2-H: abs550aer
- Missing data for CMIP5_GISS-E2-R: abs550aer
- Missing data for CMIP5_IPSL-CM5B-LR: abs550aer
- Missing data for CMIP5_MIROC-ESM-CHEM: abs550aer
- Missing data for CMIP5_MIROC-ESM: abs550aer
- Missing data for CMIP5_MIROC5: abs550aer
- Missing data for CMIP5_NorESM1-M: abs550aer
- Missing data for CMIP5_NorESM1-ME: abs550aer
- Missing data for OBS: abs550aer
2020-12-21 11:57:32,628 UTC [31263] ERROR   Missing data for preprocessor toz/toz_derive_input_tro3:
- Missing data for CMIP5_CESM1-WACCM: tro3
- Missing data for CMIP5_CNRM-CM5: tro3
- Missing data for CMIP5_GFDL-CM3: tro3
- Missing data for CMIP5_GISS-E2-H: tro3
- Missing data for CMIP5_GISS-E2-R: tro3
- Missing data for CMIP5_MIROC-ESM-CHEM: tro3
- Missing data for OBS_ESACCI-OZONE: tro3
- Missing data for OBS_NIWA-BS: tro3
2020-12-21 11:57:32,783 UTC [31263] INFO    Maximum memory used (estimate): 0.2 GB
2020-12-21 11:57:32,784 UTC [31263] INFO    Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
2020-12-21 11:57:32,785 UTC [31263] ERROR   Program terminated abnormally, see stack trace below for more information
Traceback (most recent call last):
  File "/home/jvegas/git/esmval/ESMValCore/esmvalcore/_recipe.py", line 936, in __init__
    self.tasks = self.initialize_tasks() if initialize_tasks else None
  File "/home/jvegas/git/esmval/ESMValCore/esmvalcore/_recipe.py", line 1322, in initialize_tasks
    raise ex
esmvalcore._recipe_checks.RecipeError: Could not create all tasks

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jvegas/git/esmval/ESMValCore/esmvalcore/_main.py", line 430, in run
    fire.Fire(ESMValTool())
  File "/home/jvegas/miniconda3/envs/esmvaltool/lib/python3.8/site-packages/fire/core.py", line 138, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/home/jvegas/miniconda3/envs/esmvaltool/lib/python3.8/site-packages/fire/core.py", line 463, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/home/jvegas/miniconda3/envs/esmvaltool/lib/python3.8/site-packages/fire/core.py", line 672, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/home/jvegas/git/esmval/ESMValCore/esmvalcore/_main.py", line 407, in run
    process_recipe(recipe_file=recipe, config_user=cfg)
  File "/home/jvegas/git/esmval/ESMValCore/esmvalcore/_main.py", line 94, in process_recipe
    recipe = read_recipe_file(recipe_file, config_user)
  File "/home/jvegas/git/esmval/ESMValCore/esmvalcore/_recipe.py", line 63, in read_recipe_file
    return Recipe(raw_recipe,
  File "/home/jvegas/git/esmval/ESMValCore/esmvalcore/_recipe.py", line 941, in __init__
    raise RecipeError('Some tasks could not be initalized')
esmvalcore._recipe_checks.RecipeError: Some tasks could not be initalized
2020-12-21 11:57:32,788 UTC [31263] INFO    If you suspect this is a bug or need help, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues and attach the run/recipe_*.yml and run/main_log_debug.txt files from the output directory.

[valeriupredoi]

nice initiative @jvegasbsc - do you remember the old dry-run PR we worked on back in pre-pandemic times? #307 - I think we should have a go at making the dry run work proper since that is offering a lot to the user (cmor checks and all). What do you think would be a good strategy - the old PR is obviously very old - should we revive it or we can implement bits of that functionality here?

[jvegreg]

I see this as complimentary to the dry-run: this just tell you which data are you missing, the dry run will tell you which data needs to be fixed.

I think we can get this and then focus on the other.

[jvegreg]

The branch name is misleading because I started thinking on doing both at the same time but then realize it will be better to do it in separated PR

[valeriupredoi]

right you are @jvegasbsc - I will try revive that dry-run PR in the coming days, am dreading the amount of conflicts I'll get at merge master 😆

[jvegreg]

I think it will be great to have this for v2.2, specially to help us with all the tesing. Anybody in the @ESMValGroup/technical-lead-development-team @ESMValGroup/scientific-lead-development-team can find a bit of time to test it today? Just try to run a recipe that you don't have all the data for (check_obs is great for that)

[valeriupredoi]

will be testing on JASMIN, am just recreating the env for iris3 right now 👍

[valeriupredoi]

sorry man, JASMIN is painfully slow today - am still running the recipe_check_obs and it hasn't hit a no data issue, am approving this since it looks good in theory 😁

[bettina-gier]

Worked nicely for me on Mistral using the check_obs recipe! Also no spelling mistakes in the code changes ;)

Definitely in favour of merging this for the release, useful tool to have.