Cmorize cru v4.04 & stn contraint

[mwjury]

Description

• This is an update from CRU v4.02 to v4.04.
• Additionally an error in the time coordinate is fixed.
• Masking of data was added for grid points that have not been informed by station information and for which data is provided by statistic infilling of climatologic values. This results in additional cmorized output with version string TS4.04-stn1.

Checklist

It is the responsibility of the author to make sure the PR is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🛠 PR has a descriptive title for the changelog
• 🛠 Code follows the style guide
• 🛠 Circle/CI tests pass
• 🛠 Codacy code quality checks pass

New or updated data reformatting script:

• 🛠 Dataset is added to the table in the documentation
• 🛠 Documentation contains instructions to obtain the data
• 🛠 Tests for the CMORized data are available
• 🧪 Numbers/units of the data look physically meaningful
• 🛠 Data set is added to the OBS data pool @remi-kazeroni

[mwjury]

#1980 causing failed tests

[valeriupredoi]

cheers @mwjury - I will merge once we merge the fix to the iris documentation from #2012 🍺

[remi-kazeroni]

Nice work @mwjury! This is almost ready to be merged I think. Please just merge the master branch into this branch and extend the time_range to 2019 in the recipe_check_obs.yml. We could merge this PR after that.

[hb326]

This seems to be pretty much done, right? @remi-kazeroni fixed the end year to 2019, and #2021 seems to be merged, right @valeriupredoi?

[hb326]

@valeriupredoi: and this was one of the PR I wanted you to comment on.

[valeriupredoi]

looks good to me, I am just a bit quizzical about a relatively involved function that could be as simple as changing all days to the 15th of the month?

[remi-kazeroni]

Everything works fine for me, thanks @mwjury!

[remi-kazeroni]

This is another example where updating the cmorizer would delete the previous version (c.f. #2189). This case is a but simpler because the cmorizer script has not changed much (but the fix for the time coordinate differs between the 2 versions). So I'm not sure what is optimal between having:

• one config file and one cmorizer script per dataset version
• one config file per dataset version and a script that works on both
• a config file and a script for the latest version and leaving the information about older versions commented out.

Besides, we have a few recipes that are using the version TS4.02 of the CRU data for which we would delete the cmorizer by merging this PR: recipe_anav13jclim.yml, recipe_martin18grl.yml, schlund20jgr/recipe_schlund20jgr_gpp_change_rcp85.yml and schlund20jgr/recipe_schlund20jgr_gpp_abs_rcp85.yml. So it would be better for reproducibility reasons to keep both versions of the cmorizer.

I'd preferto solve that issue before merging this PR.

[katjaweigel]

How to continue with this PR?
There are already newer CRU Versions by now, latest at the moment:
[20 April 2023 CRU TS & CRU CY v4.07 released] (https://crudata.uea.ac.uk/cru/data/hrg/index.htm#current)
I would like to add additional Variables from this data set, e.g. evspsblpot (PET).

[remi-kazeroni]

I would recommend to this close this PR. It is outdated and still based on the old CMORizer interface. We could keep the branch for some time if it helps for future developments. Instead, you could open a new PR from latest main. Regarding the supported versions of the dataset, we now have a policy for that: https://docs.esmvaltool.org/en/latest/community/multiple_dataset_versions.html. Note that CRU is used in quite a few recipes. The science question is: shall we retire support for the old version or shall we support 2 versions in parallel? If the former, I would recommend to use the naming convention and implementation for GCP2018/2020. If the latter, you could try to reproduce what's done for the WOA CMORizer. This is a good example on how we can support multiple versions without creating too many datasets (like CRU4-02 and CRU4-07).

[katjaweigel]

Dear @remi-kazeroni thanks a lot for your answer! Because there are several recipes using CRU and because the data set is not too large (monthly data, limited number of variables) I think it would be best to support two versions in parallel. I will have a look at the the WOA CMORizer. We should keep the branch at the moment, because in addition to the version change it contains a fix for an error in the time coordinate and I first need to check if that is relevant for CRU 4.02 and 4.07 as well.

[lukruh]

I checked the CRU (TS) v4.06 and v4.07 and the time coordinate are not exactly centered. They are set to 16th 00:00:00 of each month (15th for februrary). Fixing them would provide exact boundaries and centered points. If this cause any changes in the results of the mentioned recipes they should be small. However, to be sure not to break them we could add the fix for v4.06+ when approaching a multi version CMORizer as in WOA. Should the download script also support multiple versions?

Another thing that seems to be missing in the current formatters/cru.py is the number of stations. I'm not sure if it's needed, but would it be a good idea to add it to the cube somehow (i.e. via auxiliary coordinate)?

[lukruh]

As the PR for latest CRU Versions is merged now (including the time fix) I think this PR can be closed. @mwjury?