Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Obs4MIPs names in perfmetrics_CMIP5 #3325

Merged
merged 10 commits into from
Nov 14, 2023

Conversation

rbeucher
Copy link
Contributor

Description

Following discussion here #2974

CERES-EBAF Ed2-7 is not available on ESGF anymore.
CERES-EBAF Ed2-8 is available for variable rlut, rlutcs, rsdt, rsut, rsutcs.
This PR removes the version facet for CERES-EBAF which fix automatic download from ESGF.

GPCP-SG v2.2 ['pr'] is GPCP-V2.2 on ESGF.
AIRS RetStd-v5 ['hus'] is AIRS-2-1 on ESGF

Note that when trying to download AIRS-2-1, ESMValTool adds the time_frequency facet to the ESGF query while AIRS-2-1 uses frequency.
The query does not return the data successfully. We have noticed that issue for a few other datasets, see #3317 discussion.
I have updated the version to align with ESGF anyway.


Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

New or updated recipe/diagnostic


To help with the number of pull requests:

Copy link
Member

@bouweandela bouweandela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for taking the time to contribute! It looks like the timerange for which the data is available on ESGF also differs between CERES-EBAF 2.7 and 2.8. The recipe still uses 2001-2015, but on ESGF there is only data for timerange: 200003-201311:

In [18]: esmvalcore.esgf.find_files(dataset='CERES-EBAF_Surface', short_name='*', project='obs4MIPs')
Out[18]: 
[ESGFFile:obs4MIPs/CERES-EBAF_Surface/v20160610/rlds_CERES-EBAF_L3B_Ed2-8_200003-201311.nc on hosts ['dpesgf03.nccs.nasa.gov'],
 ESGFFile:obs4MIPs/CERES-EBAF_Surface/v20160610/rldscs_CERES-EBAF_L3B_Ed2-8_200003-201311.nc on hosts ['dpesgf03.nccs.nasa.gov'],
 ESGFFile:obs4MIPs/CERES-EBAF_Surface/v20160610/rlus_CERES-EBAF_L3B_Ed2-8_200003-201311.nc on hosts ['dpesgf03.nccs.nasa.gov'],
 ESGFFile:obs4MIPs/CERES-EBAF_Surface/v20160610/rsds_CERES-EBAF_L3B_Ed2-8_200003-201311.nc on hosts ['dpesgf03.nccs.nasa.gov'],
 ESGFFile:obs4MIPs/CERES-EBAF_Surface/v20160610/rsdscs_CERES-EBAF_L3B_Ed2-8_200003-201311.nc on hosts ['dpesgf03.nccs.nasa.gov'],
 ESGFFile:obs4MIPs/CERES-EBAF_Surface/v20160610/rsus_CERES-EBAF_L3B_Ed2-8_200003-201311.nc on hosts ['dpesgf03.nccs.nasa.gov'],
 ESGFFile:obs4MIPs/CERES-EBAF_Surface/v20160610/rsuscs_CERES-EBAF_L3B_Ed2-8_200003-201311.nc on hosts ['dpesgf03.nccs.nasa.gov']]

In [19]: esmvalcore.esgf.find_files(dataset='CERES-EBAF', short_name='*', project='obs4MIPs')
Out[19]: 
[ESGFFile:obs4MIPs/CERES-EBAF/v20160610/rlut_CERES-EBAF_L3B_Ed2-8_200003-201404.nc on hosts ['dpesgf03.nccs.nasa.gov'],
 ESGFFile:obs4MIPs/CERES-EBAF/v20160610/rlutcs_CERES-EBAF_L3B_Ed2-8_200003-201404.nc on hosts ['dpesgf03.nccs.nasa.gov'],
 ESGFFile:obs4MIPs/CERES-EBAF/v20160610/rsdt_CERES-EBAF_L3B_Ed2-8_200003-201404.nc on hosts ['dpesgf03.nccs.nasa.gov'],
 ESGFFile:obs4MIPs/CERES-EBAF/v20160610/rsut_CERES-EBAF_L3B_Ed2-8_200003-201404.nc on hosts ['dpesgf03.nccs.nasa.gov'],
 ESGFFile:obs4MIPs/CERES-EBAF/v20160610/rsutcs_CERES-EBAF_L3B_Ed2-8_200003-201404.nc on hosts ['dpesgf03.nccs.nasa.gov']]

@mattiarighi Is it safe to update the timeranges to the available data for this dataset? Or would this impact the validity of the scientific results?

@zklaus
Copy link

zklaus commented Nov 9, 2023

@hb326 or @axel-lauer, could you weigh in on the shortened time span?

@axel-lauer
Copy link
Contributor

@hb326 or @axel-lauer, could you weigh in on the shortened time span?

I think the differences when going from edition 2.7 to 2.8 are probably similar to what could be expected when shortening the time period. So in short I think this should be fine, particularly since this is a legacy recipe. For scientific analyses done today, I would recommend using edition 4.2 (or later once available) anyways.

Copy link

@zklaus zklaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rbeucher, could you please shorten the time for the CERES-EBAF_Surface datasets according to the review?

esmvaltool/recipes/recipe_perfmetrics_land_CMIP5.yml Outdated Show resolved Hide resolved
esmvaltool/recipes/recipe_perfmetrics_land_CMIP5.yml Outdated Show resolved Hide resolved
esmvaltool/recipes/recipe_perfmetrics_land_CMIP5.yml Outdated Show resolved Hide resolved
esmvaltool/recipes/recipe_perfmetrics_land_CMIP5.yml Outdated Show resolved Hide resolved
@zklaus
Copy link

zklaus commented Nov 14, 2023

Thanks, @rbeucher! There are two more places that need adapting. To match the new names, the reference_dataset needs to also be changed from GPCP-SG to the appropriate version in recipe_perfmetrics_CMIP5.yml, line 616, and recipe_perfmetrics_CMIP5_4cds.yml, line 1274. This should also fix the test problems.

@rbeucher
Copy link
Contributor Author

Oh yes. on it

@rbeucher
Copy link
Contributor Author

:-/ What is going on here?

@zklaus
Copy link

zklaus commented Nov 14, 2023

I think there is a few more reference_datasets that need adapting. So far, I found AIRS in line 974; will look further.

@zklaus
Copy link

zklaus commented Nov 14, 2023

Yeah, seems to be just the AIRS in both those recipes in the same line 974.

I'm sorry the error message is not more helpful. Perhaps you could open an issue about that?

@rbeucher
Copy link
Contributor Author

OK looks good now...

Copy link
Contributor Author

@rbeucher rbeucher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be good to go

@zklaus zklaus dismissed bouweandela’s stale review November 14, 2023 12:21

Comments have been addressed.

@zklaus zklaus merged commit 7473e8b into ESMValGroup:main Nov 14, 2023
6 checks passed
@zklaus
Copy link

zklaus commented Nov 14, 2023

Cheers, @rbeucher! Thanks for all the work.

jvegreg pushed a commit that referenced this pull request Jan 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants