Add Support for Intake ESM Catalogues in ESMValCore / ESMValTool

[rbeucher]

Description:

It would be valuable to build a mechanism within ESMValCore/ESMValTool to support Intake ESM catalogues (https://github.com/intake/intake-esm), which would extend to the ACCESS-NRI intake catalogue (https://access-nri-intake-catalog.readthedocs.io/en/latest/). Supporting Intake ESM would facilitate better data discovery and access, streamlining the process for users working with data collections hosted on platforms such as NCI.

There have already been several discussions regarding this topic:

• FROM: ACCESS-NRI Intake Catalog --> TO: ESMValTool .yml Recipe 'Datasets' CMIP7-Hackathon#32
• Add support for intake-esm ESMValGroup/ESMValCore#1218
• Consider using the intake-esm library ESMValGroup/ESMValCore#31

We aim to retain the current user interface of ESMValCore/ESMValTool, so any support for Intake ESM should involve mapping facets to the structure used in Intake catalogues. This will ensure a smooth and consistent user experience while leveraging the flexibility of Intake ESM for cataloguing and accessing data.

Proposed Approach:

• Begin with adding support for the CMIP Intake catalogue on NCI, focusing on the fs38 and io10 data collections.
• Investigate the possibility of incorporating the ESGF Intake catalogue as well, which could broaden the data access capabilities of ESMValTool.

Next Steps:

• Assess how facets in ESMValTool can be mapped to the structure of Intake catalogues.
• Collaborate with the Intake ESM team to ensure seamless integration and to address any challenges.

[rbeucher]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[rbeucher]

@charles-turner-1 for reference

[charles-turner-1]

@rbeucher I've put a proof of concept here, using intake to load the data for one of the ENSO recipes you and @flicj191 have been working on.

There are a few issues that need ironing out related to guessing bounds & potentially we could improve the way that we're converting xr.DataArrays into cubes (ncdata?), but honestly it looks like this might be relatively straightforward (touch wood).