Issues found with the new multi-model statistics

[zklaus]

ESMValCore 2.3.0 comes with a new multi-model statistics implementation. Recently, tests with this version have uncovered a number of issues that we would like to address before we release ESMValTool 2.3.0. This probably leads to a bugfix release for ESMValCore.

To allow for discussions on the individual issues, I'll link to those in individual messages.

[zklaus]

There is a question of whether different scalar coordinates associated with near-surface temperature tas lead to problems in the mult-model statistics. This is discussed in ESMValGroup/ESMValCore#1204.

[zklaus]

This turns out to have been a misdiagnosis. This issue is considered resolved.

[zklaus]

Another issue is with daily data using different calendars. This is discussed in ESMValGroup/ESMValCore#1210.

[zklaus]

This issue has been resolved for now, a better long-term solution is under discussion in ESMValGroup/ESMValCore#1210 and ESMValGroup/ESMValCore#744.

[zklaus]

There are also some open questions with regards to performance, though for now we have decided to put them to one side until the functional questions are resolved. Some aspects are discussed in issue ESMValGroup/ESMValCore#781; a PR targeting 2.4.0 is at ESMValGroup/ESMValCore#968.

[sloosvel]

There also issues with datasets that may or may not have ancillary variables loaded in the cube. Although some cases may be because they are lost at another step (ESMValGroup/ESMValCore#1189), this would still be an issue for those datasets that don't have all the fx vars available. So I guess they should be removed before computing the statistics?

[zklaus]

Good point, @sloosvel. I have this on my list to handle, but I think there is no issue so far. Since I would assign you anyway 😏, would you mind opening the issue? Previous mentions/discussions can be found in
#2198 (comment) and #2198 (comment).

[zklaus]

The issue can be found now at ESMValGroup/ESMValCore#1213

[zklaus]

This issue has been resolved.

[zklaus]

Should multi-model statistics also be calculated on a single model? Or is that something that should be refused? This question is discussed in ESMValGroup/ESMValCore#1211.

[zklaus]

After some discussion, we decided to keep the current behavior, which is to refuse computation of multi-model statistics for a single cube. Hence, the issue is resolved.

[Peter9192]

Another issue is with conflicting dtypes: #2223

File "/mnt/esmvaltool_disk1/esmvalbot-work/esmvalbotwork-kcs_resamples-kqulx5ru/conda/envs/esmvaltool/lib/python3.6/site-packages/esmvalcore/preprocessor/_multimodel.py", line 262, in _combine
    merged_cube = cubes.merge_cube()
  File "/mnt/esmvaltool_disk1/esmvalbot-work/esmvalbotwork-kcs_resamples-kqulx5ru/conda/envs/esmvaltool/lib/python3.6/site-packages/iris/cube.py", line 404, in merge_cube
    proto_cube.register(cube, error_on_mismatch=True)
  File "/mnt/esmvaltool_disk1/esmvalbot-work/esmvalbotwork-kcs_resamples-kqulx5ru/conda/envs/esmvaltool/lib/python3.6/site-packages/iris/_merge.py", line 1358, in register
    match = cube_signature.match(other, error_on_mismatch)
  File "/mnt/esmvaltool_disk1/esmvalbot-work/esmvalbotwork-kcs_resamples-kqulx5ru/conda/envs/esmvaltool/lib/python3.6/site-packages/iris/_merge.py", line 465, in match
    raise iris.exceptions.MergeError(msgs)
iris.exceptions.MergeError: failed to merge into a single cube.
  cube data dtype differs: float64 != float32

Is this something the multi-model module should catch and maybe correct, or should it be part of the cmor fixes?

[zklaus]

The dtype issue has been reported before and I am investigating it. In principle, I think no float64 data should reach the preprocessors, so let's see where this leads.

[zklaus]

This happens because some preprocessors change the dtype prior to the multi-model stage. This has been resolved in ESMValGroup/ESMValCore#1237 and ESMValGroup/ESMValCore#1239.

[axel-lauer]

There also seems to be a problem with variables on hybrid levels. Multi-model statistics fail even after converting the hybrid levels into pressure or height levels as described in ESMValGroup/ESMValCore#1222

[zklaus]

Since all affected recipes are still under development, this issue has been relegated to the next release, 2.4.0.

[zklaus]

There also is a problem with missing bounds, see #2198 (comment) and ESMValGroup/ESMValCore#1201 (comment).

[zklaus]

Another problem is with bounds on pressure levels as mentioned at #2198 (comment).

The main problem here seems to be that ERA-Interim puts bounds on the plev19 coordinate, which intuitively makes sense, but is not requested by the CMIP data request and not provided by CMIP models. This prevents the merge.

[zklaus]

This is now tracked in #2241.