recipe_perfmetrices crashes due to standard name error #1561

mattiarighi · 2020-03-03T10:53:59Z

Traceback (most recent call last):
  File "/pf/b/b309057/SOFTWARE/miniconda3/envs/esmvaltool/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_task.py", line 683, in _run_task
    output_files = task.run()
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/_task.py", line 238, in run
    self.output_files = self._run(input_files)
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/preprocessor/__init__.py", line 426, in _run
    product.apply(step, self.debug)
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/preprocessor/__init__.py", line 294, in apply
    self.cubes = preprocess(self.cubes, step, **self.settings[step])
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/preprocessor/__init__.py", line 236, in preprocess
    result.append(_run_preproc_function(function, items, settings))
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/preprocessor/__init__.py", line 222, in _run_preproc_function
    return function(items, **kwargs)
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/cmor/fix.py", line 140, in fix_metadata
    cube = checker(cube).check_metadata()
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/cmor/check.py", line 101, in check_metadata
    self._check_var_metadata()
  File "/mnt/lustre01/pf/b/b309057/ESMValTool/core/esmvalcore/cmor/check.py", line 207, in _check_var_metadata
    self._cube.standard_name = self._cmor_var.standard_name
  File "/pf/b/b309057/SOFTWARE/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/iris/_cube_coord_common.py", line 248, in standard_name
    self._standard_name = get_valid_standard_name(name)
  File "/pf/b/b309057/SOFTWARE/miniconda3/envs/esmvaltool/lib/python3.7/site-packages/iris/_cube_coord_common.py", line 84, in get_valid_standard_name
    name))
ValueError: 'atmosphere_optical_thickness_due_to_pm1_ambient_aerosol' is not a valid standard_name

Not sure what is causing the problem.
The last successfull test with this recipe was on Feb 18.

The text was updated successfully, but these errors were encountered:

valeriupredoi · 2020-03-03T11:07:43Z

perfmetrics is a monster and would really not want to run it - can you pls tell us what's the offending dataset (full specs, so I can look it up on Jasmin too, might be that the actual data file changed in the meantime) 🍺

mattiarighi · 2020-03-03T11:08:38Z

Investigating... 🔍

The variable should be obs550lt1aer.

jvegreg · 2020-03-03T11:12:50Z

It is a legal standard_name, so it looks like an Iris problem. Which version are you using?

From last CF table:

atmosphere_optical_thickness_due_to_pm1_ambient_aerosol_particles
alias: atmosphere_optical_thickness_due_to_pm1_ambient_aerosol

mattiarighi · 2020-03-03T11:15:03Z

iris                      2.4.0                    py37_0    conda-forge

mattiarighi · 2020-03-03T11:26:19Z

Another hint:

2020-03-03 11:25:05,987 UTC [10236] ERROR   Failed to run fix_metadata([<iris 'Cube' of Length of average period / (days) (time: 60)>, <iris 'Cube' of Start time for average period / (days since 1860-01-01 00:00:00) (time: 60)>, <iris 'Cube' of Ambient Fine Aerosol Opitical Thickness at 550 nm / (1) (time: 60; latitude: 90; longitude: 144)>, <iris 'Cube' of End time for average period / (days since 1860-01-01 00:00:00) (time: 60)>], {'project': 'CMIP5', 'dataset': 'GFDL-CM3', 'short_name': 'od550lt1aer', 'mip': 'aero', 'frequency': 'mon'})

jvegreg · 2020-03-03T11:37:27Z

iris 2.4.0 py37_0 conda-forge

It has the alias in its standard_names XML... Iris should accept it

mattiarighi · 2020-03-03T11:38:22Z

Apparently it is failing to apply a fix to GFDL-CM3

jvegreg · 2020-03-03T11:46:09Z

It is the automatic one correcting the standard_name when it does not match the CMOR tables. But Iris is complaining it is not legal for some reason

valeriupredoi · 2020-03-03T12:56:38Z

right so that dam file has got no standard_name and the only way to load is via long_name - see here (standard_name in commented out bit):

c = iris.load("/badc/cmip5/data/cmip5/output1/NOAA-GFDL/GFDL-CM3/historical/mon/aerosol/aero/r1i1p1/latest/od550lt1aer/od550lt1aer_aero_GFDL-CM3_historical_r1i1p1_200001-200412.nc", "Ambient Fine Aerosol Opitical Thickness at 550 nm")  #, "atmosphere_optical_thickness_due_to_pm1_ambient_aerosol")
print(c[0].standard_name)
None

it has, however, an

invalid_standard_name: atmosphere_optical_thickness_due_to_pm1_ambient_aerosol

and c[0].var_name is indeed od550lt1aer
Data issue, not iris issue 🍺

valeriupredoi · 2020-03-03T13:03:10Z

BTW where is the entry for od550lt1aer in perfmetrics recipe? Checked out latest master and it ain't there

mattiarighi · 2020-03-03T13:04:25Z

Line 1847

valeriupredoi · 2020-03-03T13:17:19Z

Line 1847

nevermind I'm blind 🙈

jvegreg · 2020-03-03T13:57:27Z

Data issue, not iris issue

Iris issue because it is a legal standard_name and should not be treated as invalid.

valeriupredoi · 2020-03-03T14:09:22Z

we battling in legalities @jvegasbsc 😁 is that invalid_standard_name attr set by iris or by the data constructor? Coz if it's the second then it's the constructor (cmorizer/ESGF) at fault

mattiarighi · 2020-03-03T14:10:47Z

But it used to work, were there changes in the fix files?

valeriupredoi · 2020-03-03T14:17:37Z

I am actually using iris=2.3 for this case so am saying - not iris the problemo 😁 Playing devil's advocate @bjlittle owes me a 🍺

mattiarighi · 2020-03-03T14:25:54Z

This means you could reproduce the issue?

valeriupredoi · 2020-03-03T14:26:54Z

yes, outside esmvaltool, see #1561 (comment)

jvegreg · 2020-03-03T14:40:23Z

we battling in legalities @jvegasbsc grin is that invalid_standard_name attr set by iris or by the data constructor? Coz if it's the second then it's the constructor (cmorizer/ESGF) at fault

Iris uses that attribute when it detects a standard_name that is not part of the valid list, so is Iris not the CMOR who is using it. In our case, this is wrong because it is a valid standard_name, search it here http://cfconventions.org/Data/cf-standard-names/71/build/cf-standard-name-table.html.

We can also reproduce the error without any file:

>>> import numpy as np
>>> import iris.cube
>>> cube = iris.cube.Cube(np.arange(10))
>>> cube.standard_name = 'atmosphere_optical_thickness_due_to_pm1_ambient_aerosol'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/Earth/jvegas/.conda/envs/esmvaltool/lib/python3.6/site-packages/iris/_cube_coord_common.py", line 206, in standard_name
    self._standard_name = get_valid_standard_name(name)
  File "/home/Earth/jvegas/.conda/envs/esmvaltool/lib/python3.6/site-packages/iris/_cube_coord_common.py", line 58, in get_valid_standard_name
    name))
ValueError: 'atmosphere_optical_thickness_due_to_pm1_ambient_aerosol' is not a valid standard_name
>>> cube.standard_name = 'atmosphere_optical_thickness_due_to_pm1_ambient_aerosol_particles'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/Earth/jvegas/.conda/envs/esmvaltool/lib/python3.6/site-packages/iris/_cube_coord_common.py", line 206, in standard_name
    self._standard_name = get_valid_standard_name(name)
  File "/home/Earth/jvegas/.conda/envs/esmvaltool/lib/python3.6/site-packages/iris/_cube_coord_common.py", line 58, in get_valid_standard_name
    name))
ValueError: 'atmosphere_optical_thickness_due_to_pm1_ambient_aerosol_particles' is not a valid standard_name
>>> cube.standard_name = 'latitude'                                                         
>>>

valeriupredoi · 2020-03-03T16:15:28Z

yep, same here, assigning it to the GFDL-CM3 cube:

Traceback (most recent call last):
  File "test_aer.py", line 7, in <module>
    cc.standard_name = "atmosphere_optical_thickness_due_to_pm1_ambient_aerosol"
  File "/home/users/valeriu/anaconda3R/envs/esmvaltool/lib/python3.7/site-packages/iris/_cube_coord_common.py", line 206, in standard_name
    self._standard_name = get_valid_standard_name(name)
  File "/home/users/valeriu/anaconda3R/envs/esmvaltool/lib/python3.7/site-packages/iris/_cube_coord_common.py", line 58, in get_valid_standard_name
    name))
ValueError: 'atmosphere_optical_thickness_due_to_pm1_ambient_aerosol' is not a valid standard_name

@jvegasbsc is correct and I have to retract my support for iris in this matter 🍺

valeriupredoi · 2020-03-03T16:20:15Z

funny thing is that name is actually present in STD_NAMES dict in /anaconda3R/envs/esmvaltool/lib/python3.7/site-packages/iris/std_names.py - also, why is that a fixed object (STD_NAMES) and not a collection from the CF library, @bjlittle ? 🍺

valeriupredoi · 2020-03-03T16:33:15Z

ok the problem is in iris/_cube_coord_common.py l/47:

        valid_name_pattern = re.compile(r'''^([a-zA-Z_]+)( *)([a-zA-Z_]*)$''')
        name_groups = valid_name_pattern.match(name)

name_groups is returned as None for name_groups = valid_name_pattern.match("atmosphere_optical_thickness_due_to_pm1_ambient_aerosol")
and even if it is a valid name it's discarded right away

valeriupredoi · 2020-03-03T16:46:32Z

SciTools/iris#3677

valeriupredoi · 2020-03-05T11:27:19Z

@jvegasbsc have a look at SciTools/iris#3677 (comment) and then my cheeky comment SciTools/iris#3677 (comment) - do you mind taking care of it? 🍺

jvegreg · 2020-03-05T11:53:44Z

@jvegasbsc have a look at SciTools/iris#3677 (comment) and then my cheeky comment SciTools/iris#3677 (comment) - do you mind taking care of it?

Setting an Iris development environment right now...

mattiarighi · 2020-05-22T10:24:00Z

Any news here?

bjlittle · 2020-06-03T15:02:44Z

@mattiarighi Just merged @jvegasbsc PR SciTools/iris#3679 on iris to resolve this issue here.

Thanks Javier 🍻

This change will be included in the forthcoming iris v3.0.0 release, which is Python3 only.

We're aiming to get v3.0.0 out this summer... when I know firmer dates I'll be sure to let you guys know. Alternatively just ping me (I don't mind) and ask if you need an ETA.

Cheers 😄

mattiarighi · 2020-06-03T15:05:27Z

Thanks @bjlittle

mattiarighi · 2020-06-24T11:56:15Z

Remove from the milestone, since we will not get Iris 3 before ESMValTool v2.0 release.

bouweandela · 2021-10-11T07:30:23Z

I think this problem has been fixed by now. Is any work on this still needed?

valeriupredoi · 2021-10-25T15:23:44Z

@remi-kazeroni have you seen this happen in @bouweandela's last run of all the recipes from yesterday?

remi-kazeroni · 2021-10-27T11:50:50Z

The recipe_perfmetrics_CMIP5.yml was not fully ran in #2354 since the automoatic download didn't work due to the unavailable ESGF node (see comment).

I have made a few attempts to run the recipe enabling the automatic download feature:

1st attempt: unavailable CESM1-CAM5-1-FV2 for tas neither on at DKRZ nor on ESGF nodes
2nd attempt: downloaded missing 5.9 GB of missing data and started to run tasks sequentially. Failed due to the MMM preprocessor (see main_log_debug.txt:
main_log_debug_mmm.txt)
3rd attempt: uncommenting the recipe bits for od550lt1aer that is still commented out "until Iris 3.0 becomes available" and commenting other variables out. The problem mentionned in this issue is actually solved but the recipe crashed because of a diagnostic error related to #1372, see main_log_debut.txt:
main_log_debug_od550lt1aer.txt

So I guess we could uncomment the "Needs Iris3.0" bit to close this issue. But I'm not sure what to do with the error related to the mmm.

zklaus · 2021-10-28T08:13:19Z

@remi-kazeroni, ESMValGroup/ESMValCore#1372 has been resolved. Could you try again? Don't forget to re-create the environment to get the second release candidate of the core.

remi-kazeroni · 2021-10-28T09:11:02Z

@remi-kazeroni, ESMValGroup/ESMValCore#1372 has been resolved. Could you try again? Don't forget to re-create the environment to get the second release candidate of the core.

I have just recreated an environment with ESMValCore 2.4.0rc2 and rerun the od550lt1aer bit of the recipe_perfmetrics_CMIP5.yml succesfully. I can make a PR to uncomment the "Needs Iris3" section of the recipe_perfmetrics_CMIP5.yml in order to close this PR. In this PR, we could also uncomment this line of the examples/recipe_check_obs.yml related to od550lt1aer. I have just tested that succesfully with the latest environment.

zklaus · 2021-10-28T09:57:18Z

Sounds good!

mattiarighi added the bug label Mar 3, 2020

mattiarighi assigned jvegreg, valeriupredoi and bouweandela Mar 3, 2020

jvegreg mentioned this issue Mar 6, 2020

PI-3677: Remove regex from standard names check SciTools/iris#3679

Merged

mattiarighi mentioned this issue Mar 10, 2020

Implement three levels of CMOR checks strictness ESMValGroup/ESMValCore#374

Merged

9 tasks

mattiarighi added this to the v2.0.0 milestone Jun 5, 2020

mattiarighi mentioned this issue Jun 9, 2020

Workaround for permetrics recipe until Iris3 #1674

Merged

mattiarighi removed this from the v2.0.0 milestone Jun 24, 2020

TomasTorsvik mentioned this issue Jul 2, 2020

Problem running recipe_ocean_bgc.yml #1695

Closed

mattiarighi mentioned this issue Jul 12, 2020

Exclude od550lt1aer from recipe_check_obs.yml #1720

Merged

bouweandela added this to the v2.4.0 milestone Oct 11, 2021

remi-kazeroni mentioned this issue Oct 28, 2021

Enabling variable obs550lt1aer in recipes #2388

Merged

8 tasks

zklaus closed this as completed in #2388 Oct 28, 2021

recipe_perfmetrices crashes due to standard name error #1561

recipe_perfmetrices crashes due to standard name error #1561

Comments

mattiarighi commented Mar 3, 2020

valeriupredoi commented Mar 3, 2020

mattiarighi commented Mar 3, 2020

jvegreg commented Mar 3, 2020

mattiarighi commented Mar 3, 2020

mattiarighi commented Mar 3, 2020

jvegreg commented Mar 3, 2020

mattiarighi commented Mar 3, 2020

jvegreg commented Mar 3, 2020

valeriupredoi commented Mar 3, 2020

valeriupredoi commented Mar 3, 2020

mattiarighi commented Mar 3, 2020

valeriupredoi commented Mar 3, 2020

jvegreg commented Mar 3, 2020

valeriupredoi commented Mar 3, 2020

mattiarighi commented Mar 3, 2020

valeriupredoi commented Mar 3, 2020

mattiarighi commented Mar 3, 2020

valeriupredoi commented Mar 3, 2020

jvegreg commented Mar 3, 2020 • edited Loading

valeriupredoi commented Mar 3, 2020 • edited Loading

valeriupredoi commented Mar 3, 2020

valeriupredoi commented Mar 3, 2020

valeriupredoi commented Mar 3, 2020

valeriupredoi commented Mar 5, 2020

jvegreg commented Mar 5, 2020

mattiarighi commented May 22, 2020

bjlittle commented Jun 3, 2020 • edited Loading

mattiarighi commented Jun 3, 2020

mattiarighi commented Jun 24, 2020

bouweandela commented Oct 11, 2021

valeriupredoi commented Oct 25, 2021

remi-kazeroni commented Oct 27, 2021

zklaus commented Oct 28, 2021

remi-kazeroni commented Oct 28, 2021

zklaus commented Oct 28, 2021

jvegreg commented Mar 3, 2020 •

edited

Loading

valeriupredoi commented Mar 3, 2020 •

edited

Loading

bjlittle commented Jun 3, 2020 •

edited

Loading