Allow exemptions to coordinate axis guessing on save

[bayliffe]

Within the iris NetCDF saver there is a call to guess_coord_axis. This simple utility relies on coordinate names, units, or other attributes to ascribe an axis identifier to each dimension coordinate where possible. One of these guesses checks whether the units of the coordinate are convertible to hectopascals, and if so presumes the dimension is a pressure coordinate corresponding to vertical levels; an axis identifier of Z is added.

Working with probabilistic data in which a dimension coordinate may describe different pressure thresholds, i.e. probability of pressure at mean sea level exceeding 1000hPa, 1010hPa, 1020hPa etc. this results in the threshold coordinate being identified as a vertical spatial dimension with an axis identifier of Z. This is undesirable and causes issues downstream of where the data is created (e.g. in Visual Weather). As this is added on save it is not possible to remove the attribute without using an alternative tool to Iris.

This issue requests that there be a means within Iris of preventing the axis identifiers being added to specific named (or var named) coordinates on save.

Example cube

Below is shown a simple print out of a probabilistic PMSL forecast with a threshold coordinate named air_pressure_at_sea_level that has units of Pa, which is not a height coordinate, but which is assigned a Z axis identifier.

probability_of_air_pressure_at_sea_level_above_threshold / (1) (air_pressure_at_sea_level: 85; projection_y_coordinate: 970; projection_x_coordinate: 1042)
     Dimension coordinates:
          air_pressure_at_sea_level                                                      x                            -                             -
          projection_y_coordinate                                                        -                            x                             -
          projection_x_coordinate                                                        -                            -                             x
     Scalar coordinates:
          ...

ncdump header information corresponding to this coordinate:

float threshold(threshold) ;
	threshold:axis = "Z" ;
	threshold:units = "Pa" ;
	threshold:standard_name = "air_pressure_at_sea_level" ;

[bjlittle]

@bayliffe Thanks for raising this issue, much appreciated. We think this is an issue that needs addressed 👍

Are you happy to take this issue on yourself with some hand holding?

If you don't have time just let us know and we can move this issue forwards for you.

[bjlittle]

After a wee bit of developer discussion your issues raises questions how the best mechanism to best control the behaviour that you want... so this may require more thought on our behalf.

[bjlittle]

@bayliffe Hmmm there seems to be a few ways to approach this problem in a generic useful way, and TBH this is going to need some considered thought (sorry, no avoiding that).

At this point, I suggest that you up-vote your own issue (and get other like minded users/devs to do so too 😉) to make sure that the priority of this issue bubbles to the top of tasks that require to be addressed by @SciTools/iris-devs 👍

Hope that's okay for now?

[tjtg]

I've upvoted as requested - this issue affects our usage of Iris-produced NetCDF with downstream systems (including Visual Weather) at BOM.

I'd argue that this issue should be labelled as a bug rather than an enhancement.
Metadata for probabilistic pressures will always be marked (incorrectly) as a vertical/Z axis. I'm also not aware of any way to work around this issue on the user side, besides editing the saved NetCDF file with a non-Iris tool like netcdf4-python or NCO.

[rcomer]

That’s good insight @tjtg. I added the “feature” label, as any mechanism to opt out of current behaviour would be a new feature. However, it never makes sense to think that “air_pressure_at_sea_level” could describe height. If we could change guess_coord_axis to produce a sensible return value for this coordinate, then we would not need an opt-out mechanism.

[ESadek-MO]

Discussed in Peloton, added to 3.5 for time being.

[trexfeathers]

Had a design discussion with @HGWright recently.

None of the below options are without compromise. The current preferred option is a new DimCoord member: space_time.

Tagging all voters: @rcomer @bjlittle @pp-mo @bayliffe @neilCrosswaite @bjwheltor @s-boardman @BelligerG @MoseleyS @PaulAbernethy @simonjackson900 @tjtg @anja-bom @nivnac @thbom001 @btrotta-bom @dmentipl @ivorblockley @benowen-bom @cpelley @gavinevans

Adjust guess_coord_axis()

Don't use HPa for z-guessing

• ➕: avoids all mistakes like this
• ➖: makes Iris less active in adding CF-compliant information
• ➖: likely break existing workflows that are currently fine

Exclude particular names from z-guessing

Suggestion is to exclude anything that includes *pressure_at*.

• ➕: keeps utility of current z-guessing, but with fewer mistakes
• ➖: more difficult to understand/predict
• ➖: hard to be exhaustive with the exclusion list
  Even likely to change over time
• ➖: very reliant on outside expertise for this, and each user has different cases that they need to work

Adjust saving

Don't alter an existing axis attribute

Intention is that the user will set an appropriate attribute before saving e.g. "NA".

• ➕: a simple addition with few breakages likely
• ➖: requires manual user intervention
• ➖: don't know how downstream readers might interpret this (only T, Z, Y, X are covered in CF)
• ➖: difficult to inform users that they should do this - no obvious place in the docs

A new kwarg in the NetCDF saver

• ➕: a visible feature, easy to spot in the documentation
• ➕: non-breaking addition
• ➖: requires manual user intervention
• ➖: the saving API is already too busy
• ➖: this is pretty specific for the top-level saver, and sets a precedent for adding lots of other 'modes'

A new DimCoord member: space_time

Boolean, default None. Set to True when coord has an axis attribute of T/Z/Y/X. Set to False if a different axis attribute value is present.

Axis guessing during save is not performed if my_coord.space_time is False (since the guessing was only ever intended to add the spatiotemporal axis labels).

Intention is that the user will set my_coord.space_time=False before saving.

• ➕: a visible feature, easy to spot in the documentation
• ➕: precedent in the circular and climatological flags - information to determine Iris behaviour
• ➕: potential opportunities for future use in distinguishing experimental repeats, pertubations etc.
• ➕: non-breaking addition
• ➖: requires manual user intervention
• ➖: fairly significant API addition

[trexfeathers]

Team conclusion:

A Coord property: guess_axis. Defaults to True. User can set to False to 'opt-out'. This means that this coordinate always gets the result None when run through guess_coord_axis().

Using a property provides a space for documenting it, too.

We know this affects NetCDF save, plotting, merging, maybe regridding.

[trexfeathers]

Not currently planning a context manager, but can be persuaded 😉🎂

[bayliffe]

@HGWright @trexfeathers @bjlittle thank you for your work on this. I'm afraid in the turmoil of operational releases, the progress made on this issue completely passed me by at the end of last year. Your efforts are much appreciated and will help us to avoid spurious behaviour in downstream systems ingesting probabilistic pressure data. ❤️