File open fails when lon and lat are variables (not dimensions) #32
Comments
|
when did you create your conda env? I uploaded an updated cdms2 last night. |
|
@doutriaux1 here we go: |
|
please do a force |
|
@doutriaux1 same issue, it's a problem with cdms: (uvcdat280) bash-3.2$ ipython
Python 2.7.12 | packaged by conda-forge | (default, Sep 8 2016, 14:41:48)
Type "copyright", "credits" or "license" for more information.
IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import cdms2 as cdm
In [2]: infile = '161122_RobertPincus_multiple_input4MIPs_radiation_RFMIP_UColorado-RFMIP-20161122_none.nc'
In [3]: f = cdm.open(infile)
---------------------------------------------------------------------------
CDMSError Traceback (most recent call last)
<ipython-input-3-da3bd29babf2> in <module>()
----> 1 f = cdm.open(infile)
/uvcdat280/lib/python2.7/site-packages/cdms2/dataset.pyc in openDataset(uri, mode, template, dods, dpath, hostObj)
357
358 # The file exists
--> 359 file1 = CdmsFile(path, "r")
360 if libcf is not None:
361 if hasattr(file1, libcf.CF_FILETYPE):
/uvcdat280/lib/python2.7/site-packages/cdms2/dataset.pyc in __init__(self, path, mode, hostObj, mpiBarrier)
1137 # Get grid information for the variable. gridkey has the form
1138 # (latname,lonname,order,maskname, abstract_class).
-> 1139 gridkey, lat, lon = var.generateGridkey(self._convention_, self.variables)
1140
1141 # If the variable is gridded, lookup the grid. If no such grid exists,
/uvcdat280/lib/python2.7/site-packages/cdms2/avariable.pyc in generateGridkey(self, convention, vardict)
239 lon, nlon = convention.getVarLonId(self, vardict)
240 if (lat is not None) and (lat is lon):
--> 241 raise CDMSError, "Axis %s is both a latitude and longitude axis! Check standard_name and/or axis attributes."%lat.id
242
243 # Check for 2D grid
CDMSError: Axis lon is both a latitude and longitude axis! Check standard_name and/or axis attributes. |
|
@doutriaux1 curiously, in a previous (and not identical) version of the file, if the @dnadeau4 pinging you into this thread |
|
I will enhanced CDMS2 to make sure your lat/lon are detected correctly |
|
@dnadeau4 out of curiosity, I tried to update the file to include And the error: (uvcdatNightly) duri@ocean:[RFMIP]:[2858]> ipython
Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul 2 2016, 17:42:40)
Type "copyright", "credits" or "license" for more information.
IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import cdms2 as cdm
In [2]: infile = 'test.nc'
In [3]: f = cdm.open(infile)
---------------------------------------------------------------------------
CDMSError Traceback (most recent call last)
<ipython-input-3-da3bd29babf2> in <module>()
----> 1 f = cdm.open(infile)
/uvcdatNightly/lib/python2.7/site-packages/cdms2/dataset.pyc in openDataset(uri, mode, template, dods, dpath, hostObj)
307
308 # The file exists
--> 309 file1 = CdmsFile(path,"r")
310 if libcf is not None:
311 if hasattr(file1, libcf.CF_FILETYPE):
/uvcdatNightly/lib/python2.7/site-packages/cdms2/dataset.pyc in __init__(self, path, mode, hostObj, mpiBarrier)
1063 # Get grid information for the variable. gridkey has the form
1064 # (latname,lonname,order,maskname, abstract_class).
-> 1065 gridkey, lat, lon = var.generateGridkey(self._convention_, self.variables)
1066
1067 # If the variable is gridded, lookup the grid. If no such grid exists,
/uvcdatNightly/lib/python2.7/site-packages/cdms2/avariable.pyc in generateGridkey(self, convention, vardict)
236 lon, nlon = convention.getVarLonId(self, vardict)
237 if (lat is not None) and (lat is lon):
--> 238 raise CDMSError, "Axis %s is both a latitude and longitude axis! Check standard_name and/or axis attributes."%lat.id
239
240 # Check for 2D grid
CDMSError: Axis lon is both a latitude and longitude axis! Check standard_name and/or axis attributes. |
|
@dnadeau4 if you amend the And for completeness the ncdump of the top of the file: |
|
@dnadeau4 just wondering where this issue is sitting? We're hoping to get @RobertPincus's data published in the input4MIPs project, however the publisher/cdms can't deal with his data currently.. |
|
Nothing yet, I am working on issue #7. @doutriaux1 gave me a priority list check label #priority and talk with @doutriaux1. |
|
@sashakames with #76 it seems that @dnadeau4 has solved this issue and so this version of bug-fixed |
|
@durack1 this is now working. The issue was the wrong units for latitude/longitude. I used nco to change them and now cdms2 works. ncatted -O -h -a units,lon,o,c,degree_east ~/Downloads/161122_RobertPincus_multiple_input4MIPs_radiation_RFMIP_UColorado-RFMIP-20161122_none.nc
ncatted -O -h -a units,lat,o,c,degree_north ~/Downloads/161122_RobertPincus_multiple_input4MIPs_radiation_RFMIP_UColorado-RFMIP-20161122_none.nc |
|
@dnadeau4 the issue wasn't incorrect units, these were changed to prevent the CF checker (which uses a much earlier version of "chat-lite") from crashing.. I confirmed the same behaviour in the 2.8 |
|
@dnadeau4 if you see the original ncdump output, these are the same as your |
|
@durack1 I was able to work with these files when I changed the units. Why is longitudes degrees_north in the original ncdump? |
|
|
|
longitude needs to be set to |
|
Once ESGF has CDMS imported via a CONDA python, we will have this and any other cdms2 features. Targeting the next major release --we have one this/next week but too soon for inclusion now.
…-Sasha
From: "Paul J. Durack" <[email protected]>
Reply-To: UV-CDAT/cdms <[email protected]>
Date: Sunday, January 15, 2017 at 1:38 PM
To: UV-CDAT/cdms <[email protected]>
Cc: Sasha Ames <[email protected]>, Mention <[email protected]>
Subject: Re: [UV-CDAT/cdms] File open fails when lon and lat are variables (not dimensions) (#32)
@sashakames<https://github.com/sashakames> with #76<#76> it seems that @dnadeau4<https://github.com/dnadeau4> has solved this issue and so this version of bug-fixed cdms will need to be pulled across so the ESGF publisher can open and publish the file (attached above). Once this PR is merged, can we prioritise this bug fix patch into ESGF?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#32 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AHf2-5HQJux9qSIMouYeGhyF2_bN3KO_ks5rSpHggaJpZM4K5v2A>.
|
|
dnadeau4 ok now I get you.. So the unit values were flipped I wonder if there would be a way, in the case of a failure testing the units against the possible values and returning a "units invalid" error, rather than ungracefully exiting as it does. The fact that |
|
@dnadeau4 you might want to create a user error tag, and assign it here.. I know @doutriaux1 likes that label alot |
|
@durack1 see, as I 'm getting older I become weaker (and wiser?) and I'm willing to listen to users even when they are caught in a flagrant |
|
That's good @doutriaux1, because our new users not only do not read the docs, but they tend to assume that things will work as they expect (regardless of what the functions are actually supposed to do). They must believe in magic and fairy tales |
|
@doutriaux1 that |
|
@durack1 agreed the software should point this out to the user, I am personally |
|
I guess I could try to use "standard_name" if "units" is not right for variables lat/lon and warn the user. If there is an attribute axis='Y' then we are done! To pass the test, the variable must start wit 'lat' (mixed cases are fine). def isLatitude(self):
id = self.id.strip().lower()
if (hasattr(self,'axis') and self.axis=='Y'): return 1
units = getattr(self,"units","").strip().lower()
if units in ["degrees_north","degree_north","degree_n","degrees_n","degreen","degreesn"]:
return 1
return (id[0:3] == 'lat') or (id in latitude_aliases) |
|
@dnadeau4 there is always the range check approach too, so for a valid The preferred behaviour to me would be for these |
|
range depends on the units though. But it's probably worth adding for known units. |
|
we can't enforce a valid range for longitudes though |
|
@doutriaux1 why no valid range for longitudes? These values have to sit within the range -180 to 360 degrees. That deals with the -180 to 180 grids and the 0 to 360 grids too, what other ranges are required? |
|
@doutriaux1 Can CDMS read other projections like Mercator? If so, I cannot check the range the @durack1 asked. |
|
@dnadeau4 as i stated range depends on units, I think mercator and such will have units in |
The attached file is unreadable in the current version of cdms2 (v2.8). I believe the issue is that the variables (not dimensions)
lonandlatincluded in the file are assumed to be standard dimensions (which they aren't), and these assumptions lead to a read failure.cdmsReadFail.zip
The file looks like:
And the read failure looks like:
The text was updated successfully, but these errors were encountered: