-
Notifications
You must be signed in to change notification settings - Fork 283
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Performing metadata equality via different methods provides different results in Iris v3.0.1 #4506
Comments
@ehogan Under the hood for metadata The fact that you can put scalar This is why Perhaps it might be more helpful to the user if the You might reasonably ask why oh why do we care about such a difference? ... one concrete example is for cube arithmetic when the metadata for two operand cubes are combined into one resultant cube. Given your example above, which version of Note that, I can't imagine that we'll subclass a At the very least this behaviour should be highlighted and clarified... but do note that you are highlighting a special case here of using a >>> import numpy as np
>>> a = dict(values=np.arange(3))
>>> b = dict(values=[0, 1, 2])
>>> a == b
a == b
Traceback (most recent call last):
File "<input>", line 1, in <module>
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() Also, note that metadata |
And here is view from someone that all to frequently (to say the least...) have to deal with all sorts of metadata (and data) imperfections. Model data, that is supposed to be one homogeneous set, that was produced in different batches, on different machines, quality controlled by different people, using different [versions of] software tools, and good know what is the reason behind the imperfections. In the best of worlds such problems should be non-existing or data just rejected/fixed by the producers. But I am afraid that I do not live in that world. Last year we processed something like 10,000-15,000 files from a large online database, and something like 60% had metadata (or data) imperfections. Most of these were easy to spot and could be fixed with a few steps of preprocessing. Remained something like 500 files making up about 30-50 supposedly homogeneous dataset units. Homogeneous in the sense that So, what conclusions to to draw from this with any bearings on iris?
I think that steps in this direction will be immensely useful all and everyone that are dealing with data from various sources. |
@larsbarring have you looked at cube_helper? I've not used it myself but think it was written for CMIP ensembles. I think we're all agreed that Iris's own |
Thanks @bjlittle :) I'm happy with the discrimination between
could the output be:
or similar? Also, would it be worth adding to the documentation that |
In order to maintain a backlog of relevant issues, we automatically label them as stale after 500 days of inactivity. If this issue is still important to you, then please comment on this issue and the stale label will be removed. Otherwise this issue will be automatically closed in 28 days time. |
This issue is still important to me 😊 |
🐛 Bug Report
I have two cubes. When I use
cube1.metadata.difference(cube2.metadata)
my metadata shows differences, even though the values look the same. However, when I useassert cube1.attributes == cube2.attributes
I get noAssertionError
.I discovered this issue because:
scitools/default-previous
) I can runassert cube1.metadata == cube2.metadata
and I get noAssertionError
.scitools/default-current
) I get anAssertionError
when I runassert cube1.metadata == cube2.metadata
.How To Reproduce
Expected behaviour
I'm not sure what would be best?
Either
cube1.metadata.difference(cube2.metadata)
shows the types in the difference, so it's clear where the difference is (because the values look the same and this took longer than I care to admit to figure out) 😝Or
assert cube1.attributes == cube2.attributes
failsOr both?
The text was updated successfully, but these errors were encountered: