-
Notifications
You must be signed in to change notification settings - Fork 89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: replace axis_wrap_if_negative
with maybe_posaxis
, simpler and more correct
#1986
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…d implement ak.is_none.
Codecov Report
Additional details and impacted files
|
…eliminate the weird axis=0 output for records.
…ler and correct for deep unions.
agoose77
approved these changes
Dec 9, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I can't believe it:
axis_wrap_if_negative
has always been wrong. This function dates back to the early days of Awkward 1.x. We've been using it for so long, in so many things, that I would have thought that it's been through every possible test.To convert a negative
axis
into its non-negative equivalent, you have to know the absolute depth of the array. Withdepth == 1
for a 1-dimensional array and increasing for each nested list, the non-negative axis isaxis + depth
whereaxis
is negative. For instance, ifaxis == -1
(deepest) anddepth == 2
(array of lists of primitives),axis + depth == 1
: the equivalent non-negativeaxis == 1
.If there's branching (a union or a record), then there isn't a single depth; we can't evaluate
axis + depth
. However, as we recurse down, we'll eventually recurse through the branching structure. Inside of a record of{x: var * float64, y: var * var * int32}
, for instance, we can evaluate onedepth
inside thex
branch and a differentdepth
inside they
branch. We want to do that so thataxis=-1
means "deepest for all branches," i.e. thefloat64
forx
and theint32
fory
.That's why
axis_wrap_if_negative
passesaxis
through unchanged if (a) it's already non-negative or (b) thelayout
it's looking at is branched. The idea is that we'll repeatedly callaxis_wrap_if_negative
while the value we get back is negative. As soon as it's non-negative, we keep that value.However, if we're recursing, the
layout
that we have at some level is not the whole layout. If we only have thex
branch of a record, for instance, we'll see itsvar * float64
as "layout
", and ifx
's record were embedded within other lists, we wouldn't see them. The depth we compute form that "layout
" is therefore relative to the depth of last branching (the record withx
andy
, in this example).Since
axis_wrap_if_negative
takes this partial "layout
" and the negativeaxis
as arguments, it can never determine the absolute depth, and therefore cannot correctly convert theaxis
into a non-negativeaxis
... in general.There's only one case that works anyway: if the branching happens to be at the top level, the relative depth is equal to the absolute depth and we get the right answer. I've looked around, and I've only ever seen tests of negative
axis
with non-branching arrays (no records or unions) or the branching is a RecordArray at top level (this special case). There are tests in which different fields have different depths, likebut never within another list, like
The examples in #1914 were apparently the first general test of negative to non-negative
axis
conversion.So I started this PR by introducing a new function (which will eventually live in
ak._util
, but right now it's inak._do
to be next toaxis_wrap_if_negative
for juxtaposition), which does the conversion correctly. It needs the same arguments asaxis_wrap_if_negative
and also the currentdepth
, to correct for the fact that thelayout
it's seeing is not the whole array.I've implemented
ak.is_none
with it. Now I'm going to look for other functions that were usingaxis_wrap_if_negative
incorrectly. I don't think we'll fix all of these bugs before the 2.0.0 release tomorrow, but we'll get the pattern established.They're bugs and we can change them without a deprecation cycle.
📚 The documentation for this PR will be available at https://awkward-array.readthedocs.io/en/jpivarski-improved-axis-to-posaxis/ once Read the Docs has finished building 🔨