Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes nonlocal reducers in which the first list is empty. #1378

Merged
merged 2 commits into from
Mar 22, 2022

Conversation

jpivarski
Copy link
Member

This error was reported in #1371, and the fix is in the kernel:

   int64_t j = 0;
   int64_t k = 0;
-  int64_t maxdistinct = -1;
+  int64_t maxdistinct = -2;
   int64_t lasti = -1;
   for (int64_t i = 0;  i < lendistincts;  i++) {
     if (maxdistinct < distincts[i]) {

The initial value of maxdistinct is supposed to be different from all possible values so that the first item is identified as a distinct list, but empty lists are represented in the distincts array as -1. Setting the initial value to a smaller number allows it to trigger on the first list, even if that list is empty.

This bug goes way back: in the first implementation of this kernel (Feb 15 2020)

https://github.com/scikit-hep/awkward-1.0/blob/b4ea752b641e1162e6e3ac9b2778ac544a52d5ff/src/cpu-kernels/reducers.cpp#L462-L466

and the studies that preceded it

https://github.com/scikit-hep/awkward-1.0/blob/fd94fd6978e06795f2a36aefa8b1bc2a7772387e/studies/reducers.py#L1014-L1018

Thanks, @masonproffitt!

@jpivarski jpivarski linked an issue Mar 22, 2022 that may be closed by this pull request
@jpivarski jpivarski enabled auto-merge (squash) March 22, 2022 19:51
@codecov
Copy link

codecov bot commented Mar 22, 2022

Codecov Report

Merging #1378 (3a0045f) into main (b2fd2be) will decrease coverage by 0.97%.
The diff coverage is 52.14%.

Impacted Files Coverage Δ
src/awkward/_v2/_connect/cling.py 0.00% <0.00%> (ø)
src/awkward/_v2/_connect/cuda/__init__.py 0.00% <0.00%> (ø)
src/awkward/_v2/_lookup.py 97.50% <0.00%> (ø)
src/awkward/_v2/_prettyprint.py 66.09% <0.00%> (+2.29%) ⬆️
src/awkward/_v2/_typetracer.py 69.14% <0.00%> (ø)
src/awkward/_v2/identifier.py 55.69% <0.00%> (ø)
src/awkward/_v2/operations/convert/ak_from_jax.py 75.00% <0.00%> (ø)
src/awkward/_v2/operations/convert/ak_to_jax.py 75.00% <0.00%> (ø)
src/awkward/_v2/operations/io/ak_from_parquet.py 75.00% <0.00%> (ø)
src/awkward/_v2/operations/io/ak_to_parquet.py 75.00% <0.00%> (ø)
... and 145 more

@jpivarski jpivarski merged commit e55c914 into main Mar 22, 2022
@jpivarski jpivarski deleted the jpivarski/nonlocal-reducer-with-first-list-empty branch March 22, 2022 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Strange and inconsistent ordering behavior with ak.sum
1 participant