Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C++ refactoring: layout.completely_flatten should not concatenate (performance issue). #1245

Merged
merged 1 commit into from
Jan 21, 2022

Conversation

jpivarski
Copy link
Member

This was raised on Slack by @martindurant and @douglasdavis.

Before:

In [1]: import awkward._v2 as ak, numpy as np

In [2]: x = np.ones((100, 100, 100))

In [3]: arr = ak.Array(x)

In [4]: %timeit ak.sum(arr, axis=None)
813 µs ± 6.08 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [5]: %timeit x.sum()
332 µs ± 5.38 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

After:

In [1]: import awkward._v2 as ak, numpy as np

In [2]: x = np.ones((100, 100, 100))

In [3]: arr = ak.Array(x)

In [4]: %timeit ak.sum(arr, axis=None)
317 µs ± 20.4 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [5]: %timeit x.sum()
295 µs ± 8.65 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

It removes an unnecessary concatenation.

@codecov
Copy link

codecov bot commented Jan 21, 2022

Codecov Report

Merging #1245 (1e45797) into main (ab65e20) will increase coverage by 0.25%.
The diff coverage is 66.97%.

Impacted Files Coverage Δ
src/awkward/_v2/_util.py 82.24% <ø> (ø)
src/awkward/_v2/contents/recordarray.py 80.60% <ø> (+0.86%) ⬆️
src/awkward/_v2/operations/reducers/ak_argmin.py 55.00% <0.00%> (-13.75%) ⬇️
src/awkward/_v2/operations/describe/ak_type.py 45.71% <38.70%> (-34.29%) ⬇️
src/awkward/_v2/operations/reducers/ak_all.py 87.50% <80.00%> (-4.17%) ⬇️
src/awkward/_v2/operations/reducers/ak_any.py 87.50% <80.00%> (-4.17%) ⬇️
src/awkward/_v2/operations/reducers/ak_count.py 87.50% <80.00%> (-4.17%) ⬇️
...wkward/_v2/operations/reducers/ak_count_nonzero.py 87.50% <80.00%> (-4.17%) ⬇️
src/awkward/_v2/operations/reducers/ak_prod.py 87.50% <80.00%> (-4.17%) ⬇️
src/awkward/_v2/operations/reducers/ak_sum.py 93.75% <80.00%> (-6.25%) ⬇️
... and 22 more

@jpivarski jpivarski merged commit b204131 into main Jan 21, 2022
@jpivarski jpivarski deleted the jpivarski/completely_flatten-should-not-concatenate branch January 21, 2022 20:00
@jpivarski jpivarski changed the title layout.completely_flatten should not concatenate (performance issue). C++ refactoring: layout.completely_flatten should not concatenate (performance issue). Feb 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant