Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

run_first_last_multi_partitions fails in some environments #8069

Closed
ghost opened this issue Nov 6, 2023 · 2 comments · Fixed by #8072
Closed

run_first_last_multi_partitions fails in some environments #8069

ghost opened this issue Nov 6, 2023 · 2 comments · Fixed by #8072
Labels
bug Something isn't working

Comments

@ghost
Copy link

ghost commented Nov 6, 2023

Describe the bug

run_first_last_multi_partitions fails when I run cargo test on my M1 Macbook Air with 16 GB RAM. It passes on my Ubuntu workstation with 128 GB RAM.

---- aggregates::tests::run_first_last_multi_partitions stdout ----
Error: ResourcesExhausted("Failed to allocate additional 632 bytes for GroupedHashAggregateStream[0] with 1829 bytes already allocated - maximum available is 605")

To Reproduce

No response

Expected behavior

No response

Additional context

No response
@ghost ghost added the bug Something isn't working label Nov 6, 2023
@alamb
Copy link
Contributor

alamb commented Nov 6, 2023

Perhaps this is somehow related to the updates in f1f8d79 to support spilling group by hash: #7400

https://github.com/apache/arrow-datafusion/blame/308c35404939ed39e4e398054d6fc78c89817109/datafusion/physical-plan/src/aggregates/mod.rs#L1891-L1899

@alamb
Copy link
Contributor

alamb commented Nov 6, 2023

I ran this on my macbook (x86) that has 64GB of ram

Maybe the failure is related to ARM vs Intel 🤔

@viirya viirya mentioned this issue Nov 7, 2023
Merged
@andygrove andygrove mentioned this issue Nov 7, 2023
Closed
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use FairSpillPool for TaskContext with spillable config viirya/arrow-datafusion
1 participant
@alamb