Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add automatic downscaling when encountering OOM error #51

Merged
merged 13 commits into from
Mar 7, 2024

Conversation

RaczeQ
Copy link
Collaborator

@RaczeQ RaczeQ commented Mar 6, 2024

Closes #50

Grouping nodes into ways is the most memory-intensive operation for the whole algorithm. Currently, it's more or less tuned based on the system's available memory. This change will catch OOM errors, downscale the process and retry the operation.

Copy link

codecov bot commented Mar 6, 2024

Codecov Report

Attention: Patch coverage is 84.03361% with 19 lines in your changes are missing coverage. Please review.

Project coverage is 94.39%. Comparing base (8bcd4e3) to head (9d31906).

Files Patch % Lines
quackosm/pbf_file_reader.py 86.08% 16 Missing ⚠️
quackosm/_rich_progress.py 33.33% 2 Missing ⚠️
quackosm/_osm_tags_filters.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #51      +/-   ##
==========================================
+ Coverage   88.44%   94.39%   +5.94%     
==========================================
  Files          17       17              
  Lines        1212     1267      +55     
==========================================
+ Hits         1072     1196     +124     
+ Misses        140       71      -69     
Flag Coverage Δ
macos-latest-python3.11 93.29% <74.78%> (+5.08%) ⬆️
ubuntu-latest-python3.10 87.52% <84.03%> (-0.68%) ⬇️
ubuntu-latest-python3.11 87.52% <84.03%> (-0.68%) ⬇️
ubuntu-latest-python3.9 87.52% <84.03%> (-0.68%) ⬇️
windows-latest-python3.11 87.76% <84.03%> (-0.69%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@RaczeQ RaczeQ merged commit 54037a8 into main Mar 7, 2024
9 of 10 checks passed
@RaczeQ RaczeQ deleted the oom-error-grouping-filtered-ways branch March 7, 2024 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

OutOfMemoryException: Out of Memory Error
1 participant