Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OOM error when calculating distance matrix #97

Open
PatPotsdam opened this issue Jul 31, 2024 · 0 comments
Open

OOM error when calculating distance matrix #97

PatPotsdam opened this issue Jul 31, 2024 · 0 comments

Comments

@PatPotsdam
Copy link

Dear astral-Team,

Thanks for this great software package.

Describe the bug
I am trying to run a moderately large dataset with ~1500 gene trees and ~140 species using the astral-MP. Surprisingly, I am not able to run it on my university's HPC cluster. I run out of memory even with 60 threads and the option -Xmx1300g (the maximum possible on my cluster). I thought this option would set a memory limit for astral and it just runs longer but actually it stops with OOM error when calculating distance matrix.

To Reproduce
java -Xmx1300g -D"java.library.path=/path/to/Astral.5.15.5/lib/" -jar /path/to/Astral.5.15.5/astral.5.15.5.jar -T 60 -i genes.treefile -o genes_astral.tre -t 2 2>genes_astral.log

Log file
last 4 lines of the log before it stopped:
...
Using polytree-based weight calculation.
Number of gene trees: 15079
15079 trees have missing taxa
Calculating distance matrix (for completion of X) ....with 4 distance matrices for parallellism

slurm error file:
slurmstepd: error: Detected 1 oom_kill event in StepId=702545.batch. Some of the step tasks have been OOM Killed.

** Version
5.15.5

Thanks in advance and all the best
Patrick

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant