Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process finished with exit code 137 (interrupted by signal 9: SIGKILL) #32

Open
zzyycc7 opened this issue Jun 20, 2023 · 4 comments
Open

Comments

@zzyycc7
Copy link

zzyycc7 commented Jun 20, 2023

Hi,
Thank you for sharing a great work!
I met the error when I ran the script python suds/train.py suds --experiment-name test --pipeline.datamanager.dataparser.metadata_path /mnt/kitti/metadata/kitti-06.json
截图 2023-06-20 16-23-50

@hturki
Copy link
Owner

hturki commented Jun 20, 2023

That might be due to memory issues, can you monitor that your memory usage isn't getting to the state the OOM killer is terminating your process? There is an --pipeline.datamanager.items_per_chunk parameter that you can reduce is memory is a concern, and if you're using DINO features I'd recommend double-checking that the --pipeline.datamanger.load_on_demand option includes features

@zzyycc7
Copy link
Author

zzyycc7 commented Jun 21, 2023

Thank you for your reply. I use watch free command to monitor the memory usage. It is true that the code cannot be run due to insufficient memory. But even if I set items_per_chunk to a very small value, the same problem occurs. I also check that load_on_demand includes features.
截图 2023-06-21 10-21-06

@hturki
Copy link
Owner

hturki commented Jun 22, 2023

how much memory do you have available on the machine?

@zzyycc7
Copy link
Author

zzyycc7 commented Jun 23, 2023

64G

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants