Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add slurm_memory to crew_controller_slurm #44

Closed
multimeric opened this issue Jul 22, 2024 · 1 comment
Closed

Add slurm_memory to crew_controller_slurm #44

multimeric opened this issue Jul 22, 2024 · 1 comment
Assignees

Comments

@multimeric
Copy link

Proposal

Currently we only have slurm_memory_gigabytes_per_cpu for controlling the Slurm memory allocation. Unfortunately, I don't think --mem-per-cpu is the most intuitive argument to be configuring. For one, Slurm might decide to give you double the amount of memory that you requested because of hyperthreading (see discussion here). Secondly, multithreading isn't enormously popular in R, especially when we already have parallelism provided by targets. Instead, I prefer --mem which just simply sets the total memory per job.

Of course, you can currently configure this using:

    script_lines = c(
      "#SBATCH --mem 500G",
      ...
    )

However, making it a first-class argument would be even more user friendly.

@wlandau
Copy link
Owner

wlandau commented Jul 22, 2024

That's a good idea, implemented in 248d6e2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants