[JOSS review] [bug?] Submitting only one lapply job fails

[mschubert]

This is a part of the JOSS review outlined in openjournals/joss-reviews#1493.

The following fails:

library(sluRm)
Slurm_lapply(1, function(x) x*2)
# Error in splits[[j]] : subscript out of bounds
# In addition: Warning message:
# `X` is not a list. The function will coerce it into one using `as.list`

Related: Internal parallelism via mc.cores defaults to 2, which wastes a core if there are not more calls than jobs.

[gvegayon]

Thanks. This should be fixed now. Regarding the default of 2 mc.cores, that's not an issue since mclapply automatically checks for this. If length(X) == 1, then mc.cores defaults to lapply instead. I was just trying to be consistent with what is in parallel.

[gvegayon]

Also, this was a nasty bug. Thanks for flagging that!

[mschubert]

Regarding the default of 2 mc.cores, that's not an issue since mclapply automatically checks for this. If length(X) == 1, then mc.cores defaults to lapply instead.

That's true for mclapply in R, but not for the job that requests 2 cores (and only uses 1)

[gvegayon]

OK, good point. Nevertheless, I'm honestly not sure regarding this as a problem, since in most cases using Slurm_lapply/Slurm_Map/Slurm_sapply users will be submitting a job with length(X) >> 1. I do need to get more details about what happens behind scenes when you request more than you are using, and also regarding how much enforced is the request of resources (in the manual of sbatch, the "Advice the controller" phrase is a bit confusing for me). Either way, I follow your advice and will set the default to equal 1L. [ ] Set `mc.cores = 1L` in `Slurm_lapply`/`Map` George G. Vega Yon +1 (626) 381 8171 https://ggvy.cl

…

On Tue, Jun 11, 2019 at 10:06 AM Michael Schubert ***@***.***> wrote: Regarding the default of 2 mc.cores, that's not an issue since mclapply automatically checks for this. If length(X) == 1, then mc.cores defaults to lapply instead. That's true for mclapply in R, but not for the job that requests 2 cores (and only uses 1) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#12>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAG2FM3OZJA2YUIBNYMT32DPZ7LSHANCNFSM4HWUHN4Q> .

[gvegayon]

Now in 91909a7