-
Notifications
You must be signed in to change notification settings - Fork 116
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error using summarize in ddply with .parallel=T ("'...' used in an incorrect context")) #204
Comments
The following works for me:
The The following alternative is safer:
You can make this shorter with the help of a helper function:
I'm not sure if it's at all possible to enhance The |
Is there an update to this? I am getting similar warnings on Windows 8.1 (Update 1) x64, R 3.1.0, plyr 1.8.1. |
Seems to work for me with this code: library(plyr)
dfx <- data.frame( group = c(rep('A', 8), rep('B', 15), rep('C', 6)),
sex = sample(c("M", "F"), size = 29, replace = TRUE),
age = runif(n = 29, min = 18, max = 54))
doParallel::registerDoParallel(cores = 3)
ddply(dfx, .(group, sex), .parallel=T, .fun=summarize,
mean = round(mean(age), 2), sd = round(sd(age), 2)) |
@hadley Is there any update or work around on this? When I run your example I get
R 3.2.2 |
For the records, I'm confirming there's a still a problem on R 3.3.0 on Windows (details below), or more precisely with SNOW clusters. Reproducible example (also on non-Windows machines)The reason is most likely that doParallel uses a multicore backend on Unix/OS X, but on Windows it falls back to using a SNOW cluster of local R sessions. For instance, on Linux you get: > doParallel::registerDoParallel(cores = 3)
> foreach::getDoParName()
[1] "doParallelMC" whereas on Windows you get: > doParallel::registerDoParallel(cores = 3)
> foreach::getDoParName()
[1] "doParallelSNOW" @hadley and other non-Windows users, to reproduce this SNOW-related issues reported here, use the following setup instead: > cl <- parallel::makeCluster(3)
> doParallel::registerDoParallel(cl)
> foreach::getDoParName()
[1] "doParallelSNOW" and you'll get the same error. Troubleshooting guessI can only guess, but I believe this has to do with how "global" variables are identified by foreach/SNOW and somehow it falls flat when it gets to Detailslibrary(plyr)
dfx <- data.frame( group = c(rep('A', 8), rep('B', 15), rep('C', 6)),
sex = sample(c("M", "F"), size = 29, replace = TRUE),
age = runif(n = 29, min = 18, max = 54))
## doParallel::registerDoParallel(cores = 3) on Windows gives:
cl <- parallel::makeCluster(3)
doParallel::registerDoParallel(cl)
ddply(dfx, .(group, sex), .parallel=T, .fun=summarize,
mean = round(mean(age), 2), sd = round(sd(age), 2))
Error in do.ply(i) : task 1 failed - "'...' used in an incorrect context"
In addition: Warning messages:
1: <anonymous>: ... may be used in an incorrect context: '.fun(piece, ...)'
2: <anonymous>: ... may be used in an incorrect context: '.fun(piece, ...)'
> sessionInfo()
R version 3.3.0 Patched (2016-05-03 r70575)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] doParallel_1.0.10 iterators_1.0.8 foreach_1.4.3 plyr_1.8.3
loaded via a namespace (and not attached):
[1] compiler_3.3.0 Rcpp_0.12.4.5 codetools_0.2-14 |
First of all, thanks for the amazing
plyr
package! I have looked around in the old issues and google, but could not find a solution to my problem. I suspect it might be a bug.Basically, when I use
ddply
with the.fun=summarize
function, it fails when used together with.parallel=T
.I run windows 7 x64, R 3.0.1, plyr 1.8.1.
To reproduce with one of your plyr examples:
Adding a
.parallel=T
option throws the error:The error does not allow me to proceed, unlike the warnings mentioned in #203 .
Any help warmly appreciated, let me know if I can test anything further.
The text was updated successfully, but these errors were encountered: