Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

small change to attempt to stop tempfile() problems in larger documents. #2293

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

abelew
Copy link

@abelew abelew commented Sep 22, 2023

Greetings,
When knitting larger documents with many images I sometimes get tempfile errors saying it ran out of files. When I looked more closely, it seemed to me that tempfile() was just not trying very hard; so I hacked a quick md5 tempfile generator which in theory takes all the same arguments as base::tempfile(). I was thinking to make it more robust and use some actual pseudorandomly generated material, but this seems to have worked fine for all of my largest and most troublesome documents.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Owner

@yihui yihui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish to understand the problem before fixing it. I don't see the limitation mentioned on the help page ?tempfile. I tested a million temp files and it seems to work fine for me:

for (i in 1:1e6) {
  f = tempfile(); file.create(f); file.remove(f)
}

Do you have more than a million plots in your document?

> xfun::session_info()
R version 4.3.1 (2023-06-16)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.5.2, RStudio 2023.9.1.494

Locale: en_US.UTF-8 / en_US.UTF-8 / en_US.UTF-8 / C / en_US.UTF-8 / en_US.UTF-8

Package version:
  compiler_4.3.1    graphics_4.3.1    grDevices_4.3.1   rstudioapi_0.15.0 stats_4.3.1       tools_4.3.1       utils_4.3.1      
  xfun_0.40.2 

@abelew
Copy link
Author

abelew commented Nov 20, 2023

I think I replied via email, but the following is an example more similar to what happens in my documents and shows that you only need ~ 30 plots (at least in my hands) to hit a duplicate tmpnam:

library(parallel)
library(doParallel)
library(iterators)
library(foreach)
iterators <- 10
returns <- c()
## I haven't used parallel in a while, I can't remember if it is like make -j and I can't remember if returns or res gets the result back...
cl <- parallel::makeCluster(iterators + 1)
registered <- doParallel::registerDoParallel(cl)
res <- foreach(c = 1:(iterators * 10)) %dopar% {
  filename <- tempfile()
  returns <- c(returns, filename)
}
stopped <- parallel::stopCluster(cl)
table(duplicated(unlist(res)))

@yihui
Copy link
Owner

yihui commented Dec 6, 2023

Sorry but I don't quite understand the code since I'm not familiar with these packages. In particular, I don't understand what returns <- c(returns, filename) is trying to do. sapply(res, duplicated) is all FALSE, and I don't know what duplicated(unlist(res)) implies.

Anyway, it will be clearer if you can show a minimal reproducible example that actually uses knitr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants