tar_make error with Rmd files: Error in eval(x, envir = envir) : object 'params' not found

[yonicd]

Prework

• Read and agree to the code of conduct and contributing guidelines.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• For any problems you identify, post a minimal reproducible example so the maintainer can troubleshoot. A reproducible example is:
  • Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • Readable: format your code according to the tidyverse style guide.

Question

When I have a tar_bind of pipelines and one of them has a parameterized Rmd file and run tar_visnetwork(targets_only = TRUE) or tar_make() i get this error in the console but the output is created regardless.

r$> library(targets)
r$> tar_visnetwork(targets_only = TRUE)
Error in eval(x, envir = envir) : object 'params' not found

It looks like targets is not finding the object that is created via the yaml header in the rmd and generating this error, is there any way to suppress this message since it doesnt really effect the output?

thanks

Reproducible example

• For any problems you identify, post a minimal reproducible example so the maintainer can troubleshoot. A reproducible example is:
  • Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • Readable: format your code according to the tidyverse style guide.

[wlandau]

Hard to say why this would be happening. Would you post a reproducible example so I can troubleshoot? A traceback would also help.

tar_manifest(callr_function = NULL)
traceback()

[yonicd]

library(targets)

tar_dir({

lines <- c(
  "---",
  "title: '`r params$paramcd`'",
  "output: html_document",
  "params:",
  "  paramcd: TRUE",
  "---",
  "",
  "```{r, eval = params$paramcd}",
  "print(params$paramcd)",
  "print(object)",
  "```"
)

writeLines(lines, "source.Rmd")

  tar_script({
    library(tarchetypes)
    
    pipe1 <- tar_pipeline(
      tar_target(domains,c("domain1", "domain2"))
    )
    
    pipe2 <- tar_pipeline(
       tarchetypes::tar_render(report,"source.Rmd")
    )
    
    targets::tar_bind(pipe1)
  })

  tar_manifest(callr_function = NULL)
traceback()
})
#> Error in eval(x, envir = envir) : object 'params' not found
#> No traceback available

^{Created on 2021-01-04 by the reprex package (v0.3.0)}

[wlandau]

Thanks. The problem seems to come from knitr::knit(tangle = TRUE), which tar_render() needs in order to detect dependencies.

lines <- c(
  "---",
  "title: '`r params$paramcd`'",
  "output: html_document",
  "params:",
  "  paramcd: TRUE",
  "---",
  "",
  "```{r, eval = params$paramcd}",
  "print(params$paramcd)",
  "print(object)",
  "```"
)
writeLines(lines, "report.Rmd")
knitr::knit("report.Rmd", tangle = TRUE, quiet = TRUE)
#> Error in eval(x, envir = envir) : object 'params' not found
#> [1] "report.R"

^{Created on 2021-01-04 by the reprex package (v0.3.0)}

I think the problem comes from the fact that this example report uses an R Markdown parameter inside the metadata for the code chunk: {r, eval = params$paramcd}. I am not sure if knitr allows this. When I set eval to TRUE, the error message disappears.

lines <- c(
  "---",
  "title: '`r params$paramcd`'",
  "output: html_document",
  "params:",
  "  paramcd: TRUE",
  "---",
  "",
  "```{r, eval = TRUE}",
  "print(params$paramcd)",
  "print(object)",
  "```"
)
writeLines(lines, "report.Rmd")
knitr::knit("report.Rmd", tangle = TRUE, quiet = TRUE)
#> [1] "report.R"

^{Created on 2021-01-04 by the reprex package (v0.3.0)}

Related: I actually just deprecated user-side pipeline objects: #254. Now, you can just end _targets.R with a list of target objects. For multiple sub-pipelines, you can just use c() or list() instead of tar_bind(). I believe this is much simpler and easier.

[wlandau]

Well, it looks like you actually can use variables for code chunk options: https://bookdown.org/yihui/rmarkdown-cookbook/chunk-variable.html. So maybe submit a bug report to knitr?

[yonicd]

Thanks for checking.

It also seems to pop up from this angle too REditorSupport/vscode-R#504

@cderv where would be a good place to start an issue for this problem?

[cderv]

Yes a variable can be pass in a chunk option. The issue here is with parametrized report.
The issue is with knitr which currently does not know about params object whereas rmarkdown makes it available to chunks...

lines <- c(
  "---",
  "title: knitr with params",
  "output: html_document",
  "params:",
  "  paramcd: TRUE",
  "---",
  "",
  "```{r, eval = TRUE}",
  "str(params)",
  "```"
)
writeLines(lines, "report.Rmd")
res <- knitr::knit("report.Rmd", quiet = TRUE)
xfun::file_string(res)
#> ---
#> title: knitr with params
#> output: html_document
#> params:
#>   paramcd: TRUE
#> ---
#> 
#> ```r
#> str(params)
#> #> Error in str(params): object 'params' not found
#> ```

^{Created on 2021-01-05 by the reprex package (v0.3.0.9001)}

I would open an issue in knitr to investigate this, I am not sure if this is a bug or a missing feature (for knitr alone to be aware of parameters in YAML).

This indeed become an issue with tools like target as you rely on knitr only to transform a .Rmd file into a .R file using knitr. With the .Rmd file using parameters, it seems it does not work.

The problem seems to come from knitr::knit(tangle = TRUE), which tar_render() needs in order to detect dependencies.

For my understanding, why the need of purling the Rmd file ?

Maybe there is a missing piece in R Markdown to provide a tooling for a specific need where knitr::purl is used for currently.

[cderv]

I would open an issue in knitr to investigate this, I am not sure if this is a bug or a missing feature (for knitr alone to be aware of parameters in YAML).

What is supported though is using tangle = TRUE with a Rmd containing params. As you can see in the report.R script, params are parsed and this line is added params <- list(paramcd = TRUE) so that the script will work if sourced.

However, there is an issue in knitr when tangle = TRUE and params is used in chunk option. Thanks for the reprex @wlandau !

[wlandau]

For my understanding, why the need of purling the Rmd file?

Because some R Markdown reports depend on upstream targets computed earlier in the pipeline. Users register these dependency targets with tar_read() or tar_load() in code chunks, and targets needs some way to statically detect them. tarchetypes::tar_render() automatically scans the report and produces a target with the correct dependency relationships. This section walks through that functionality.

[wlandau]

So in the case of targets, we just want to extract the language objects in the code chunks. From that point of view, it does not matter which variables are undefined.

[yonicd]

@cderv Can purl/tangle preserve the yaml header?

[cderv]

@wlandau so you need to find tar_read() and tar_load() in the code chunks of the Rmd and for that you purl to an R script and then use parse()

It is some times now that I think we miss a tool to directly extract the R code chunks to do some advanced stuff in other tools. Today, everything rely on purl. there are some internal functions that does that but they can't be used on their own for now. I am wondering if it could help for this type of usage.

Thanks for sharing.

Can purl/tangle preserve the yaml header?

@yonicd You can keep all the text in the Rmd as roxygen comments in the R script

lines <- c(
  "---",
  "title: knitr with params",
  "output: html_document",
  "params:",
  "  paramcd: TRUE",
  "---",
  "",
  "```{r, eval = TRUE}",
  "str(params)",
  "```"
)
writeLines(lines, "report.Rmd")
res <- knitr::purl("report.Rmd", documentation = 2, quiet = TRUE)
xfun::file_string(res)
#> params <-
#> list(paramcd = TRUE)
#> 
#> #' ---
#> #' title: knitr with params
#> #' output: html_document
#> #' params:
#> #'   paramcd: TRUE
#> #' ---
#> #' 
#> ## ---- eval = TRUE-------------------------------------------------------------
#> str(params)

^{Created on 2021-01-05 by the reprex package (v0.3.0.9001)}

Obviously it must be in comments so that you have a valid R script at a result.

[wlandau]

@wlandau so you need to find tar_read() and tar_load() in the code chunks of the Rmd and for that you purl to an R script and then use parse()

Yes, exactly.

It is some times now that I think we miss a tool to directly extract the R code chunks to do some advanced stuff in other tools. Today, everything rely on purl. there are some internal functions that does that but they can't be used on their own for now. I am wondering if it could help for this type of usage.

I would love that!