Creating Dynamic Documents with RMarkdown and Knitr

By: Marian L. Schmidt, @micro_marian, marschmi at umich.edu
May 11th, 2016

Welcome!

This workshop was hosted on May 11th, 2016 and has 2 supplemental materials available:

Class notes can be found on the workshop etherpad.
This workshop was recorded live and is available on YouTube. Please be welcome to tune in on YouTube!

This tutorial was constructed as a part of Dr. C Titus Brown's Data Intensive Biology (DIB) training program at the University of California, Davis. The DIB training program hosts local + remote workshops covering topics in bioinformatic tools and data analysis.

There were three remote classrooms that tuned in:

University of California, Davis
Simon Fraser University
Ontario Institute for Cancer Research

The Github repository for this lesson can be found here.

Installation Instructions

Before working through the workshop materials, please do the following in preparation:

Open up RStudio.
Install and download the devtools R package by running the following in the R console.

install.packages("devtools") # To include the session information
library("devtools") # To include the session information

Check that you have the correct versions of R and RStudio by running devtools::session_info() in the R console.
Here, devtools:: tells R to use the session_info() function within the devtools package rather than the sessionInfo() function within the utils package. Running devtools::session_info() allows us to see the version of R and RStudio.

Do you have the following versions of R and RStudio?

R: Version 3.3.0 (2016-05-03)
RStudio: 0.99.1172
- If you do then you are good to go!
- If you do not have the correct versions of R and RStudio, please follow the Setup directions on our course page.

Install other R packages necessary for the workshop.

## Install the correct packages
install.packages("rmarkdown") # Make cool dynamic documents
install.packages("knitr") # Run R Code Chunks
install.packages("ggplot2") # For plotting
install.packages("DT") # Interactive HTML tables

## Add these packages to the current session to make sure they load correctly
library("rmarkdown") # Make cool dynamic documents
library("knitr") # Run R Code Chunks
library("ggplot2") # For plotting
library("DT") # Interactive HTML tables

If your packages loaded without any errors, then you are ready for the workshop!

If there were issues with the above packages please place a pink sticky to get help from a helper!

Amazing Resources

I could not have made this tutorial without these amazing resources:

The RMarkdown website hosted by RStudio.
Dr. Yuhui Xie's book: Dynamic Documents with R and Knitr 2^nd^ Edition [@Xie2015] and his Knitr website.
- A BIG thank you to Dr. Xie for writing the Knitr Package!!
Dr. Karl Broman's "Knitr in a Knutshell".
Cheatsheets released by RStudio.

Dynamic Documents

Literate programming is the basic idea behind dynamic documents and was proposed by Donald Knuth in 1984. Originally, it was for mixing the source code and documentation of software development together. Today, we will create dynamic documents in which program or analysis code is run to produce output (e.g. tables, plots, models, etc) and then are explained through narrative writing.

The 3 steps of Literate Programming:

Parse the source document and separate code from narratives.
Execute source code and return results.
Mix results from the source code with the original narratives.

So that leaves 2 steps for us which includes writing:

Analysis code
A narrative to explain the results from the analysis code.

Traditionally, people used comments to include narrative in their code file (for R that would be a .R file). For example, the file may include the following:

# Title:  Relationship of Car Weight and Gas Efficiency 
# By:  Marian Schmidt  
# Date:  May 11th, 2016

# I predict that there will be a relationship between weight and miles per gallon.  
# I will test this by running a linear model with the mtcars dataset is within the R datasets

# What does the data look like?
datatable(mtcars) # Interactive table 

# Is there a relationship between the weight of a car and the miles per gallon?
lm_mpg <- lm(mpg ~ wt, data = mtcars) # Run linear model predicting mpg based on wt
coef_lm_mpg <- coef(summary(lm_mpg)) # Extract coefficients to table 
kable(coef_lm_mpg) # Produce non-interactive table - function in knitr

# Plot the relationship between weight and miles per gallon  
plot <- ggplot(mtcars, aes(x = wt, y = mpg)) + geom_point() + # 
  geom_smooth(method = "lm") + theme_bw() + # add linear model and make black and white
  xlab("Weight (1000lbs)") + ylab("Miles per Gallon") # Add axis labels
ggplotly(plot) # Make the plot interactive  

# It appears that with a gain in every 1000 pounds that there will be a decrease in fuel efficiency by 5.34 miles per gallon
# The end

Then the user would have to read the comments and run the code themselves.

However, literate programming allows us to run the code and describe the code all within one document that we could share. So for example we could do the following:

Relationship of Car Weight and Gas Efficiency
By: Marian Schmidt
Date: May 11th, 2016

I predict that there will be a relationship between weight and miles per gallon.

I will test this by running a linear model with the mtcars dataset is within the R data sets.

# What does the data look like?
datatable(mtcars) # Interactive table

\u003c/th>\n

mpg\u003c/th>\n

cyl\u003c/th>\n

disp\u003c/th>\n

hp\u003c/th>\n

drat\u003c/th>\n

wt\u003c/th>\n

qsec\u003c/th>\n

vs\u003c/th>\n

am\u003c/th>\n

gear\u003c/th>\n

carb\u003c/th>\n \u003c/tr>\n \u003c/thead>\n\u003c/table>","options":{"columnDefs":[{"className":"dt-right","targets":[1,2,3,4,5,6,7,8,9,10,11]},{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false},"callback":null,"filter":"none"},"evals":[],"jsHooks":[]}</script>

# Is there a relationship between the weight of a car and the miles per gallon?
lm_mpg <- lm(mpg ~ wt, data = mtcars) # Run linear model predicting mpg based on wt
coef_lm_mpg <- coef(summary(lm_mpg)) # Extract coefficients to table 
kable(coef_lm_mpg) # Produce non-interactive table - function in knitr

            Estimate   Std. Error     t value   Pr(>|t|)

(Intercept) 37.285126 1.877627 19.857575 0 wt -5.344472 0.559101 -9.559044 0

# Plot the relationship between weight and miles per gallon  
plot <- ggplot(mtcars, aes(x = wt, y = mpg)) + geom_point() + # 
  geom_smooth(method = "lm") + theme_bw() + # add linear model and make black and white
  xlab("Weight (1000lbs)") + ylab("Miles per Gallon") # Add axis labels
ggplotly(plot) # Make the plot interactive

It appears that with a gain in every 1000 pounds that there will be a decrease in fuel efficiency by 5.3444716 miles per gallon

The end

If we use literate programming, we could also:

Tangle: Extract the source code out of the document.
Weave: Execute the code to get the compiled results.

Reproducible Research

Reproducible research is one possible product of dynamic documents, however, it is not guaranteed! Good practices for reproducible research include:

Encapsulate the full project into one directory that is supported with version control.
Release your code and data.
Document everything and use code as documentation!
Make figures, tables, and statistics the results of scripts and inline code.
Write code that uses relative paths.
Always Set your seed.
Always include session information in the code file. For example, you can use devtools::session_info().

To read more about reproducibility and data management check out Vince Buffalo's Book[@Buffalo2015].

Markdown

To fully understand RMarkdown, we first need to cover Markdown, which is a system for writing simple, readable text that is easily converted to HTML. Markdown essentially is two things:

A plain text formatting syntax
A software tool written in Perl.
- Converts the plain text formatting into HTML.

Main goal of Markdown:
Make the syntax of the raw (pre-HTML) document as readable possible.

Would you rather read this code in HTML?

<body>
  <section>
    <h1>Rock Climbing Packing List</h1>
    <ul>
      <li>Climbing Shoes</li>
      <li>Harness</li>
      <li>Backpack</li>
      <li>Rope</li>
      <li>Belayer</li>
    </ul>
  </section>
</body>

Or this code in Markdown?

# Rock Climbing Packing List

* Climbing Shoes
* Harness
* Backpack  
* Rope
* Belayer

If you are human, the Markdown code is definitely easier to read!

We will talk more about the syntax of Markdown after we introduce RMarkdown but let us take a moment to soak in how much easier our lives are/will be because Markdown exists! Thank you John Gruber and Aaron Swartz (RIP) for creating Markdown in 2004!

RMarkdown

RMarkdown is a variant of Markdown that makes it easy to create dynamic documents, presentations and reports within RStudio. It has embedded R code chunks to be used with knitr to make it easy to create reproducible (web-based) reports in the sense that they can be automatically regenerated when the underlying code it modified.

RMarkdown lets you combine Markdown with images, links, tables, LaTeX, and actual code.
RStudio makes creating documents from RMarkdown easy
RStudio (like R) is free and runs on any operating system.

RMarkdown renders many different types of files including:

HTML
PDF
Markdown
Microsoft Word
Presentations:
- Fancy HTML5 presentations:
  - ioslides
  - Slidy
  - Slidify
- PDF Presentations:
  - Beamer
- Handouts:
  - Tufte Handouts
HTML R Package Vignettes
Even Entire Websites!

While there are a lot of different types of rendered documents in RMarkdown, today we will focus primarily on HTML output files, as I have found these files to be the most useful and flexible for my research.

Why R Markdown?

A convenient tool for reproducible and dynamic reports!

While it was created for R, it now accepts many programming languages. For simplicity, we will only work with R today.
Execute code in a few ways:
1. Inline Code: Brief code that takes place during the written part of the document.
2. Code Chunks: Parts of the document that includes several lines of program or analysis code. It may render a plot or table, calculate summary statistics, load packages, etc.
It is easy to:
- Embed images.
- Learn Markdown syntax.
- Include LaTeX equations.
- Include interactive tables.
- Use version control with Git.
  - Even easier to share and collaborate on analyses, projects and publications!
- Add external links - Rmarkdown even understands some html code!
- Make beautifully formatted documents.
Do not need to worry about page breaks or figure placement.
Consolidate your code and write up into a single file:
- Slideshows, PDFs, html documents, word files

Simple Workflow

Briefly, to make a report:

Open a .Rmd file.
- Create a YAML header (more on this in a minute!)
Write the content with RMarkdown syntax.
Embed the R code in code chunks or inline code.
Render the document output.

Overview of the steps RMarkdown takes to get to the rendered document:

Create .Rmd report that includes R code chunks and and markdown narratives (as indicated in steps above.).
Give the .Rmd file to knitr to execute the R code chunks and create a new .md file.
- Knitr is a package within R that allows the integration of R code into rendered RMarkdown documents such as HTML, latex, pdf, word, among other document types.
Give the .md file to pandoc, which will create the final rendered document (e.g. html, Microsoft word, pdf, etc.).
- Pandoc is a universal document converter and enables the conversion of one document type (in this case: .Rmd) to another (in this case: HTML)

While this may seem complicated, we can hit the "Knit" button at the top of the page like this:

or we can run the following code:

rmarkdown::render("RMarkdown_Lesson.Rmd", "html_document")

Creating a `.Rmd` File

It's go time! Let's start working with RMarkdown!

In the menu bar, click File -> New File -> RMarkdown
- Or simply click on the green plus sign in the top left corner of RStudio.

The window below will pop up.

Inside of this window, choose the type of output by selecting the radio buttons. Note: this output can be easily changed later!

Click OK

YAML Headers

YAML stands for "YAML Ain't Markup Language" and is basically a nested list structure that includes the metadata of the document. It is enclosed between two lines of three dashes --- and as we saw above is automatically written by RStudio. A simple example:

---
title:  "Analysis Report"  
Author:  "Marian L. Schmidt"  
date: "May 11th, 2016"  
output:  html_document
---

The above example will create an HTML document. However, the following options are also available.

html_document
pdf_document
word_document
beamer_presentation (pdf slideshow)
ioslides_presentation (HTML slideshow)
and more...

Today, we will be focused on HTML files. However, please be welcome to play around with creating word and pdf documents. Presentation slides take on a slightly different syntax (e.g. to specify when one slide ends and the next one starts) and so there is a bit of markdown syntax specific to presentations that are beyond the focus of this workshop.

We will build on the details of YAML headers throughout the workshop.

Markdown Basics

Check out the RMarkdown Reference Guide

To list a few from the RMarkdown Cheatsheet:

Helpful Hints:

End a line with two spaces to start a new paragraph.
Words formatted like code should be surrounded by back ticks on both sides: `
To make something superscript surround it with ^ on each side. Super^script^ was created by typing Super^script^.
Equations can be inline code using $ and centered as a blocked equation within the document with $$. For example $E = mc^2$ is inline while the following is a blocked equation is: $$E = mc^2$$
- Note: To make it superscript with $ and $$ a ^ is needed before each alphanumeric that is superscript.
- Other fun math stuff:
  - Square root: $\sqrt{b}$ will create $\sqrt{b}$
  - Fractions: $\frac{1}{2}$ = $\frac{1}{2}$
    - - Fractional Equations: $f(x)=\frac{P(x)}{Q(x)}$ = $f(x)=\frac{P(x)}{Q(x)}$
  - Binomial Coefficients: $\binom{k}{n}$ = $\binom{k}{n}$
  - Integrals: $$\int_{a}^{b} x^2 dx$$ = $$\int_{a}^{b} x^2 dx$$
- ShareLaTeX is an awesome source for LaTeX code.

Some more mathy stuff:

Description	Code	Examples
Greek letters	$\alpha$ $\beta$ $\gamma$ $\rho$ $\sigma$ $\delta$ $\epsilon$ $mu$	$\alpha$ $\beta$ $\gamma$ $\rho$ $\sigma$ $\delta$ $\epsilon$ $\mu$
Binary operators	$\times$ $\otimes$ $\oplus$ $\cup$ $\cap$	$\times$ $\otimes$ $\oplus$ $\cup$ $\cap$ $\times$
Relation operators	$< >$ $\subset$ $\supset$ $\subseteq$ $\supseteq$	$< >$ $\subset$ $\supset$ $\subseteq$ $\supseteq$
Others	$\int$ $\oint$ $\sum$ $\prod$	$\int$ $\oint$ $\sum$ $\prod$

Challenge: Try to mimic the output of the following:

Today, I am bold and am learning RMarkdown.
honey is very sweet.
YAS!!!^!!!^
R^2^ values are informative!
$R^{2}$ describe the variance explained in the model.
~~I do not know RMarkdown~~ Today I learned RMarkdown
RStudio link
Output from the following:

# RMarkdown   
## R   
### Knitr   
#### Pandoc  
##### HTML

$\sqrt{b^2 - 4ac}$
$$\sqrt{b^2 - 4ac}$$
$X_{i,j}$

Today is the day I will make a dynamic document!

The following list:

Chocolate Chip cookie Recipe

butter
sugar
- A mix of brown & white sugar makes it more delicious
  - mix with butter before you add the eggs
eggs
vanilla
Mix dry ingredients:
- flour, salt, baking soda
chocolate chips

Fun Fact! The table of contents of this website was created from headers with 1-3 pound symbols! (More on this later)

Embed Code

There are 2 ways to embed code within an RMarkdown document.

Inline Code: Brief code that takes place during the written part of the document.
Code Chunks: Parts of the document that includes several lines of program or analysis code. It may render a plot or table, calculate summary statistics, load packages, etc.

Inline R Code

Inline code is created by using a back tick (`) and the letter r followed by another back tick.

For example: 2^11^ is 2048.

Imagine that you're reporting a p-value and you do not want to go back and add it every time the statistical test is re-run. Rather, the p-value is 0.0045.

This is really helpful when writing up the results section of a paper. For example, you may have ran a bunch of statistics for your scientific questions and this would be a way to have R save that value in a variable name.

For example: Is the gas mileage of automatic versus manual transmissions significantly different within the mtcars data set?

mpg_auto <- mtcars[mtcars$am == 0,]$mpg # automatic transmission mileage
mpg_manual <- mtcars[mtcars$am == 1,]$mpg # manual transmission mileage
transmission_ttest <- t.test(mpg_auto, mpg_manual)

To extract the p-value we can type transmission_ttest$p.value within inline code.

The p-value is 0.0013736.

R Code Chunks

R code chunks can be used to render R output into documents or to display code for illustration.

The Anatomy of a code chunk:

To insert an R code chunk, you can type it manually by typing ```{r} followed by ``` on the next line. You can also press the Insert a new code chunk button or use the shortcut key. This will produce the following code chunk:

`�``{r}
n <- 10
seq(n)
```

Name the code chunk something meaningful as to what it is doing. Below I have named the code chunk 10-random-numbers:

`�``{r 10-random-numbers}
n <- 10
seq(n)
```

The code chunk input and output is then displayed as follows:

n = 10
seq(n)

##  [1]  1  2  3  4  5  6  7  8  9 10

Knitr

Knitr is an R-Package that works with

Identifies code including chunks and inline
Evaluates all the code and returns the results
Renders a formatted results and combines with original file.

Knitr runs code as if it were being run in the R console.

Mainly Knitr works with code chunks.

A code chunk looks like:

```r
x <- rnorm(100)  
y <- 2*x + rnorm(100)
```

Best practices regarding code chunks:

Always name/label your code chunks!
Instead of specifying the chunk options in every chunk, set the global chunk options at the beginning of the document. More on this in a minute!

Chunk Labels

Chunk labels must be unique IDs in a document and are good for:

Generating external files such as images and cached documents.
Chunk labels often are output when errors arise (more often for line of code).
Navigating throughout long .Rmd documents.

When naming the code chunk: Use - or _ in between words for code chunks labels instead of spaces. This will help you and other users of your document to navigate through.

Chunk labels must be unique throughout the document - otherwise there will be an error!

Chunk Options

Pressing tab when inside the braces will bring up code chunk options.

results = "asis" stands for "as is" and will output a non-formatted version.
collapse is another chunk option which can be helpful. If a code chunk has many short R expressions with some output, you can collapse the output into a chunk.

There are too many chunk options to cover here. After the workshop take a look around at the options.

Great website for exploring Knitr Chunk Options.

Challenge
Run the code chunk below and play with the following knitr code chunk options:

eval = TRUE/FALSE

echo = TRUE/FALSE

collapse = TRUE/FALSE

results = "asis","markup, and "hide

In markdown, record your results.
Note: Be sure to name your chunks!

1+1
2*5
seq(1, 21, by = 3)
head(mtcars)

Some examples from the chunk above

Results from results="markup", collapse = TRUE}:

1+1
## [1] 2
2*5
## [1] 10
seq(1, 21, by = 3)
## [1]  1  4  7 10 13 16 19
head(mtcars)
##                    mpg cyl disp  hp drat    wt  qsec vs am gear carb
## Mazda RX4         21.0   6  160 110 3.90 2.620 16.46  0  1    4    4
## Mazda RX4 Wag     21.0   6  160 110 3.90 2.875 17.02  0  1    4    4
## Datsun 710        22.8   4  108  93 3.85 2.320 18.61  1  1    4    1
## Hornet 4 Drive    21.4   6  258 110 3.08 3.215 19.44  1  0    3    1
## Hornet Sportabout 18.7   8  360 175 3.15 3.440 17.02  0  0    3    2
## Valiant           18.1   6  225 105 2.76 3.460 20.22  1  0    3    1

Results from results="asis", collapse = TRUE}:

1+1

[1] 2

2*5

[1] 10

seq(1, 21, by = 3)

[1] 1 4 7 10 13 16 19

head(mtcars)

               mpg cyl disp  hp drat    wt  qsec vs am gear carb

Mazda RX4 21.0 6 160 110 3.90 2.620 16.46 0 1 4 4 Mazda RX4 Wag 21.0 6 160 110 3.90 2.875 17.02 0 1 4 4 Datsun 710 22.8 4 108 93 3.85 2.320 18.61 1 1 4 1 Hornet 4 Drive 21.4 6 258 110 3.08 3.215 19.44 1 0 3 1 Hornet Sportabout 18.7 8 360 175 3.15 3.440 17.02 0 0 3 2 Valiant 18.1 6 225 105 2.76 3.460 20.22 1 0 3 1

Global Options

You may wish to have the same chunk settings throughout your document and so it might be nice to type options once instead of always re-typing it for each chunk. To do so, you can set global chunk options at the top of the document.

knitr::opts_chunk$set(echo = FALSE, 
                      eval = TRUE, 
                      message = FALSE,
                      warning = FALSE, 
                      fig.path = "Figures/",
                      fig.width = 12, 
                      fig.height = 8)

For example, if you're working with a collaborator who does not want to see the code - you could set eval = TRUE and echo = FALSE so the code is evaluated but not shown. In addition, you may want to use message = FALSE and warning = FALSE so your collaborator does not see any messages or warnings from R.

If you would like to save and store figures within a sub directory within the project, fig.path = "Figures/". Here, the "Figures/" denotes a folder named Figures within the current directory where the figures produced within the document will be stored. Note: by default figures are not saved.

Global chunk options will be set for the rest of the document. If you would like to have a particular chunk be different from the global options, specify at the beginning of that particular chunk.

Figures

Knitr makes producing figures really easy. If analysis code within a chunk is supposed to produce a figure, it will just print out into the document.

Some knitr chunk options that relate to figures:

fig.width and fig.height
- Default: fig.width = 7, fig.height = 7
fig.align: How to align the figure
- Options include: "left", "right", and "center"
fig.path: A file path to the directory to where knitr should store the graphic output created by the chunk.
- Default: 'figure/'
There's even a fig.retina(only for HTML output) for higher figure resolution with retina displays.

Making a single figure:

With fig.align = "center"

ggplot(mtcars, aes(x = mpg)) + xlab("Miles per Gallon") +
    geom_histogram(bins = 30, fill = "cornflowerblue", color = "black") +
    geom_vline(xintercept=mean(mtcars$mpg), col="red")

With fig.align = "right"

ggplot(mtcars, aes(x = mpg)) + xlab("Miles per Gallon") +
    geom_histogram(bins = 30, fill = "cornflowerblue", color = "black") +
    geom_vline(xintercept=mean(mtcars$mpg), col="red")

With fig.align = "left"

ggplot(mtcars, aes(x = mpg)) + xlab("Miles per Gallon") +
    geom_histogram(bins = 30, fill = "cornflowerblue", color = "black") +
    geom_vline(xintercept=mean(mtcars$mpg), col="red")

With fig.width = 2, fig.height = 2

ggplot(mtcars, aes(x = mpg)) + xlab("Miles per Gallon") +
    geom_histogram(bins = 30, fill = "cornflowerblue", color = "black") +
    geom_vline(xintercept=mean(mtcars$mpg), col="red")

With fig.width = 10, fig.height = 10

ggplot(mtcars, aes(x = mpg)) + xlab("Miles per Gallon") +
    geom_histogram(bins = 30, fill = "cornflowerblue", color = "black") +
    geom_vline(xintercept=mean(mtcars$mpg), col="red")

myplots <- list()  # new empty list
for(i in 1:ncol(mtcars)){
  col <- names(mtcars)[i]
  ggp <- ggplot(mtcars, aes_string(x = col)) +
    geom_histogram(bins = 30, fill = "cornflowerblue", color = "black") +
    geom_vline(xintercept = mean(mtcars[[col]]), col = "red") 
  myplots[[i]] <- ggp  # add each plot into plot list
}
multiplot(plotlist = myplots, cols = 4) # must load in multiplot function from the Rcookbook see http://www.cookbook-r.com/Graphs/Multiple_graphs_on_one_page_(ggplot2)/

Tables

Hand writing tables in Markdown can get tedious. We will not go over this here, however, if you'd like to learn more about Markdown tables check out the documentation on tables at the RMarkdown v2 website.

There's a few types of tables that are useful. Here, we will use our previous example of a looking at the mtcars data

In his Knitr in a Knutshell, Dr. Karl Broman introduces: kable, pander, and xtable and I have especially enjoyed using the first two:

kable: Within the knitr package - not many options but looks nice with ease.
pander: Within the pander package - has many more options and customization. Useful for bold-ing certain values (e.g. values below a threshold).

kable and pander tables are nice as they are useful for making non-interactive tables:

kable(head(mtcars, n = 4)) # kable table with 4 rows

               mpg   cyl   disp    hp   drat      wt    qsec   vs   am   gear   carb

Mazda RX4 21.0 6 160 110 3.90 2.620 16.46 0 1 4 4 Mazda RX4 Wag 21.0 6 160 110 3.90 2.875 17.02 0 1 4 4 Datsun 710 22.8 4 108 93 3.85 2.320 18.61 1 1 4 1 Hornet 4 Drive 21.4 6 258 110 3.08 3.215 19.44 1 0 3 1

# Pander table
# install.packages("pander") # install pander first
library(pander)
pander(head(mtcars, n = 4))

   &nbsp;         mpg   cyl   disp   hp   drat   wt    qsec   vs   am   gear   carb

Mazda RX4 21 6 160 110 3.9 2.62 16.46 0 1 4 4

Mazda RX4 Wag 21 6 160 110 3.9 2.875 17.02 0 1 4 4

Datsun 710 22.8 4 108 93 3.85 2.32 18.61 1 1 4 1

Hornet 4 Drive 21.4 6 258 110 3.08 3.215 19.44 1 0 3 1

HTML Widgets

With the release of the new RMarkdown v2 it is now easier than ever to include HTML Widgets. Follow the link to explore widgets you're interested in!

Recently, I discovered the DT package which makes the tables interactive in the HTML output. In addition, Plotly for R produces really beautiful interactive graphics based on Plotly.

Cool, huh?

# DT table = interactive
# install.packages("DT") # install DT first
library(DT)
datatable(head(mtcars, n = nrow(mtcars)), options = list(pageLength = 5))

\u003c/th>\n

mpg\u003c/th>\n

cyl\u003c/th>\n

disp\u003c/th>\n

hp\u003c/th>\n

drat\u003c/th>\n

wt\u003c/th>\n

qsec\u003c/th>\n

vs\u003c/th>\n

am\u003c/th>\n

gear\u003c/th>\n

carb\u003c/th>\n \u003c/tr>\n \u003c/thead>\n\u003c/table>","options":{"pageLength":5,"columnDefs":[{"className":"dt-right","targets":[1,2,3,4,5,6,7,8,9,10,11]},{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[5,10,25,50,100]},"callback":null,"filter":"none"},"evals":[],"jsHooks":[]}</script>

# plotly
# install.packages("plotly")
library(plotly)
mtcars$car <- row.names(mtcars)
plot_ly(mtcars, x = wt, y = mpg, 
        text = paste("Car: ", car),
        mode = "markers", color = wt, size = wt)

Spell Check

Spelling may not be our strong suit and as a result, we may need to check the spelling in our document. There are two ways to check the spelling:

Click on the "ABC check mark" to the left of the magnifying glass button in RStudio.
Use the aspell() function from the utils package. However, it would be desirable to skip code chunks. The aspell() function can take a filter function to skip certain lines in the files and can be used with the function knit_filter() which was designed to skip over code chunks in a file.

Knitr Themes

The knitr syntax theme can be adjusted or completely customized. If you do not prefer the default themes, use the object knit_theme to change it. There are 80 themes contained within knitr and we can view the names of them by knit_theme$get().

What are the first 30 knitr themes?

head(knit_theme$get(), 30)

##  [1] "acid"          "aiseered"      "andes"         "anotherdark"  
##  [5] "autumn"        "baycomb"       "bclear"        "biogoo"       
##  [9] "bipolar"       "blacknblue"    "bluegreen"     "breeze"       
## [13] "bright"        "camo"          "candy"         "clarity"      
## [17] "dante"         "darkblue"      "darkbone"      "darkness"     
## [21] "darkslategray" "darkspectrum"  "default"       "denim"        
## [25] "dusk"          "earendel"      "easter"        "edit-anjuta"  
## [29] "edit-eclipse"  "edit-emacs"

We can use knit_theme$set() to set the theme. For example, to set the theme to fruit we could run the following code:

knit_theme$set("fruit")

Here's the link to find your favorite theme of all the 80 knitr highlight themes.

Other Programming Languages

While knitr must be run within the R environment, it also supports many other programming languages including:

Python
Ruby
Haskell
awk/gawk
sed
shell scripts
Perl
SAS
TikZ
Graphviz
C++
And others...

However, we have to install the corresponding software package in advance to use an engine.

Enter the engine function in Knitr. This function allows the user to specify the language for a chunk.

engine = "bash" will load in bash and allow the user to write shell scripts within that code chunk.

Appearance

There are many options that we can control within our .Rmd document. This section helps to introduce and explore some of these customization options for HTML documents.

Code Folding

As you may have noticed, each one of the code chunks in this document has an interactive button. This is controlled in the YAML header and is new as of RMarkdown v2.

When the knitr code chunk option echo = TRUE is specified (default = TRUE) the R code within the chunk will be included in the output document. However, there may be times in which a user would like to exclude code entirely (echo = FALSE)

-code_folding:
- code_folding: hide: Can include R code but have it hidden by default.
- code_folding: show: Shows R code and then the reader can click the button to hide that chunk if they wish.

output: html_document
    code_folding: show

toc: Whether to include to the table of contents:
- toc: true: Includes a table of contents
- Default:toc: false: Does not include a table of contents
toc_depth:: How many levels within the table of contents should be included in the table of contents?
- Default: doc_depth: 3 will include headers with ###.
number_sections: Add section numbers to the headers. For example, this document has number_sections: true
- Default: number_sections: false
- Note: With each # there will be a decimal point added to all headers.
toc_float:
- 2 other optional parameters within toc_float:
  - collapsed: Controls if the table of contents appears with only the top level. It will expand with the cursor over it.
    - Default: collapsed: TRUE
  - smooth_scroll: Controls whether the page scrolls are animated when the items within the table of contents are navigated to via clicking on them.
    - Default: smooth_scroll: true

For example:

output:
  html_document:
    toc: true
    toc_depth: 2
---

Challenge: Make the YAML header for an HTML document that includes:

Table of contents

Float the table of contents

Section headers with two hashtags (##)

Numbered sections

No smooth scrolling

Themes

RMarkdown has several options that control the appearance of HTML documents. Some arguments to choose from are:

theme
highlight
smart

The HTML output themes are drawn from the Bootswatch library. Valid HTML themes include the following:

cerulean, cosmo,flatly, journal, readable,spacelab, and united.
- For example, the theme of this page is readable.
Pass null for no theme (in this case you can use the css parameter to add your own styles).

Highlight specifies the syntax highlighting style. Supported styles include the following:

default, espresso, haddock, kate, monochrome, pygments, tango, textmate, and zenburn.
Pass null to prevent syntax highlighting.

Smart indicates whether to produce typographically correct output, converting straight quotes to curly quotes, --- to em-dashes, -- to en-dashes, and ... to ellipses. Smart is enabled by default.

For example:

---
output:
  html_document:
    theme: slate
    highlight: tango
---

If you felt inclined, you could also produce and use your own theme. If you did so, the output section of your YAML header would like like:

output:
  html_document:
    css: styles.css

If you wanted to go the extra mile and write your own theme in addition to highlight, the YAML header would look like:

---
output:
  html_document:
    theme: null
    highlight: null
    css: styles.css
---

Here's a link to learn more about the Appearance and Style in HTML output.

Caching

Problem: Some code chunks take a long time to run and may not be updated very often.

Solution: Caching! If a code chunk has not been modified since the last rendering of the document, the old results will be directly used instead of re-running the chunk.

Very simple solution: Exit knitting early -> if the rest of the document does not need to be rendered, simply put the knit_exit() and the rest of the document will be ignored. Therefore, results from all previous text and code chunks will be returned.

lazy-loading is when an object will not be loaded into memory until it is used. A "promise" is instead created, which is computationally easy (to learn more run?promise).

Therefore, when rendering the document, cached chunks are skipped and the output created previously from these chunks will be (lazy-) loaded from the cache folder.

However, if a single change to the chunk occurs (even a white space counts!) then knitr will acknowledge this change and will actually run the chunk when the document is rendered.

The user can also set the path for where the cached files should be by using cache.path

Default:cache.path = "file_name_cache/"
- For example: Check out the github repo for this lesson previously cached chunks are in the RMarkdown_Lesson_cache directory.

Some issues with caching:

R is updated every few months. R.version.string
External files that knitr does not track changes, may be updated and results will need to be re-run.
Sometimes a cached chunk might rely on objects from other cached chunk. This can be a serious problem - so be careful! Therefore, we should cover chunk dependencies!

Chunk Dependencies

Manual

We can manually specify whether chunks depend on each other.

dependson specifies which other chunks the current chunk relies on by setting a vector of chunk labels.
- Examples with integers:
  - depesndson = 1: Chunk relies on first Chunk
  - dependson = c(6,8): Chunk relies on 6^th^ and 8^th^ chunks
  - dependson = -1: Chunk relies on previous chunk.
  - dependson = c(-1, -2): Chunk relies on the two previous chunks.
  - Note: When dependson takes on an integer value, it cannot make a chunk depend on later chunks - only previous chunks. Therefore, it may be useful to use the chunk names or vectors of chunk names.
- Examples with chunk names:
  - dependson = c("Chunk-1", "Chunk-2", "Chunk-3")
  - dependson = c("data-generation", "data-transforamtion")
As a result, each time one of the cached chunks "Chunk-1", "Chunk-2", and "Chunk-3" are re-built, the current chunk will lose its cache and also be rebuilt!

Automatic

Enter: autodep chunk option and function dep_auto()

autodep and dep_auto() state that if any objects in the current chunk are created in previous chunks, then the current chunk depends on a previous chunk.

For a more conservative approach, enter dep_prev().

dep_prev states that a cached chunk will depend on all of its previous chunks. Therefore, whenever previous chunks are updated, all the later chunks will also be updated.

Knitr only tracks changes with cached chunks, not in uncached chunks! Fortunately, knitr will give a warning when it sees a dependency on an uncached chunk.

Load Cache Manually

Suppose you calculated a value z in a chunk towards the end of the document, however, you would like to use z in an earlier chunk. This is impossible because knitr compiles the document in a linear fashion and it cannot use objects created in the future.

Enter: load_cache, which takes a chunk label to find the cache database

load_cache(label, object, notfound = "NOT AVAILABLE", 
  path = opts_chunk$get("cache.path"), lazy = TRUE)

Therefore, if you use z in an inline R expression, it will output NOT AVAILABLE and since you have specified notfound = "NOT AVAILALBE" it will go back at the end and replace it with the value of z.

So cool!

Side Effects

A side effect refers to a state change that occurs outside of a function that is not the returned value.

par() and options() are side effects in the sense that they are not cached.
Set all the global options in the first chunk and never cache that chunk.

We need to be careful with the chunk options to be sure that results from cached chunks are up-to-date.

We can also turn off lazy-loading by setting the cache.lazy = FALSE.

Bibliography

It's also possible to include a bibliography file in the YAML header. Bibliography formats that are readable by Pandoc include the following:

| Format | File extension | |-----+-------| | MODS | .mods | | BibLaTeX | .bib | | BibTeX | .bibtex | | RIS | .ris | | EndNote | .enl | | EndNote XML | .xml | | ISI | .wos | | MEDLINE | .medline | | Copac | .copac | | JSON citeproc | .json |

To create a bibliography in RMarkdown, two files are needed:

A bibliography file with the information about each reference.
A citation style language (CSL) to describe how to format the reference

An example YAML header with a bibliography and a citation style language (CSL) file:

output: html_document
bibliography: bibliography.bib
csl: nature.csl

Check out the very helpful webpage by the R Core team on bibliographies and citations.

If you would like to cite R packages, knitr even includes a function called write_bib() that creates a .bib entries for R packages. It will even write it to a file!

write_bib(file = "r-packages.bib") # will write all packages  
write_bib(c("knitr", "ggplot2"), file = "r-packages2.bib") # Only writes knitr and ggplot2 packages

Placement

Automatically the bibliography will be placed at the end of the document. Therefore, you should finish your .Rmd document with # References so the bibliography comes after the header for the bibliography.

final words...

# References

Citation Styles

Citation Sylte Language (CSL) is an XML-based language that identifies the format of citations and bibliographies. Reference management programs such as Zotero, Mendeley and Papers all use CSL.

Search for your favorite journal and CSL in the Zotero Style Repository, which currently has 8,152 CSLs. Is there a style that you're looking for that is not there?

output: html_document
bibliography: bibliography.bib
csl: nature.csl

In the github repo for this workshop I have included nature.csl and the-isme-journal.csl to play around with. Otherwise, download a citation from the Zotero Style Repository!

Citations

Citations go inside square brackets [ ]and are separated by semicolons ;. Each citation must have a key, composed of @ + the citation identifier from the database, and may optionally have a prefix, a locator, and a suffix. To check what the citation key is for a reference, take a look at the .bib file. Here in this file, you can also change key for each reference. However, be careful that each ID is unique!

Here are some examples with their code:

Microbes control Earth's biogeochemical cycles [@Falkowski2008].
- Code: Microbes contorl Earth's biogeochemical cycles [@Falkowski2008].
I love making beautiful plots with ggplot2 [@R-ggplot2]
- Code: I love making beautiful plots with ggplot2 [@R-ggplot2]
Dr. Yuhui Xie's book about Dynamic Documents [@Xie2015] inspired me to host this workshop.
- Code: Dr. Yuhui Xie's book about Dynamic Documents [@Xie2015] inspired me to host this workshop.
A great article in Science regarding biogeography of microbes asks readers to imagine their Alice in Wonderland to shrink down to understand the microbial world [@Green2008].
- Code: A great article in *Science* regarding biogeography of microbes asks readers to imagine they are Alice in Wonderland to and shrink down to understand the microbial world [@Green2008].

The coolest thing is that the only references added to the document will be the ones that you have cited!

Publishing on RPubs

Once you make a beautiful dynamic document you may wish to share it with others. One option to share it with the world is to host it on RPubs. With RStudio, this makes is very easy! Do the following:

Create your awesome .Rmd document.
Click the button to render your HTML document to be published.
In the top right corner of the preview window, click the publish button and follow the directions.
- Note: You will need to create an RPubs profile.
Once you have a profile you can choose the following:
- The title of the document.
- A description of the document.
- The URL in which the website will be hosted.
  - Note: The beginning of the URL will be: www.rpubs.com/your_username/name_of_your_choice

Updating RPubs

If you make some changes to your document it is very easy to update the webpage. Once you have rendered your edited document click the button on the top right corner of the preview window. The edited document will be in the same URL as the original document.

Thank you

Thank you for participating in this tutorial.

If you have updates you would like to make to the lesson, please send me a pull request. Alternatively, if you have any questions, please e-mail me.

Marian Schmidt
E-mail: marschmi at umich.edu
Twitter @micro_marian

Good luck with your pursuit of dynamic documents!

Session Info

devtools::session_info()

## Session info --------------------------------------------------------------

##  setting  value                       
##  version  R version 3.3.0 (2016-05-03)
##  system   x86_64, darwin13.4.0        
##  ui       X11                         
##  language (EN)                        
##  collate  en_US.UTF-8                 
##  tz       America/Detroit             
##  date     2016-05-12

## Packages ------------------------------------------------------------------

##  package     * version date       source        
##  assertthat    0.1     2013-12-06 CRAN (R 3.3.0)
##  base64enc     0.1-3   2015-07-28 CRAN (R 3.3.0)
##  colorspace    1.2-6   2015-03-11 CRAN (R 3.3.0)
##  DBI           0.4     2016-05-02 CRAN (R 3.3.0)
##  devtools      1.11.1  2016-04-21 CRAN (R 3.3.0)
##  digest        0.6.9   2016-01-08 CRAN (R 3.3.0)
##  dplyr       * 0.4.3   2015-09-01 CRAN (R 3.3.0)
##  DT          * 0.1     2015-06-09 CRAN (R 3.3.0)
##  evaluate      0.9     2016-04-29 CRAN (R 3.3.0)
##  formatR       1.3     2016-03-05 CRAN (R 3.3.0)
##  ggplot2     * 2.1.0   2016-03-01 CRAN (R 3.3.0)
##  gridExtra     2.2.1   2016-02-29 CRAN (R 3.3.0)
##  gtable        0.2.0   2016-02-26 CRAN (R 3.3.0)
##  highr         0.5.1   2015-09-18 CRAN (R 3.3.0)
##  htmltools     0.3.5   2016-03-21 CRAN (R 3.3.0)
##  htmlwidgets   0.6     2016-02-25 CRAN (R 3.3.0)
##  httr          1.1.0   2016-01-28 CRAN (R 3.3.0)
##  jsonlite      0.9.19  2015-11-28 CRAN (R 3.3.0)
##  knitr       * 1.13    2016-05-09 CRAN (R 3.3.0)
##  labeling      0.3     2014-08-23 CRAN (R 3.3.0)
##  magrittr      1.5     2014-11-22 CRAN (R 3.3.0)
##  memoise       1.0.0   2016-01-29 CRAN (R 3.3.0)
##  munsell       0.4.3   2016-02-13 CRAN (R 3.3.0)
##  pander      * 0.6.0   2015-11-23 CRAN (R 3.3.0)
##  plotly      * 3.4.13  2016-04-12 CRAN (R 3.3.0)
##  plyr          1.8.3   2015-06-12 CRAN (R 3.3.0)
##  R6            2.1.2   2016-01-26 CRAN (R 3.3.0)
##  Rcpp          0.12.4  2016-03-26 CRAN (R 3.3.0)
##  rmarkdown   * 0.9.6   2016-05-01 CRAN (R 3.3.0)
##  scales        0.4.0   2016-02-26 CRAN (R 3.3.0)
##  stringi       1.0-1   2015-10-22 CRAN (R 3.3.0)
##  stringr       1.0.0   2015-04-30 CRAN (R 3.3.0)
##  tidyr         0.4.1   2016-02-05 CRAN (R 3.3.0)
##  viridis       0.3.4   2016-03-12 CRAN (R 3.3.0)
##  withr         1.0.1   2016-02-04 CRAN (R 3.3.0)
##  yaml          2.1.13  2014-06-12 CRAN (R 3.3.0)

Files

RMarkdown_Lesson.md

Latest commit

History