Setting up Shiny Server on Azure

[gadenbuie]

Taking over for #10 (some resources there will also be helpful).

• Dean Attali: How to get your very own RStudio and Shiny Server (with Digital Ocean)
• Deploying R, RStudio and Shiny applications on Ubuntu Server – Enhance Data
• R developer's guide to Azure
• R and Docker - Revolution Analytics

[gadenbuie]

Getting Started

Create admin user

sudo adduser adminuser
sudo adduser adminuser sudo
sudo su - adminuser

Use a really, really good password that you save somewhere and have no chance of remembering. Then copy your local ssh key into /home/adminuser/.ssh/authorized_keys.

# locally
ssh-keygen # if you don't have an id_rsa in ~/.ssh yet
cat ~/.ssh/id_rsa.pub

# on server
sudo mkdir /home/adminuser/.ssh/
sudo vim /home/adminuser/.ssh/authorized_keys
# paste contents of id_rsa.pub, save & quit

sudo chown adminuser:adminuser /home/adminuser/.ssh
sudo chmod 700 /home/adminuser/.ssh
sudo chmod 600 /home/adminuser/.ssh/authorized_keys

Then force the use of ssh keys for remote access. In /etc/ssh/sshd_config set the following:

LoginGraceTime 60
PermitRootLogin no
PasswordAuthentication no
Banner /etc/banner

Then add a fancy login message to /etc/banner (requires sudo) and restart sshd.

sudo systemctl restart sshd

I also turned off the Ubuntu login new service because they're annoying.

sudo vim /etc/default/motd-news
# Set
ENABLED=0

Add other users

One-liner to request SSH keys from others (source)

( [ -e ~/.ssh/id_rsa.pub ] || (cat /dev/zero | ssh-keygen -b 2048 -t rsa -q -N "")) && (echo "\`\`\`"; echo $(cat ~/.ssh/id_rsa.pub); echo "\`\`\`") | pbcopy

sudo adduser newuser
sudo adduser newuser shiny-users
sudo adduser newuser docker
sudo passwd -e newuser

sudo su - newuser
mkdir .ssh
vim .ssh/authorized_keys # add ssh key here
chmod 700 .ssh
chmod 600 .ssh/authorized_keys

mkdir ShinyApps

Setup SSH

# inside admin user
mkdir ~/.ssh
vim ~/.ssh/authorized_keys
# Save your local ~/.ssh/id_rsa.pub output here

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Install nginx

sudo apt-get update
sudo apt-get -y install nginx

Install Docker

sudo apt-get install -y docker.io

Install R

# If on Ubuntu check which release
$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.1 LTS
Release:	18.04
Codename:	bionic

https://www.rstudio.com/products/shiny/download-server/

sudo sh -c 'echo "deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/" >> /etc/apt/sources.list'

Add CRAN's Ubuntu Public Key

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9

Install global packages

sudo apt-get install -y git libssl-dev libgit2-dev libxml2-dev libssl-dev libcurl4-openssl-dev
sudo apt-get install -y git libssl-dev libgit2-dev libxml2-dev libssl-dev libcurl4-gnutls-dev
sudo su - -c "R -e \"install.packages(c('tidyverse', 'devtools', 'rmarkdown', 'knitr', 'shiny'), repos='https://cran.rstudio.com/')\""

These packages will be installed and available for all users, but they will be able to install different (or different versions of these) packages locally. We're just saving them the hassle of re-installing tidyverse.

Install Shiny Server

Install Shiny Server

sudo apt-get install gdebi-core
wget https://download3.rstudio.org/ubuntu-14.04/x86_64/shiny-server-1.5.9.923-amd64.deb
sudo gdebi shiny-server-1.5.9.923-amd64.deb

Setup /srv/shiny-server/

Create shiny-users group for users who have access to shiny, can deploy apps, etc., then add any relevant users.

sudo groupadd shiny-users
sudo adduser adminuser shiny-users

Then create /srv/shiny-server/ and make sure that shiny-users have read/write access.

sudo chown -R shiny:shiny-users /srv/shiny-server/
sudo chmod g+w /srv/shiny-server/
sudo chmod g+s /srv/shiny-server/

Configure Shiny Server

Setup /etc/shiny-server/shiny-server.conf following the admin guide. My goal is to have anything going to apps.gerkelab.com/ be passed to Shiny Server (unless we set up a specific app, see below), or to have user accounts connect apps.gerkelab.com/u/username/appname/ to /home/username/ShinyApps/appname/.

sudo vim /etc/shiny-server/shiny-server.conf

server {
  listen 3838;

  # User Apps
  location /u {
    run_as :HOME_USER:;
    user_dirs;
    directory_index on;
  }

  location / {
    # ... default config here ...
  }
}

TODO: custom templates

Setup nginx

Configure nginx

sudo vim /etc/nginx/sites-enabled/default

Added the following so that the default behavior is to hand traffic off to shiny server.

location / {
    proxy_pass http://127.0.0.1:3838/;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    #rewrite ^(/shiny/[^/]+)$ $1/ permanent;
    #try_files $uri $uri/ =404;
}

Also change server_name

server_name apps.gerkelab.com;

Apps living in docker containers will be connected through nginx with blocks like

location /docker-1/ {
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_pass docker.server:8001;
}

location /docker-2/ {
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_pass docker.server:8002;
}

This reference may also be helpful here.

Enable HTTPS

TODO: In short, use certbot. This does require that your instance be associated with a web domain, i.e. apps.gerkelab.com and not 123.456.1.2.

Docker

Setup Docker Compose

Create a user for hosting the shiny apps, shiny-docker and add this user to the docker group. (Also remember to add other users to this group as well.)

sudo adduser shiny-docker
sudo adduser shiny-docker docker
sudo apt-get install -y docker-compose
sudo su - shiny-docker

See https://github.com/rocker-org/shiny/blob/master/docker-compose.yml and https://docs.docker.com/compose/reference/overview/.

For a specific app I created

$ cat epitad/docker-compose.yml
version: "3"

services:
  epitad:
    image: grrrck/epitad:latest
    ports:
      - "9000:3838"
    restart: always
    volumes:
      - 'shiny_logs:/var/log/shiny-server'
      - './mountpoints/apps:/srv/shiny-server'

volumes:
  shiny_logs:

~/epitad$ docker-compose up -d
Creating network "epitad_default" with the default driver
Creating volume "epitad_shiny_logs" with default driver
Creating epitad_epitad_1 ...
Creating epitad_epitad_1 ... done

Then update /etc/nginx/sites-enabled/default

location /epitad/ {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_pass http://127.0.0.1:9000/epiTAD/;
}

and restart nginx sudo systemctl restart nginx.

[gadenbuie]

Dockerizing Shiny Apps

• rocker-org/shiny
• Environment variables in Compose | Docker Documentation
• Deploying Apps - ShinyProxy

FROM rocker/shiny:3.5.1
LABEL maintainer="Travis Gerke ([email protected])"

# Install system dependencies for required packages
RUN apt-get update && apt-get install -y libssl-dev libxml2-dev \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/ \
  && rm -rf /tmp/downloaded_packages/ /tmp/*.rds

RUN Rscript -e "install.packages(c('BiocManager', 'shinydashboard', 'data.table'))" \
  && rm -rf /tmp/downloaded_packages/ /tmp/*.rds

RUN Rscript -e "BiocManager::install(c('biomaRt'))" \
  && rm -rf /tmp/downloaded_packages/ /tmp/*.rds

ARG SHINY_APP_IDLE_TIMEOUT=60
RUN sed -i "s/directory_index on;/app_idle_timeout ${SHINY_APP_IDLE_TIMEOUT};/g" /etc/shiny-server/shiny-server.conf
COPY . /srv/shiny-server/app_name

Can also use

FROM rocker/shiny-verse:3.5.1

Iterate: build image, some package installation fails because missing dependency, add deps or adjust Dockerfile, repeat.

docker build -t appname:latest .

Run your app, then watch the logs

docker run -d -p 3838:3838 --name appname appname:latest
docker logs --follow appname

If you need to enter the container to debug

docker exec -it appname bash

drops you into the container as root. If things are badly broken, or for debugging/developing the app directly in RStudio within a similar container, comment out the last three lines, and change

#FROM rocker/shiny:3.5.1
FROM rocker/tidyverse:3.5.1

and then rebuild and run

docker build -t grrrck/appname .
docker run -d -p 9797:8787 -v (pwd):/home/rstudio/appname -e PASSWORD=aPassWordGoesHere --name appname appname

This might take a little while to rebuild the intermediate containers, but you'll end up with an almost identical environment with RStudio. Go to 127.0.0.1:9797/rstudio to edit/run your app. Keep track of anything you install (change Dockerfile as needed).

[gadenbuie]

Monitoring shiny apps: https://www.rcharlie.com/post/shiny-monitor/