AWS Deployment Guide

This is a guide for Shortcut 1.0. You probably want this guide instead.

Setting up Shortcut for production on AWS

For ease of production deployment we provide an Amazon Machine Image that you can deploy with one click to EC2. There is still manual customization involved, but at least this prevents you from having to set up all the dependencies from scratch.

This guide assumes you are at least passingly familiar with AWS EC2 and Linux generally.

Our AMI is based on Ubuntu 16.04 LTS, and it comes pre-installed with Shortcut as well as all of its opereating system dependencies, including ffmpeg and node-canvas. The Shortcut application server is also already installed as a systemd service so it will restart if the process dies, or if the server reboots.

Launching the AMI

To launch an EC2 instance with the Shortcut AMI, simply click this bookmark link, which will begin the instance creation process for you. You should now see the "Step 2: Choose an Instance Type" prompt—select a t2.small or larger instance.

Note — If you're doing a small beta deployment and would like to stick to the free tier, t2.micro is okay, but we highly recommend upgrading to t2.small or larger once you're in full production.

Press "Next" until you get to "Configure Security Groups" and configure a new security group with the following rules:

Type	Protocol	Port Range	Source
SSH	TCP	22	My IP/Custom/Anywhere**
HTTP	TCP	80	Anywhere
HTTPS	TCP	443	Anywhere
Custom TCP	TCP	8443	Anywhere
Custom TCP	TCP	3000	Anywhere

** Be smart! Set this to your static work IP(s) that you'll be SSHing from, or use Amazon's "My IP" feature which detects the IP where you're currently sitting. We don't recommend setting this to "Anywhere"!

Press "Review and Launch". Review the settings (there will be a warning about your instance being open to the public due to the custom TCP ports we opened for authentication, it's okay) and click "Launch".

Select your key pair options in the popup dialog, then click "Launch".

Assign a static IP

We're going to use Amazon's Elastic IP to assign a static IP to the new instance. In the EC2 console in the left menu, under "Network & Security", click "Elastic IPs". Press the "Allocate new address" button, then "Allocate". You should see a success message. Close this and then right-click on the new IP. Select your instance you just launched and click "Associate". Now you have a static IP for your server, which will come in handy later when we configure DNS.

If you go to your new IP in a web browser window, you should see this:

Configure DNS and HTTPS

At this point in the tutorial, we have a static IP and Apache is serving via HTTP. But you probably don't want to serve Shortcut off http://123.45.67.890! More likely you want to serve it off a subdomain like https://shortcut.mypodcastwebsite.com, which will require configuring DNS and HTTPS. The extremely short version of this is:

• Go to your DNS provider and set an A record for the subdomain you want, and point it to the Elastic IP address you assigned above. This will create the subdomain.
• Set up SSL/HTTPS on the Apache server. Here's a good tutorial. Once this is set up, you need to take note of where your SSL key and cert are stored for the next step.

Configure and run Shortcut

Things are looking pretty good right now, but Shortcut still isn't actually running. We're serving some default static files from /var/www/html, but the Shortcut server isn't running and there are no podcast files or anything. What we need to do now is get Shortcut up and running for real. Go ahead and ssh to your EC2 instance (it'll be as user ubuntu since that's the default username for an EC2 Ubuntu instance).

First things first: be a good sysadmin and update the OS!

sudo apt update && sudo apt upgrade -y

Next pull the latest Shortcut release from the master branch:

cd shortcut/
git pull origin master

Then install the Node.js dependencies for the client and server:

cd client/
npm install
cd ../server
npm install

Next we'll run a bash script that copies template configuration files (which are checked in to git with placeholder data) to the real production configuration files (which are ignored by git for security). Go back to your root Shortcut directory and run our setup script:

cd ../
./setup.sh

You should see some messages saying that a number of different files were created.

The different configuration files are as follows:

• server/.env contains all variables related to social media authentication, as well as AWS and Cloudfront credentials, SSL key locations for serving over https, and a few other basic configuration items
• client/src/config/base.js lets you configure some of the menus and titles, set the min/max length for a Shortcut clip, specify a Google Analytics ID if you have one, and set your S3 bucket info
• client/src/config/dev.js contains configurations specifically for a development environment. In most cases these should stay at their default values. At any rate, this isn't terribly relevant to a production deployment.
• client/src/config/dist.js is the config file for a distribution (production) environment. Here we point the production client to the location of our authentication server, shortcut API endpoint, and media and transcript files
• client/src/images/logo.png can be replaced with your podcast logo in PNG format
• client/src/images/animation-footer.png is a 400x86 pixel transparent PNG that gets superimposed on the bottom of any generated clips as a watermark
• client/src/styles/_variables.scss contains the SCSS variables that let us configure what the website looks like. See Skinning Shortcut for details.
• client/src/styles/Custom.scss is a file where you can put any custom CSS overrides you want. By default it contains the hero-space class configuration for the front page as an example. You can reference variables from the _variables.scss file here as well.

Set up your AWS S3 bucket

The Shortcut server creates the videos and then uploads them to S3 for semi-permanent storage and access. We recommend also using S3 to serve your basic media assets as well. If you already set up S3 when you were getting your local environment working, these steps should be familiar. We recommend doing them again anyway, setting up a separate S3 bucket for production.

• Log in to AWS
• Go to your S3 dashboard.
• Click "Create Bucket"
• Name it something like shortcut-my-project-production
• Select a region, ideally one geographically close to most of your listeners
• Click "Next" twice to get to "Set Permissions" and under "Manage public permissions" select "Grant public read access to this bucket" (videos are created and uploaded to S3, so users need to be able to download those videos)
• Hit "Next" and then "Create Bucket"

Now we'll set up CORS (cross-domain file access).

• Select the bucket you just named
• Click the "Permissions" tab
• Click the "CORS configuration" button
• Overwrite the example CORS configuration with the one below and hit "Save"

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

Save that and now you should be good to go!

Next, we need to give Shortcut programmatic write access to this bucket. We'll do this the way Amazon suggests, which is to create an IAM user just for Shortcut.

• Go to your IAM dashboard.
• Click "Users" in the left-hand menu
• Click "Add User" at the top
• Enter a user name
• Select "Programmatic access" under "Access type" and hit "Next"
• Select "Create Group"
• Enter a group name ("s3-full-access" is fine)
• Scroll down (or use the search filter) and find "AmazonS3FullAccess" and tick the box
• Click "Create Group"
• Ensure that your new group is selected in the group selection dialog, then click "Next: Review"
• Click "Create User"

Now you should see an access key and a secret key (the secret key needs to be revealed by clicking "Show"). Copy both of those values and put them in your server/.env file as

AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-access-key

Put your media files on S3

Prep your media files following the instructions in Importing Your Own Podcast under Create the episode metadata and Convert your episode to HLS format. On your local machine you should have the following files in some working directory, with an episodes.json file containing metadata on your episodes and one subdirectory for each episode (s01e03 here is an example unique episode id).

/path/to/data/
/path/to/data/episodes.json
/path/to/data/s01e03/s01e03.mp3
/path/to/data/s01e03/s01e03.m3u8
/path/to/data/s01e03/s01e03.txt
/path/to/data/s01e03/s01e03.json
/path/to/data/s01e03/s01e03000.ts
/path/to/data/s01e03/s01e03001.ts
/path/to/data/s01e03/s01e03002.ts
...(etc)

Upload episodes.json and each episode subdirectory to the root of your S3 bucket.

Configure your API hash and data bucket

The API_HASH variable is an arbitrary string for access to the private API you'll use to update and add new episodes to Shortcut. This is never accessed from the client and should be treated like a password. (In production you'll make GET requests to a url like https://shortcut.mypodcastwebsite.com/api/API_HASH/update/EPISODE_ID to add new episodes.) Set this to a long, URL-safe string (Ubuntu's uuidgen program is a great choice to generate this).

The DATA_BUCKET variable points the server to the location of your media files. We just configured this to be the root of your S3 bucket, so add that here:

DATA_BUCKET=https://s3-[YOUR_S3_REGION].amazonaws.com/[YOUR-BUCKET-NAME]/

Process all of your episodes

Assuming all your media is in place on S3, you can run the following utility script to enable all of your episodes for viewing:

cd ~/shortcut/server/
node update-all-episodes.js

If you only want to individually enable certain episodes, just send a GET request to this URL for each episode you want to update:

http://YOUR_SERVER:3000/api/API_HASH/update/EPISODE_ID

It should take about 30 seconds for an episode to process.

Configure social media credentials

Follow the instructions in Getting started (for developers) in the Set up social media sharing section to get your Twitter and Facebook application keys.

Restart the server

Now we restart the Shortcut application server to use all these new settings.

sudo systemctl restart shortcut.service

Configure, build, and deploy the client

The Shortcut client is the portion of the software served as static assets to a user's web browser. In this portion of the tutorial we're going to configure it to match our server settings above, run a webpack build to generate the newly configured static assets, and then deploy those assets to our Apache public directory.

Configure the client

Open client/src/config/base.js in a text editor and update the following variables:

• parentSiteName - the human-language name of the parent podcast website. 'My Awesome Podcast' or similar.
• parentSiteUrl - the url of the parent podcast website, i.e., 'https://www.myawesomepodcast.com'
• tagline - a short phrase that will appear on the front page of the Shortcut site under your podcast logo. Set to an empty string to leave blank.
• s3Region - your Amazon S3 region you set up above. Will be something like 'us-west-2'
• s3Bucket - the name of your S3 bucket

Open client/src/config/dist.js in a text editor and replace every instance of "localhost" with your EC2 IP or host name. So for example, if you're hosting at https://shortcut.example.com, it should look like this:

let config = {
  appEnv: 'dist',
  authServer: 'http://shortcut.example.com:3000',
  authServerSsl: 'https://shortcut.example.com:8443',
  apiEndpoint: 'http://shortcut.example.com:3000',
  apiEndpointSsl: 'https://shortcut.example.com:8443',
  dataBucket: 'http://shortcut.example.com:8080/',
};

Make sure to leave all the port designations in tact!

Next, replace client/src/images/logo.png with your podcast logo, and client/src/images/animation-footer.png with a 400x86 watermark footer, if you so wish (it's a blank transparent image by default).

If you'd like to edit the default color palette, you can now edit client/src/styles/_variables.scss as described in Skinning Shortcut. (Try updating $primary-color to red if you want to test out and see a major change.)

Build the client

Shortcut uses webpack to bundle a complex set of dependencies into a small number of highly optimized assets. This maximizes performance, but does introduce some overhead. We thought it was worth it for this project.

To build the client, make sure you're in the client directory and then run npm run dist:

cd ~/shortcut/client
npm run dist

This step might take up to a minute. If you've done everything right you should see a bunch of log messages but no error messages.

Deploy the client

Next run the helper script we've provided in the client/ directory:

./deploy.sh

This copies all the webpack-built files (which live in client/dist/) over to /var/www/html/, our Apache public directory. If all goes well you should see a Copied files to Apache public directory! message.

Test it out

At this point, you should be fully deployed! Go to your IP or hostname in a web browser and you should see your episodes listed and they should be fully browsable and clippable!