Getting started (for developers)

If you're looking at this page it's because you're a developer and you're interested in getting Shortcut up and running for a podcast! The first thing you need to do is get it set up in a local development environment.

Right now, we have instructions for getting this environment set up on MacOS. In the future we'll have Linux and Windows guides as well.

Running locally on macOS

First, we recommend you install Homebrew as this makes installing dependencies much, much easier.

Next, install Git if you don't already have it.

brew install git

Next install ffmpeg:

brew install ffmpeg

(Note: this has to be ffmpeg v3.2 or higher, installed from homebrew, as we use the native aac encoding and it's pretty poor quality in lower versions.)

Now install Node.js. We recommend at least version 6.0 or higher. Homebrew should give you the latest "current" build by default (v8.6.0 as of writing this).

brew install node

This also installs npm, a package manager for Node.js, which we'll use to install some more dependencies. Use npm to install node-gyp (also a slur for Romani people, you can thank whatever engineers at Google thought "Generate Your Projects" would be fun and catchy).

npm install -g node-gyp

And now a whole bunch of dependencies we need for the server-side rendering of videos (plus the Amazon Web Services command line interface):

brew install pkg-config cairo libpng jpeg giflib awscli

Next up, grab the Shortcut project from Github!

git clone https://github.com/FeelTrainCoop/shortcut.git

...and now install EVEN MORE dependencies for the client and the server portions of the project, because computer software is a nightmare.

cd shortcut/client/
npm install
cd ../server
npm install

For the next step we'll run a bash script that copies template configuration files (which are checked in to git with placeholder data) to the real production configuration files (which are ignored by git for security). The script also installs Gentle, the software that we use to align audio to transcripts.

Go back to your root Shortcut directory and run the setup script. This script compiles Gentle, which depending on your machine can take quite a long time (several hours on a powerful macbook!):

cd ../
./setup.sh

In a new terminal window, go to the Shortcut client directory and run the local webpack-driven development server.

cd /path/to/shortcut/client
npm run serve

Open http://localhost:8000 in a web browser! It's going to take up to 10-15 seconds to load the first time because of webpack compiling, but will hotload very quickly after that.

We aren't yet running the server-side component, so this is just an empty Shortcut page with no episodes in its list because it has no data to read yet. We'll fix that next.

Add the sample episode

What we're going to do at this point is run you through the process of installing a sample episode that we provide.

The sample episode is included as a ZIP file in the doc-assets/test-assets/ directory. Or you can just grab it directly from here. The episode should be unzipped into an empty directory somewhere not in your Shortcut project folder. This directory will contain episodes.json and a folder called s01e01/, which contains a bunch of different files that comprise the transcript, the waveform data, and the audio files broken up into HLS streaming formatted files.

The data needs to be served on an http server. For now we can simply install http-server and run a simple CORS-enabled server right in the root directory. In your terminal, go to the path where you unzipped the files (where episodes.json is) and type:

npm install -g http-server
http-server --cors .

This will create a CORS-enabled http server at http://localhost:8080. Go to the URL now and make sure it's serving those files.

Run npm start to start your server up. Also make sure your client is still running in another terminal window with /path/to/shortcut/client/npm run serve.

Set up your AWS S3 bucket

The Shortcut server creates the videos and then uploads them to S3 for semi-permanent storage and access. Because users then download those videos, we need to create a public-readable S3 bucket.

• Log in to AWS
• Go to your S3 dashboard.
• Click "Create Bucket"
• Name it something like shortcut-my-project
• Select a region, ideally one geographically close to most of your listeners
• Click "Next" twice to get to "Set Permissions" and under "Manage public permissions" select "Grant public read access to this bucket" (videos are created and uploaded to S3, so users need to be able to download those videos)
• Hit "Next" and then "Create Bucket"

Now we'll set up CORS (cross-domain file access).

• Select the bucket you just named
• Click the "Permissions" tab
• Click the "CORS configuration" button
• Overwrite the example CORS configuration with the one below and hit "Save"

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

In your server/.env file, put your S3 bucket name in the following variable:

AWS_S3_BUCKET_NAME=your-bucket-name

Next in your client/src/config directory, edit base.js and similarly fill in your S3 bucket name.

'use strict';

// Settings configured here will be merged into the final config object.
export default {
  parentSiteName: 'YOUR WEBSITE',
  parentSiteUrl: 'https://example.com',
  maxClipSeconds: 30,
  minClipSeconds: 0.51,
  analyticsPropertyId: '',
  s3Bucket: 'your-bucket-name',
  episodesWithProblems: []
}

Next, we need to give Shortcut programmatic write access to this bucket. We'll do this the way Amazon suggests, which is to create an IAM user just for Shortcut.

• Go to your IAM dashboard.
• Click "Users" in the left-hand menu
• Click "Add User" at the top
• Enter a user name
• Select "Programmatic access" under "Access type" and hit "Next"
• Select "Create Group"
• Enter a group name ("s3-full-access" is fine)
• Scroll down (or use the search filter) and find "AmazonS3FullAccess" and tick the box
• Click "Create Group"
• Ensure that your new group is selected in the group selection dialog, then click "Next: Review"
• Click "Create User"

Now you should see an access key and a secret key (the secret key needs to be revealed by clicking "Show"). Copy both of those values and put them in your server/.env file as

AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-access-key

Next we'll ask Amazon what the region code is for our Bucket. First we configure the AWS cli tool:

aws configure

And enter the access key and secret access key. When it asks for your region code, put "us-east-1" as a default (this is not necessarily your actual region code, just a default one). When it asks for the output format type "json". (If you need more help with this, check out Amazon's getting started guide for the CLI.)

Now we can ask Amazon what region the bucket is in, replacing the last parameter here with your actual bucket identifier:

aws s3api get-bucket-location --bucket your-bucket-name

This should return something like this, which gives you the region code:

{
  "LocationConstraint": "us-XXXX-X"
}

In your server/.env file, put your S3 bucket region code in the following variable:

AWS_REGION=us-XXXX-X

And in your client's client/src/config/base.js file, update the s3Region variable:

  s3Region: 'us-XXXX-X'

Start your server!

cd server/
npm start

Pre-process your sample episode

Episodes need to have their waveform data pre-processed and uploaded to S3. The sample episode is ID number s01e01, so go to the following URL in a browser:

http://localhost:3000/api/ABC123/update/s01e01

ABC123 is the default API_HASH variable set in server/.env. s01e01 corresponds to the episode number. So our API endpoint for updating episodes is ${YOUR_SERVER}/api/${API_HASH}/update/${EPISODE_ID}.

Wait a while, potentially for a few minutes, for processing and uploading to occur. You'll eventually get a success message like: {"status":"success","showNumber":"s01e01"}. You can test to make sure it worked by examining your S3 bucket contents. There should be a directory called "episodes" containing a file called s01e01-data.json. (You don't need to know what's exactly in this file but it contains the metadata for your episode, the fully timing-aligned transcript, and the waveform data.)

Now your test podcast is ready to be shared via Shortcut!

Test it out!

Make sure you're running both the Shortcut server (npm start in the server directory) and the local dev server for Shortcut client (npm run serve in the client directory).

The server runs on localhost:3000 and the dev client runs on localhost:8000 -- go to http://localhost:8000 now and you should see a landing page that looks like this:

Click on the episode, select some text to share, click "Next", and you should have a playable, downloadable, shareable MP4 clip!

Set up social media sharing

In order to enable direct sharing to Twitter, you'll need to register an application on the service.

Setting up Twitter sharing

To register your application with Twitter, you'll need to have that application tied to a specific Twitter account. It might make sense to use the Twitter account of your podcast, or your personal Twitter account, or to create a new, one-off Twitter account. Note: you won't be posting things directly to this Twitter account. Twitter just needs you to be logged in as a user in order to register an app.

While logged in as the account you want tied to the app, go to this page:

https://apps.twitter.com/app/new

Once you're there, fill in the required fields: name, description, website. These are the fields that will show when a user eventually clicks through to grant your application permission to post on their behalf, so you'll want to make sure it's "customer-facing" like the name of your podcast and not testapp744 or whatever. Also you must set the callback URL to YOUR_SERVER/auth/twitter/callback, replacing YOUR_SERVER with the details for your Shortcut server, including the port. Do the captcha and submit.

A Note About The Callback URL — If you're simply testing on localhost, putting in localhost:3000 will suffice for now, since this will ultimately just be a browser redirect from the user auth page that will work on your own machine. For others on your network to test it, you could put your internally-accessible IP/port. For the general public or people outside your network to use this, you'll need to point it to a public-facing server.

Next you'll see a screen with a "Details" tab. Click on "Keys and Access Tokens" and you'll see two long strings: your consumer key and consumer secret. Put these in your server/.env file:

TWITTER_KEY=XXXXXYYYYYYZZZZZZZ
TWITTER_SECRET=AAAAAAABBBBBBBBBBCCCCCCCDDDDDDDD
TWITTER_CALLBACK=/auth/twitter/callback

A note about Facebook

We used to support posting to Facebook but they removed the publish_actions permission and so we can't post video directly to Facebook anymore. More info here.

Testing social media sharing

At this point, you should be able to restart your Shortcut server and use the "Connect" menu to log in to Twitter. It will ask you to authorize the application to post on your behalf, and then redirect you back to your Shortcut server. At this point you can share directly from the Sharing page, after you've generated a video.

Conclusion

And that's it! At this point you should have a locally-run instance of Shortcut that serves up a single episode, our test episode we provided you.

But of course, you don't want to serve up a test episode. You want to provide your own episodes to your listeners. To do that, you'll need to move on to the next step of this guide, Importing Your Own Podcast.