diff --git a/GettingStarted.md b/GettingStarted.md new file mode 100644 index 0000000..9ada1f2 --- /dev/null +++ b/GettingStarted.md @@ -0,0 +1,212 @@ +# Table of Contents + + + + +- [Why Should I Use a Service Worker?](#why-should-i-use-a-service-worker) +- [Terminology](#terminology) + - [Service Worker](#service-worker) + - [App Shell](#app-shell) + - [Dynamic Content](#dynamic-content) + - [Caching Strategy](#caching-strategy) +- [Add `sw-precache` to Your Build](#add-sw-precache-to-your-build) + - [Automation](#automation) + - [Configuration](#configuration) + - [Basic](#basic) + - [Runtime Caching for Dynamic Content](#runtime-caching-for-dynamic-content) + - [Server-side Templating](#server-side-templating) + - [Fallback URL](#fallback-url) +- [Examples](#examples) +- [Other Resources](#other-resources) + - [Articles](#articles) + - [Videos](#videos) + + + +# Why Should I Use a Service Worker? + +You have a web app, and you'd like it to load quickly and work offline. + +You'd like to use proven tools to handle the details for you, to work around +common gotchas and follow best practices. + +# Terminology + +## Service Worker + +A [service worker](http://www.html5rocks.com/en/tutorials/service-worker/introduction/) +is a background script that intercepts network requests made by your web app. +It can use the [Cache Storage API](https://developer.mozilla.org/en-US/docs/Web/API/Cache) +to respond to those requests. + +You do not have to write your own service worker script; this guide will explain +how to generate one customized for your web app. + +## App Shell + +"App Shell" refers to the local resources that your web app needs to load its +basic structure. This will always include some HTML, and will likely also +include CSS and JavaScript, either inline or in external files. + +Some static web apps consist entirely of an App Shell. + +A helpful analogy is to think of your App Shell as the code and resources that +would be published to an app store for a native iOS or Android application. + +The App Shell should ideally be loaded directly from the local cache, just like +a native iOS or Android application is loaded directly from a device's storage. + +## Dynamic Content + +The dynamic content is all of the data, images, and other resources that your +web app needs to function, but exists independently from your App Shell. + +Sometimes this data will come from external, third-party APIs, and sometimes +this will be first-party data that is dynamically generated or frequently +updated. + +For example, if your web app is for a newspaper, it might make use of a +first-party API to fetch recent articles, and a third-party API to fetch the +current weather. Both of those types of requests fall into the category of +"dynamic content". + +Unlike the App Shell, dynamic content is usually ephemeral, and it's important +to choose the right caching strategy for each source of dynamic content. + +## Caching Strategy + +You should always use a +[cache-first strategy](https://jakearchibald.com/2014/offline-cookbook/#cache-falling-back-to-network) +for your App Shell. `sw-precache` handles the details of that for you. + +However, the right caching strategy for your dynamic content is not always +clear-cut. It's recommended that you read through the +[The Offline Cookbook](https://jakearchibald.com/2014/offline-cookbook/) and +think about which strategy provides the right balance between speed and data +freshness for each of your data sources. + +Regardless of which strategy you choose, `sw-precache` handles the +implementation for you. All of the standard caching strategies, along with +control over advanced options like maximum cache size and age, are supported via +the automatic inclusion of the `sw-toolbox` library. + +# Add `sw-precache` to Your Build + +## Automation + +`sw-precache` should be automated to run as part of your site's existing build +process. It's important that it's re-run each time any of your App Shell +resources change, so that it can pick up the latest versions. + +It is available as a Node module for use in [Gulp](http://gulpjs.com/), +[Grunt](http://gruntjs.com/), or other Node-based build systems. It is also +available as a command-line binary, suitable for inclusion as part of an +[`npm`-based build](https://gist.github.com/addyosmani/9f10c555e32a8d06ddb0). + +## Configuration + +### Basic + +A basic configuration for a web app that relies entirely on local resources, all +located as subdirectories of an `app` directory, might look like: + +```js +{ + staticFileGlobs: ['app/**/*.{js,html,css,png,jpg,gif}'], + stripPrefix: 'app', + // ...other options as needed... +} +``` + +### Runtime Caching for Dynamic Content + +Once you've chosen an appropriate caching strategy to use for your dynamic +content, you can tell `sw-precache` which +[strategies](https://github.com/GoogleChrome/sw-toolbox#built-in-handlers) to +use for runtime requests that match specific URL patterns: + +```js +{ + runtimeCaching: [{ + urlPattern: new RegExp('^https://example\.com/api'), + handler: 'networkFirst' + }, { + urlPattern: new RegExp('/articles/'), + handler: 'fastest', + options: { + cache: { + maxEntries: 10, + name: 'articles-cache' + } + } + }], + // ...other options as needed... +} +``` + +If you use the `runtimeCaching` option, `sw-precache` will automatically include +the [`sw-toolbox` library](https://github.com/GoogleChrome/sw-toolbox) and the +corresponding [routing configuration](https://github.com/GoogleChrome/sw-toolbox#basic-routes) +in the service worker file that it generates. + +### Server-side Templating + +If your web app relies on server-side templating to use several partial files to +construct your App Shell's HTML, it's important to let `sw-precache` know about +those dependencies. + +For example, if your web app has two pages, `/home` and `/about`, each of which +depends on a both a shared master template and a page-specific template, you can +represent those dependencies as follows: + +```js +{ + dynamicUrlToDependencies: { + '/home': ['templates/master.hbs', 'templates/home.hbs'], + '/about': ['templates/master.hbs', 'templates/about.hbs'] + }, + // ...other options as needed... +} +``` + +### Fallback URL + +A common pattern when developing +[single page applications](https://en.wikipedia.org/wiki/Single-page_application) +(SPAs) is to bootstrap initial navigations with an App Shell, and then load +dynamic content based on URL routing rules. `sw-precache` supports this with the +concept of a "fallback URL": + +```js +{ + navigateFallback: '/app-shell', + // ...other options as needed... +} +``` + +In this configuration, whenever the service worker intercepts a +[navigate request](https://fetch.spec.whatwg.org/#concept-request-mode) for a +URL that doesn't exist in the cache, it will respond with the cached contents of +`/app-shell`. It's up to you to ensure that `/app-shell` contains all of the +resources needed to bootstrap your SPA. + +# Examples + +There are several ready-made examples of varying complexity that use +`sw-preache` as part of their build process: + +- https://github.com/GoogleChrome/sw-precache/tree/master/demo +- https://github.com/GoogleChrome/sw-precache/tree/master/app-shell-demo +- https://github.com/GoogleChrome/application-shell +- https://github.com/GoogleChrome/ioweb2015 + +# Other Resources + +## Articles +- [Service Workers in Production](https://developers.google.com/web/showcase/case-study/service-workers-iowa) +- [Instant Loading Web Apps with An Application Shell Architecture +](https://developers.google.com/web/updates/2015/11/app-shell) +- [Offline Cookbook](https://jakearchibald.com/2014/offline-cookbook/) + +## Videos +- [Instant Loading with Service Workers (Chrome Dev Summit 2015)](https://www.youtube.com/watch?v=jCKZDTtUA2A) diff --git a/README.md b/README.md index 6327e78..d4e45a6 100644 --- a/README.md +++ b/README.md @@ -9,8 +9,45 @@ precaches resources. The module is designed for use with [`gulp`](http://gulpjs.com/) or [`grunt`](http://gruntjs.com/) build scripts, though it also provides a command-line interface. The module's API provides methods for creating a service worker and saving the resulting code to a file. -Everything you need to get started is in this readme. Those of you who want more -depth can read the [background doc](background.md). +The full documentation is in this README, and the +[getting started guide](GettingStarted.md) provides a quicker jumping off point. + + +# Table of Contents + + + + +- [Install](#install) +- [Usage](#usage) + - [Overview](#overview) + - [Example](#example) + - [Considerations](#considerations) + - [Command-line interface](#command-line-interface) +- [API](#api) + - [Methods](#methods) + - [generate(options, callback)](#generateoptions-callback) + - [write(filePath, options, callback)](#writefilepath-options-callback) + - [Options Parameter](#options-parameter) + - [cacheId [String]](#cacheid-string) + - [directoryIndex [String]](#directoryindex-string) + - [dynamicUrlToDependencies [Object⟨String,Array⟨String⟩⟩]](#dynamicurltodependencies-object&x27e8stringarray&x27e8string&x27e9&x27e9) + - [handleFetch [boolean]](#handlefetch-boolean) + - [ignoreUrlParametersMatching [Array⟨Regex⟩]](#ignoreurlparametersmatching-array&x27e8regex&x27e9) + - [importScripts [Array⟨String⟩]](#importscripts-array&x27e8string&x27e9) + - [logger [function]](#logger-function) + - [maximumFileSizeToCacheInBytes [Number]](#maximumfilesizetocacheinbytes-number) + - [navigateFallback [String]](#navigatefallback-string) + - [replacePrefix [String]](#replaceprefix-string) + - [runtimeCaching [Array⟨Object⟩]](#runtimecaching-array&x27e8object&x27e9) + - [staticFileGlobs [Array⟨String⟩]](#staticfileglobs-array&x27e8string&x27e9) + - [stripPrefix [String]](#stripprefix-string) + - [templateFilePath [String]](#templatefilepath-string) + - [verbose [boolean]](#verbose-boolean) +- [Acknowledgements](#acknowledgements) +- [License](#license) + + ## Install @@ -268,6 +305,43 @@ and replace it with '/public/'. _Default_: `''` +#### runtimeCaching [Array⟨Object⟩] +Configures runtime caching for dynamic content. If you use this option, the `sw-toolbox` +library configured with the caching strategies you specify will automatically be included in +your generated service worker file. + +Each `Object` in the `Array` needs a `urlPattern`, which is either a +[`RegExp`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp) +or a string, following the conventions of the `sw-toolbox` library's +[routing configuration](https://github.com/GoogleChrome/sw-toolbox#basic-routes). Also required is +a `handler`, which should be either a string corresponding to one of the +[built-in handlers](https://github.com/GoogleChrome/sw-toolbox#built-in-handlers) under the `toolbox.` namespace, or a function corresponding to your custom +[request handler](https://github.com/GoogleChrome/sw-toolbox#defining-request-handlers). There is also +support for `options`, which corresponds to the same options supported by a +[`sw-toolbox` handler](https://github.com/GoogleChrome/sw-toolbox#options). + +For example, the following defines runtime caching behavior for two different URL patterns. It uses a +different handler for each, and specifies a dedicated cache with maximum size for requests +that match `/articles/`: + +```js +runtimeCaching: [{ + urlPattern: new RegExp('^https://example\.com/api'), + handler: 'networkFirst' +}, { + urlPattern: new RegExp('/articles/'), + handler: 'fastest', + options: { + cache: { + maxEntries: 10, + name: 'articles-cache' + } + } +}] +``` + +_Default_: `[]` + #### staticFileGlobs [Array⟨String⟩] An array of one or more string patterns that will be passed in to [`glob`](https://github.com/isaacs/node-glob). diff --git a/background.md b/background.md deleted file mode 100644 index aebebe4..0000000 --- a/background.md +++ /dev/null @@ -1,60 +0,0 @@ -# tl;dr -This project is an exploration into integrating service worker-based caching patterns into [`gulp`](http://gulpjs.com/) or [`grunt`](http://gruntjs.com/) build scripts. If you have a website, adding in offline-first support should be as easy as adding some additional logic to your build process. - -### What's all this, then? -[Service workers](https://slightlyoff.github.io/ServiceWorker/spec/service_worker/index.html) give JavaScript developers almost complete control over a browser's network stack. There are a number of [patterns](http://jakearchibald.com/2014/offline-cookbook/) around offline use cases, and one of the most useful is [cache on install as a dependency](http://jakearchibald.com/2014/offline-cookbook/#on-install-as-a-dependency). The first time a user visits your page using a browser that supports service workers, all of the resources needed to use the page offline can be automatically cached locally, and each subsequent visit to any page on the site will be a) fast (since there's no network dependency) and b) work offline (for the same reason). - -### Great! As a developer, what do I have to do? -Here's a code snippet from [@jakearchibald](https://github.com/jakearchibald)'s post describing [cache on install as a dependency](http://jakearchibald.com/2014/offline-cookbook/#on-install-as-a-dependency): - - self.addEventListener('install', function(event) { - event.waitUntil( - caches.open('mysite-static-v3').then(function(cache) { - return cache.addAll([ - '/css/whatever-v3.css', - '/css/imgs/sprites-v6.png', - '/css/fonts/whatever-v8.woff', - '/js/all-min-v4.js' - // etc - ]); - }) - ); - }); - -There are two difficulties here that someone implementing this pattern with a real site would face: - -- `mysite-static-v3` represents the name of the [cache](https://slightlyoff.github.io/ServiceWorker/spec/service_worker/index.html#cache-objects) that will store and serve the resources, and in particular, `v3` represents the unique version that's "current" for your site. As a developer, how do you handle cache versioning—what if you forget to bump up the version number for each of your named caches, and existing users of your site never pick up the changes to the cached resources that you meant to deploy? - -- There's a small list of resources, followed by an `// etc`. For a site of any significant size, that `// etc` contains multitudes. What if you forget to add in that additional image, or forgot to switch in the latest version of your minified JavaScript? - -### So what does this project aim to do? -Versioning and generating lists of local files are both solved problems, and `gulp` in particular (which you're hopefully using already as part of your build tooling) is ideal for that purpose. What's missing is something to tie the things that `gulp` can do in with the service worker logic to ensure that your cache versioning is always correct, and your lists of resources for each cache are always up to date. This project is an exploration of one approach to automating that process. - -(**Update**: The code has recently been refactored to be a standalone node module, with the goal of making the output work equally well as part of a `gulp` or `grunt` build process.) - -### How's it all work? - -Inside the sample [`gulpfile.js`](https://github.com/googlechrome/gulp-sw-precache/blob/master/demo/gulpfile.js), there's a list of [glob patterns](https://github.com/isaacs/node-glob) corresponding to static files, as well as a mapping of server-generated resource URLs to the component files that are used to generated that URL's output: - - dynamicUrlToDependencies: { - 'dynamic/page1': [rootDir + '/views/layout.jade', rootDir + '/views/page1.jade'], - 'dynamic/page2': [rootDir + '/views/layout.jade', rootDir + '/views/page2.jade'] - }, - staticFileGlobs: [ - rootDir + '/css/**.css', - rootDir + '/**.html', - rootDir + '/images/**.*', - rootDir + '/js/**.js' - ], - -For each of those entries, there's code to expand the glob pattern and calculate the [MD5 hash](http://en.wikipedia.org/wiki/MD5) of the contents of each file. The MD5 hash along with the file's relative path is used to uniquely name the [cache](https://slightlyoff.github.io/ServiceWorker/spec/service_worker/index.html#cache-objects) entry that will be used to store that resource. If the process runs again and an existing file's contents change, or a new file is added or an existing file is deleted, those changes will result in a different list of cache names being generated. Conversely, if the process runs agains and every file is exactly the same as last time, then the same MD5 hashes and names *should* be generated, and the service worker won't attempt to update anything. - -Regardless of what files we're working with, most of the logic stays the same—there's an `install` handler that caches anything that isn't already present in the cache (using the `hash` to determine whether the exact version of each resource is present or not), and an `activate` handler that takes care of deleting old caches that we no longer need. There's also a `fetch` handler that attempst to serve each response from the cache, falling back to the network only if a given resource URL isn't present. - -### Try it out! -Clone this repo, run `npm install`, and then `gulp serve-dist`. Take a look at the contents of the generated `dist` directory. Go to `http://localhost:3000` using Chrome 40 or newer (I prefer testing using [Chrome Canary](https://www.google.com/chrome/browser/canary.html)). Visit `chrome://serviceworker-internals` and check out the logged activity for the registered service worker, as well as the service worker cache inspector. - -Try changing some files in `dist` and then running `gulp generate-service-worker-dist` to generate a new `dist/service-worker.js` file, then close and re-open `http://localhost:3000`. Examine the logging via `chrome://serviceworker-internals` and notice how the service worker cache inspector has updated to include the latest set named caches. - -### Feedback, please! -There are a lot of `TODO`s in the code where I've hacked something together that seems to work, but given my rather limited experience working with node modules, may not be the right approach. I'd love to hear suggestions about improving that. And in general, let me know if this seems like something you'd find useful (or if you even do actually start using it)! diff --git a/demo/app/images/runtime-caching/four.png b/demo/app/images/runtime-caching/four.png new file mode 100644 index 0000000..c6bf900 Binary files /dev/null and b/demo/app/images/runtime-caching/four.png differ diff --git a/demo/app/images/runtime-caching/three.png b/demo/app/images/runtime-caching/three.png new file mode 100644 index 0000000..7290c6e Binary files /dev/null and b/demo/app/images/runtime-caching/three.png differ diff --git a/demo/gulpfile.js b/demo/gulpfile.js index eea49f5..3459517 100644 --- a/demo/gulpfile.js +++ b/demo/gulpfile.js @@ -52,6 +52,18 @@ function writeServiceWorkerFile(rootDir, handleFetch, callback) { // local changes from being picked up during the development cycle. handleFetch: handleFetch, logger: $.util.log, + runtimeCaching: [{ + // See https://github.com/GoogleChrome/sw-toolbox#methods + urlPattern: /runtime-caching/, + handler: 'cacheFirst', + // See https://github.com/GoogleChrome/sw-toolbox#options + options: { + cache: { + maxEntries: 1, + name: 'runtime-cache' + } + } + }], staticFileGlobs: [ rootDir + '/css/**.css', rootDir + '/**.html', diff --git a/gulpfile.js b/gulpfile.js index 4a64fdb..92299c5 100644 --- a/gulpfile.js +++ b/gulpfile.js @@ -10,8 +10,7 @@ gulp.task('default', ['test', 'lint']); gulp.task('generate-demo-service-worker', function(callback) { spawn('gulp', ['--cwd', 'demo', 'generate-service-worker-dev'], - {stdio: 'inherit'}) - .on('close', callback); + {stdio: 'inherit'}).on('close', callback); }); gulp.task('lint', ['generate-demo-service-worker'], function() { @@ -30,6 +29,15 @@ gulp.task('test', function() { }); }); +gulp.task('update-markdown-toc', function(callback) { + // doctoc only exposes a binary in node_modules/.bin/doctoc, without a + // corresponding API. To make starting this up a little bit nicer, there's a + // npm script defined in package.json, which let's us use + // 'npm run doctoc ' + spawn('npm', ['run', 'doctoc', '--', 'GettingStarted.md', 'README.md'], + {stdio: 'inherit'}).on('close', callback); +}); + gulp.task('publish', ['test', 'lint'], function(callback) { spawn('npm', ['publish'], {stdio: 'inherit'}) .on('close', callback); diff --git a/lib/sw-precache.js b/lib/sw-precache.js index 57731fc..f84e16a 100644 --- a/lib/sw-precache.js +++ b/lib/sw-precache.js @@ -169,6 +169,43 @@ function generate(params, callback) { } }); + var runtimeCaching; + var swToolboxCode; + if (params.runtimeCaching) { + var pathToSWToolbox = require.resolve('sw-toolbox/sw-toolbox.js'); + swToolboxCode = fs.readFileSync(pathToSWToolbox, 'utf8'); + + runtimeCaching = params.runtimeCaching.reduce(function(prev, curr) { + var line; + if (curr.default) { + line = util.format('\ntoolbox.router.default = toolbox.%s;', + curr.default); + } else { + if (!(curr.urlPattern instanceof RegExp || + typeof curr.urlPattern === 'string')) { + throw new Error( + 'runtimeCaching.urlPattern must be a string or RegExp'); + } + + line = util.format('\ntoolbox.router.%s(%s, %s, %s);', + // Default to setting up a 'get' handler. + curr.method || 'get', + // urlPattern might be a String or a RegExp. sw-toolbox supports both. + curr.urlPattern, + // If curr.handler is a string, then assume it's the name of one + // of the built-in sw-toolbox strategies. + // E.g. 'networkFirst' -> toolbox.networkFirst + // If curr.handler is something else (like a function), then just + // include its body inline. + (typeof curr.handler === 'string' ? 'toolbox.' : '') + curr.handler, + // Default to no options. + JSON.stringify(curr.options || {})); + } + + return prev + line; + }, ''); + } + // It's very important that running this operation multiple times with the same input files // produces identical output, since we need the generated service-worker.js file to change iff // the input files changes. The service worker update algorithm, @@ -206,6 +243,8 @@ function generate(params, callback) { // Ensure that anything false is translated into '', since this will be treated as a string. navigateFallback: params.navigateFallback || '', precacheConfig: JSON.stringify(precacheConfig), + runtimeCaching: runtimeCaching, + swToolboxCode: swToolboxCode, version: VERSION }); diff --git a/package.json b/package.json index f543fb2..71f111a 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "sw-precache", - "version": "2.3.0", + "version": "3.0.0", "description": "Generate service worker code that will precache specific resources.", "author": { "name": "Jeff Posnick", @@ -21,6 +21,7 @@ }, "devDependencies": { "del": "^1.2.0", + "doctoc": "^1.0.0", "eslint": "^1.10.3", "eslint-config-google": "^0.3.0", "express": "^4.12.4", @@ -43,13 +44,15 @@ "lodash.template": "^3.6.1", "meow": "^3.3.0", "mkdirp": "^0.5.1", - "pretty-bytes": "^2.0.1" + "pretty-bytes": "^2.0.1", + "sw-toolbox": "^3.1.1" }, "repository": "googlechrome/sw-precache", "bugs": "https://github.com/googlechrome/sw-precache/issues", "license": "Apache-2.0", "scripts": { - "test": "gulp test lint" + "test": "gulp test lint", + "doctoc": "doctoc" }, "files": [ "cli.js", diff --git a/service-worker.tmpl b/service-worker.tmpl index ae59ac2..b9e008e 100644 --- a/service-worker.tmpl +++ b/service-worker.tmpl @@ -24,6 +24,12 @@ /* eslint-disable indent, no-unused-vars, no-multiple-empty-lines, max-nested-callbacks, space-before-function-paren */ 'use strict'; +<% if (handleFetch && swToolboxCode) { %> +// *** Start of auto-included sw-toolbox code. *** +<%= swToolboxCode %> +// *** End of auto-included sw-toolbox code. *** +<% } %> + <% if (importScripts) { %> importScripts(<%= importScripts %>); <% } %> @@ -169,4 +175,10 @@ self.addEventListener('fetch', function(event) { } } }); + +<% if (runtimeCaching) { %> +// Runtime cache configuration, using the sw-toolbox library. +<%= runtimeCaching %> +<% } %> + <% } %>