Repository: matthew-andrews/workshop-making-it-work-offline Branch: master Commit: a5d9c55362d3 Files: 136 Total size: 277.6 KB Directory structure: gitextract_2g5j4x9v/ ├── .gitignore ├── .jshintrc ├── 01-introduction/ │ ├── README.md │ ├── dysfunctional-family.md │ ├── how.md │ └── why.md ├── 02-new-apis/ │ ├── README.md │ ├── fetch.md │ └── solutions.md ├── 03-offline-todo/ │ ├── 01-scaffolding/ │ │ ├── README.md │ │ ├── application.js │ │ ├── index.html │ │ ├── offline.appcache │ │ ├── promise.js │ │ └── styles.css │ ├── 02-opening-a-database/ │ │ ├── README.md │ │ ├── application.js │ │ ├── index.html │ │ ├── promise.js │ │ └── styles.css │ ├── 03-using-dev-tools/ │ │ └── README.md │ ├── 04-creating-object-stores/ │ │ ├── README.md │ │ ├── application.js │ │ ├── index.html │ │ ├── promise.js │ │ └── styles.css │ ├── 05-review-window-indexeddb/ │ │ └── README.md │ ├── 06-adding-data/ │ │ ├── README.md │ │ ├── application.js │ │ ├── index.html │ │ ├── promise.js │ │ └── styles.css │ ├── 07-getting-data/ │ │ ├── README.md │ │ ├── application.js │ │ ├── index.html │ │ ├── promise.js │ │ └── styles.css │ ├── 08-rendering-todos/ │ │ ├── README.md │ │ ├── application.js │ │ ├── index.html │ │ ├── promise.js │ │ └── styles.css │ ├── 09-deleting-data/ │ │ ├── README.md │ │ ├── application.js │ │ ├── index.html │ │ ├── promise.js │ │ └── styles.css │ ├── 10-review-requests-transactions/ │ │ └── README.md │ ├── 11-appcache/ │ │ ├── README.md │ │ ├── application.js │ │ ├── index.html │ │ ├── offline.appcache │ │ ├── promise.js │ │ └── styles.css │ ├── 12-appcache-gotcha-1/ │ │ └── README.md │ ├── 13-success/ │ │ └── README.md │ └── README.md ├── 04-offline-todo-with-sync/ │ ├── 01-architecture/ │ │ └── README.md │ ├── 02-mark-for-deletion/ │ │ ├── README.md │ │ ├── application.js │ │ ├── index.html │ │ ├── offline.appcache │ │ ├── promise.js │ │ └── styles.css │ ├── 03-adding-ajax/ │ │ ├── README.md │ │ ├── application.js │ │ ├── fetch.js │ │ ├── index.html │ │ ├── offline.appcache │ │ ├── promise.js │ │ └── styles.css │ ├── 04-synchronize/ │ │ ├── README.md │ │ ├── application.js │ │ ├── fetch.js │ │ ├── index.html │ │ ├── offline.appcache │ │ ├── promise.js │ │ └── styles.css │ ├── 05-success/ │ │ └── README.md │ └── README.md ├── 05-offline-news/ │ ├── 01-scaffolding/ │ │ ├── README.md │ │ ├── index.js │ │ ├── package.json │ │ └── public/ │ │ ├── styles.css │ │ └── templates.js │ ├── 02-single-multi-page/ │ │ ├── README.md │ │ ├── index.js │ │ ├── package.json │ │ └── public/ │ │ ├── application.js │ │ ├── fetch.js │ │ ├── promise.js │ │ ├── styles.css │ │ └── templates.js │ ├── 03-hacking-appcache/ │ │ ├── README.md │ │ ├── index.js │ │ ├── package.json │ │ └── public/ │ │ ├── application.js │ │ ├── fetch.js │ │ ├── iframe.html │ │ ├── promise.js │ │ ├── styles.css │ │ └── templates.js │ ├── 04-more-hacking-appcache/ │ │ ├── README.md │ │ ├── index.js │ │ ├── package.json │ │ └── public/ │ │ ├── appcache.js │ │ ├── application.js │ │ ├── fetch.js │ │ ├── iframe.html │ │ ├── iframe.js │ │ ├── promise.js │ │ ├── styles.css │ │ └── templates.js │ ├── 05-success/ │ │ └── README.md │ └── README.md ├── 06-offline-news-with-service-worker/ │ ├── 01-scaffolding/ │ │ ├── README.md │ │ ├── index.js │ │ ├── package.json │ │ └── public/ │ │ ├── styles.css │ │ └── templates.js │ ├── 02-registering-a-service-worker/ │ │ ├── README.md │ │ ├── index.js │ │ ├── package.json │ │ ├── public/ │ │ │ ├── application.js │ │ │ ├── service-worker.js │ │ │ ├── styles.css │ │ │ └── templates.js │ │ ├── styles.css │ │ └── templates.js │ ├── 03-service-worker-caches/ │ │ └── README.md │ ├── 04-success/ │ │ └── README.md │ └── README.md ├── 07-dexie/ │ └── README.md ├── 08-success/ │ └── README.md ├── README.md └── package.json ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gitignore ================================================ /node_modules/ ================================================ FILE: .jshintrc ================================================ { "browser": true, "laxbreak": true, "debug": true } ================================================ FILE: 01-introduction/README.md ================================================ # Introduction Today we're going to explore all the technologies available in browsers today that can be brought together to build **offline websites**. ## Me - Core dev FT web app, Economist HTML5 app (both successful offline-first applications) - Author of tutorial series: [How to make an offline HTML5 web app, FT style](http://labs.ft.com/2012/08/basic-offline-html5-web-app/) - [Building A Simple Cross-Browser Offline To-Do List With IndexedDB And WebSQL on Smashing Magazine](http://www.smashingmagazine.com/2014/09/02/building-simple-cross-browser-offline-todo-list-indexeddb-websql/) ## You - Introduction yourselves - How would you describe your JavaScript skills? - What do you hope to gain from today? (Mostly to learn about techniques to use today, or one that will be usable in a few month's time?) ## Course structure The structure of this course will be very practical and hands-on focused on building prototypes, and will typically follow this pattern: - Hack something together that works. - Talk about why it works (explaining any browser APIs that we've used that may be unfamiliar). - Discuss why it's inefficient or inelegant and then improve on it. - There are exercises at the end of many sections if you finish the core material quickly. - Schedule for the day: 9 start, 10.30 break (½ hour), 12.30 lunch (1 hour), 15.00 break (½ hour), 17.00 the end. ## What you should get out of today's course - A few working prototypes. - All the prototypes are on my GitHub (publicly so if you have any problems with any after today - please feel free to raise issues there and I will try to find time to help) ## Group chat http://tlk.io/ --- [← Back to *contents*](https://github.com/matthew-andrews/workshop-making-it-work-offline) | [Continue to *why* →](./why.md) ================================================ FILE: 01-introduction/dysfunctional-family.md ================================================ # Meet the dysfunctional family ## Quiz - Can you name all the offline storage technologies that exist in browsers today? - What are the differences between them? - What sort of data can you store offline? - Which technologies allow you to shared data between domain names? - What are the rules for offline technologies that don't share data across domains? How is https affected this? How about subfolders? - If you built an offline web application and then migrated to https what potential problems might you need to consider? ## Choices - Cookies - Local Storage / Session Storage - WebSQL - IndexedDB - Browser Cache - Application Cache - Service Worker ### Cookies - Basically work on *every* browser - Only supports very small pieces of information - Only supports strings - Sent on every http request - A bit fiddly to use without a helper library/function ### Local Storage - Works on a lot of browsers - Supports more data than cookies - But still only a small amount of space available on most browsers - Very *fast* - Only supports strings - Not sent on HTTP request (only available client side) - Synchronous, which can cause performance issues ### Local database (WebSQL, IndexedDB) - One or the other (or both) work on most modern browsers - Supports more data than local storage (typically around ~50mb, but can be much bigger on Desktop) - Supports complex data structures and multiple data types - Built in mechanism for migrations - Slower than Local Storage - Not sent on http request (only available client side) - Asynchronous ### Browser Cache - Works on every browser - *Sometimes* does work offline (which can be confusing when trying to test AppCache issues) but… - It can't be relied on ### Application Cache - Works on most (90%+) modern browsers - Best suited for storing application *code* - The only option to reliably load a website from nothing offline - Can storage a significant amount of data (50mb+) - Very difficult use without causing unintended consequences for most websites ### Service Worker - Application Cache's replacement, but unlikely to be possible to polyfill - Not available in any browser (bits of it work on some cutting-edge browsers with feature flags switched on) - Safari / Microsoft have not announced any intention to implement it - Will only work on https - Promises to be less difficult to use and cause fewer side effects than Application Cache Note. There was another storage technology relevant to offline websites called the HTML5 FileSystem API but as of April 2014, [that spec is dead](http://www.html5rocks.com/en/tutorials/file/filesystem/) #### Q. Which technology do you need to make an offline-first web application? A. Most, if not all, of them. For today's browsers I advocate the following approach and the reasons why will hopefully become clear over the course of the workshop: ### Guidance - **Delivering the application** - we will use the Application Cache to store the JavaScript, HTML and CSS to launch our application. We will not store any content here. - **Storing content** - we will use a clientside database technology, either IndexedDB or WebSQL, to store our content. We will not store any application assets here. (Note: when we are able to use Service Worker we will be able to take a more flexible approach) --- [← Back to *how*](./how.md) | [Continue to *a promising start* →](../02-new-apis) ================================================ FILE: 01-introduction/how.md ================================================ # How There are four distinct problems to solve*: 1. Delivering the application 2. Storing data 3. Syncing data 4. Third parties ## Delivering the application - How do we efficiently cache enough of the pieces that make up a website for it to be useful offline? - How do we guarantee that the browser won't delete them? ## Storing content - How do we store content downloaded to or created on users' devices? ## Synchronizing content - How do we ensure the data on users' devices are kept in sync with content on the server? - What if there are conflicts? ## Third parties One of the great things about the web is how easy it is to bring multiple products, widgets and services together onto a single page. Offline technologies have only just started to be developed - using them enable us to create incredible user experience that previously would have been impossible on the web - but, also by using them, you will discover many of the third party components you relied on to build websites don't work well offline. ### What happens to Google Analytics data? Basically, nothing. - For each tracking event Google Analytics will try to send a tracking event to the server. - As the device is offline that tracking event will fail. - Google Analytics will not retry or store that data locally for retry later. - **If you want to keep that data, you will have to implement the storing and resending of it yourself.** - Even, even if you _did_ store that data and sent it later manually yourself, as the web requires your website to be open in order to make http requests that data might only be sent after a very long time after it was recorded - _perhaps even weeks or months_. \* [Adapted from Caolan McMahon's presentation on the status of offline web](http://www.infoq.com/presentations/status-web-offline) [← Back to *why*](./why.md) | [Continue to *meet the dysfunctional family* →](./dysfunctional-family.md) ================================================ FILE: 01-introduction/why.md ================================================ # Why We live in a bubble. We work all day on high-spec machines, plugged into high-speed connections and use the latest cut of our favourite web browser. The real world isn't like that. Taking just network connection as one example. It's no longer as simple as online-versus-offline. - Connected to a high speed fibre connection via wifi every few minutes for less than a minute (the London underground) - In a region where the cost of using the local network is so high the user has _chosen_ not to be connected. - Connected to a weak wireless signal where only a small number of requests are successful. - Using a connection where only a subset of websites are blocked (an airport or in China - it's possible to make your website nearly unusable in China by poorly integration of a Facebook widget) - Device prefer to connect to wifi hotspots that require logins, even when they have perfectly good mobile connections. ## People need the web offline Whilst the web is slowly reaching ever more remote locations - wifi on planes and underground - the amount by which we depend on it is growing even faster. As developers we need to do a better job. [← Back to *introduction*](./README.md) | [Continue to *how* →](./how.md) ================================================ FILE: 02-new-apis/README.md ================================================ # Promises ## What is a Promise? > The Promise interface **represents a proxy for a value not necessarily known when the promise is created**. It allows you to associate handlers to an asynchronous action's eventual success or failure. **This lets asynchronous methods return values like synchronous methods**: instead of the final value, the asynchronous method returns a promise of having a value at some point in the future. > > A pending promise can become either **fulfilled with a value**, or **rejected with a reason**. When either of these happens, the associated handlers queued up by a promise's then method are called. (If the promise has already been fulfilled or rejected when a corresponding handler is attached, the handler will be called, so there is no race condition between an asynchronous operation completing and its handlers being attached.) > > As the `Promise.prototype.then` and `Promise.prototype.catch` methods return promises, they can be chained—an operation called composition. \- Source: [MDN](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise), emphasis mine. In short, Promises are a way to organise asynchronous logic. ## Lots of Promise libraries Or similar libraries that achieve the same thing:- - Q - async - when - RSVP But the implementation we're going to use is **ES6 Promise**. It's a browser standard that is already available and also works in Node. ## Why are we talking about it? We're going to need Promises because new browser APIs, such as **Service Worker** which we will cover later, are built on top of them. By covering them early we will be able to use them in all of our prototypes and exercises so that hopefully they will be second nature (if they aren't already) by the end of the day. Take for example messy logic that **POSTs** two todos to the server and opens an alert when both have succeeded (or an alert if one fails). ```js request({ url: "https://offline-todo-api.herokuapp.com/todos", body: { _id: 'my-first-todo', text: 'Buy some bread' }, method: 'POST' }, function(err) { var firstSuccess = !!err; request({ url: "https://offline-todo-api.herokuapp.com/todos", body: { _id: 'my-second-todo', text: 'Buy some bread' }, method: 'POST' }, function(err) { var secondSuccess = !!err; if (firstSuccess && secondSuccess) { alert('success!'); } else { alert('something failed'); } }); }); ``` ## What's wrong with this code? - Difficult to follow - The second request doesn't start until the first is complete (they could be running at the same time) - Error handling is messy ## What are we aiming for? If we didn't need to worry about the asynchronous ajax request we'd probably write something like this:- ```js try { var responses = []; responses[0] = request({ url: "https://offline-todo-api.herokuapp.com/todos", body: { _id: 'my-first-todo', text: 'Buy some bread' }, method: 'POST' }); responses[1] = request({ url: "https://offline-todo-api.herokuapp.com/todos", body: { _id: 'my-second-todo', text: 'Buy some bread' }, method: 'POST' }); alert('success!'); } catch(e) { alert('something failed'); } ``` ### Promises let us write code like this. ```js Promise.all( [ request({ url: "https://offline-todo-api.herokuapp.com/todos", body: { _id: 'my-first-todo', text: 'Buy some bread' }, method: 'POST' }), request({ url: "https://offline-todo-api.herokuapp.com/todos", body: { _id: 'my-second-todo', text: 'Buy some bread' }, method: 'POST' }) ] ).then(function success(responses) { alert('success!'); }, function failure() { alert('something failed'); }); ``` ### Nice things about promises - Abstracts the complexity of dealing with asynchronous logic - lets you write performant but (relatively) simple code. - Allows you to deal with errors in asynchronous libraries in a sensible way. ### Maybe not so nice things about promises - Most browser methods were made pre-promises and so need wrapping to be useful. - Cannot pipe data. (eg. it might be nice for ajax request that returned a lot of data to start passing that data to the application as soon as data starts being received. With promises you must wait until *all the data arrives* before your application will get to see *any of it*. ### Like them or not they're here to stay - Browsers now support them natively, Node v0.11 ships with them built in - Old familiar APIs (e.g. XMLHttpRequest) may be replaced by new shiny promising versions (e.g. request) - They are a fundamental building block to ServiceWorkers, which will be a key component of today's workshop - Many, many languages support promises (even [ReactPHP](https://github.com/reactphp/promise)) or equivalents (sometimes called futures - promises were actually first proposed in 1976) ### Getting setup for Promises ``` mkdir promises cd promises echo '{}' >> package.json npm install --save es6-promise ``` Then, in that folder create a file called `test.js` with the following contents:- ```js require('es6-promise').polyfill(); console.log(Promise); ``` Hopefully the output will be:- ``` { [Function: Promise] all: [Function: all], race: [Function: race], resolve: [Function: resolve], reject: [Function: reject] } ``` ### Exercises Create a function that returns a promise that successfully resolves after 1 second with `setTimeout` --- Create a function that returns a promise that rejects a promise after half a second with `setTimeout` --- Explain the difference between this:- ```js promise .then(function() { alert("Success!"); }, function() { alert("Fail!"); }); ``` And this:- ```js promise .then(function() { alert("Success!"); }) .catch(function() { alert("Fail!"); }); ``` --- Explain the difference between this:- ```js Promise.all([promise1, promise2]) .then(function() { alert("Success!"); }); ``` And this:- ```js Promise.race([promise1, promise2]) .then(function() { alert("Success!"); }); ``` --- When might race be useful in an offline web application? --- [← Back to *meet the dysfunctional family*](../01-introduction/dysfunctional-family.md) | [Continue to *Fetch* →](./fetch.md) ================================================ FILE: 02-new-apis/fetch.md ================================================ # Fetch By now I hope I've convinced you that ES6 Promises are the one and only solution to asynchronous flow and that — if you're anything like me — will no longer be able to look at a web api built in the Pre-Promise era without cringing. Let's talk about ajax. ```js function success(response) { /* do something */ } var request = new XMLHttpRequest(); request.onreadystatechange = function() { if (request.readyState === 4) { if (request.status === 200) { success(request.response); } } }; request.open('https://offline-news-api.herokaupp.com/stories'); request.send(); ``` OK, to be fair nobody actually writes this. They'll normally do something like:- ```js function success(response) { /* do something */ } $.get('https://offline-news-api.herokaupp.com/stories', success); ``` Which is much better but what's the point of a standard API that is so difficult to use nobody uses it? What would be nice if browsers shipped with a simple, _Promise_ based API for _fetching_ ajax requests… ```js fetch('https://offline-news-api.herokaupp.com/stories') .then(success); ``` [Luckily other people think this is a good idea too](http://fetch.spec.whatwg.org/), and is the mechanism you *must* use within Service Worker, which we'll cover later. Oddly though Chrome hasn't made this available in the front end, however the lovely people at [GitHub](https://www.github.com/github/fetch) have built a polyfill for modern browsers. ## Gotchas The Fetch API builds on top of the W3C Streams _as well as_ Promises. This is good as there are times where you won't want to wait for an ajax request to finish loading before you want to use the data it's downloading — in these cases you can pipe that raw stream of downloaded data into different bits of your application. JavaScript streams are out of scope for this workshop but you can read more about them on the [Stream Handbook](https://github.com/substack/stream-handbook). Warning: the Fetch Polyfill _only_ implements the Promise part of the API, *not* the Streams part. The reason why I mention it is that the `fetch` API returns a Promise that resolves with a Response object that gives you access to *streams* — not an object containing all the data you requested. You need to then convert that into the data format you want. For example:- ```js fetch('http://mattandre.ws/my-json-file.json') .then(function(response) { return response.json(); }); ``` ## Exercises - Use the Fetch API to create a, delete a and download all the todos from the API described: https://github.com/matthew-andrews/offline-todo-api. --- [← Back to *Promises*](./) | [Continue to *offline todo with IndexedDB* →](../03-offline-todo) ================================================ FILE: 02-new-apis/solutions.md ================================================ # Solutions ```js function successful() { return new Promise(resolve, reject) { setTimeout(resolve, 1000); }; } ``` --- ```js function unsuccessful() { return new Promise(resolve, reject) { setTimeout(reject, 500); }; } ``` --- `then` actually takes two arguments - one for success, one for failure, and `catch(function() {})` is just shorthand for `then(undefined, function() {})`. So really we are comparing the difference between:- ```js promise .then(function() { alert("Success!"); }, function() { alert("Fail!"); }); ``` And this:- ```js promise .then(function() { alert("Success!"); }) .then(undefined, function() { alert("Fail!"); }); ``` When an error occurs promises _jump forward_ to the next `then` with a rejection callback. So with`.then(funcA, funcB)` either `funcA` or `funcB` will be called (not both) - whereas with `.then(funcA).catch(funcB)` both `funcA` and `funcB` will get called (as they're separate steps). [Read more detail about error handling in promises on HTML5 rocks](http://www.html5rocks.com/en/tutorials/es6/promises/#toc-error-handling) --- - `all` resolves when **all** the promises passed into it resolve. If any rejects, it immediately reject - discarding the results of all other promises, regardless of their result. - `race` resolves **as soon as one** of the promises resolves or rejects. Race could be useful in an offline web app when retrieving fresh data from a server that is also cached locally. ```js Promise.race([getFromServer(), getFromCache()]) .then(function() { // Put some data on screen }); ``` ================================================ FILE: 03-offline-todo/01-scaffolding/README.md ================================================ # Scaffolding the application We will create the following files in a single directory: - [`/index.html`](./index.html) - [`/application.js`](./application.js) - [`/indexeddb.shim.min.js`](./indexeddb.shim.min.js) - [`/promise.js`](./promise.js) - [`/styles.css`](./styles.css) - [`/offline.appcache`](./offline.appcache) ##### `/index.html` ```html

Example: Todo

``` Nothing surprising here: just a standard HTML web page, with an input field to add to-do items, and an empty unordered list that will be filled with those items. ##### `/indexeddb.shim.min.js` Download the contents of [the minified IndexedDB polyfill](https://raw.githubusercontent.com/matthew-andrews/offline-todo/gh-pages/indexeddb.shim.min.js), and put it in this file. ##### `/promise.js` Download the contents of [the minified ES6 Promise polyfill](http://s3.amazonaws.com/es6-promises/promise-1.0.0.min.js), and put it in this file. ##### `/styles.css` ```css body { margin: 0; padding: 0; font-family: helvetica, sans-serif; } * { box-sizing: border-box; } h1 { padding: 18px 20px; margin: 0; font-size: 44px; border-bottom: solid 1px #DDD; line-height: 1em; } form { padding: 20px; border-bottom: solid 1px #DDD; } input { width: 100%; padding: 6px; font-size: 1.4em; } ul { margin: 0; padding: 0; list-style: none; } li { padding: 20px; border-bottom: solid 1px #DDD; } button { float: right; } ``` Again, this should be quite familiar: just some simple styles to make the to-do list look tidy. You may choose not to have any styles at all or create your own. We will leave `application.js` and `offline.appcache` empty for now and return to them later. ## Quick test Run any simple static web server, for example [node-static](https://github.com/cloudhead/node-static), in the directory containing these files and verify the website matches the screenshot below. ![Screenshot of the scaffolded application](./screenshot.png) Warning: you will need to use `static -c 1` rather than `static` on its own in order to see updated files. Once we've added AppCache, try leaving off the `-c 1` and see what happens when you update files. --- [← Back to *Offline todo with IndexedDB*](../) | [Continue to *opening a database* →](../02-opening-a-database) ================================================ FILE: 03-offline-todo/01-scaffolding/application.js ================================================ ================================================ FILE: 03-offline-todo/01-scaffolding/index.html ================================================

Example: Todo

================================================ FILE: 03-offline-todo/01-scaffolding/offline.appcache ================================================ ================================================ FILE: 03-offline-todo/01-scaffolding/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j

Example: Todo

================================================ FILE: 03-offline-todo/02-opening-a-database/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j

Example: Todo

================================================ FILE: 03-offline-todo/04-creating-object-stores/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j

Example: Todo

================================================ FILE: 03-offline-todo/06-adding-data/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j

Example: Todo

================================================ FILE: 03-offline-todo/07-getting-data/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j'+todo.text+''; } […] ``` - `todoToHtml` takes a single todo and converts it to a HTML string. - `renderAllTodos` takes an array of todos, converts each of them HTML and then sets the `innerHTML` of the `ul` to those strings concatenated together. - `refreshView` returns a promise then gets all the todos from the local database and passes them to `renderAllTodos`, which converts the todos to HTML and injects that HTML into the web page. - We have also enhanced another method, the `onSubmit` event handler, so that the todos re-render as new todos are added. --- [← Back to *getting data*](../07-getting-data) | [Continue to *deleting data* →](../09-deleting-data) ================================================ FILE: 03-offline-todo/08-rendering-todos/application.js ================================================ (function() { var db, input, ul; databaseOpen() .then(function() { input = document.querySelector('input'); ul = document.querySelector('ul'); document.body.addEventListener('submit', onSubmit); }) .then(refreshView); function onSubmit(e) { e.preventDefault(); var todo = { text: input.value, _id: String(Date.now()) }; databaseTodosPut(todo) .then(function() { input.value = ''; }) .then(refreshView); } function refreshView() { return databaseTodosGet().then(renderAllTodos); } function renderAllTodos(todos) { var html = ''; todos.forEach(function(todo) { html += todoToHtml(todo); }); ul.innerHTML = html; } function todoToHtml(todo) { return '
  • '+todo.text+'
    • '; } function databaseOpen() { return new Promise(function(resolve, reject) { var version = 1; var request = indexedDB.open('todos', version); request.onupgradeneeded = function(e) { db = e.target.result; e.target.transaction.onerror = reject; db.createObjectStore('todo', { keyPath: '_id' }); }; request.onsuccess = function(e) { db = e.target.result; resolve(); }; request.onerror = reject; }); } function databaseTodosPut(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.put(todo); transaction.oncomplete = resolve; request.onerror = reject; }); } function databaseTodosGet() { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readonly'); var store = transaction.objectStore('todo'); // Get everything in the store var keyRange = IDBKeyRange.lowerBound(0); var cursorRequest = store.openCursor(keyRange); // This fires once per row in the store, so for simplicity collect the data // in an array (data) and send it pass it in the resolve call in one go var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; // If there's data, add it to array if (result) { data.push(result.value); result.continue(); // Reach the end of the data } else { resolve(data); } }; }); } }()); ================================================ FILE: 03-offline-todo/08-rendering-todos/index.html ================================================

    Example: Todo

    ================================================ FILE: 03-offline-todo/08-rendering-todos/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j'+todo.text+''; } […] function databaseTodosGetById(id) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.get(id); request.onsuccess = function(e) { var result = e.target.result; resolve(result); }; request.onerror = reject; }); } function databaseTodosDelete(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.delete(todo._id); transaction.oncomplete = resolve; request.onerror = reject; }); } }()); ``` We’ve made the following enhancements: - We’ve added a new event handler (`onClick`) that listens to click events and checks whether the target element has an ID attribute. If it has one it calls databaseTodosDelete with that value and, if the item is successfully deleted, re-renders the to-do list following the same approach that we took in step 6. - We’ve enhanced the `todoToHtml` function so that every to-do item is outputted with a delete button with an ID attribute set to its `_id`. - We’ve added two new database functions: - `databaseTodosDelete`, which takes a `todo` (well, a javascript object with an `_id` property) and returns a promise, deletes the item and then resolves the promise, and - `databaseTodosGetById`, which takes an `_id` and returns a promise, which resolves with the todo keyed by the given `_id`. Our to-do app is basically feature-complete. We can add and delete items, and it works in any browser that supports WebSQL or IndexedDB (although it could be a lot more efficient). --- [← Back to *rendering todos*](../08-rendering-todos) | [Continue to *Review: `IDBRequest` and `IDBTransaction`* →](../10-review-requests-transactions) ================================================ FILE: 03-offline-todo/09-deleting-data/application.js ================================================ (function() { var db, input, ul; databaseOpen() .then(function() { input = document.querySelector('input'); ul = document.querySelector('ul'); document.body.addEventListener('submit', onSubmit); document.body.addEventListener('click', onClick); }) .then(refreshView); function onClick(e) { e.preventDefault(); if (e.target.hasAttribute('id')) { databaseTodosGetById(e.target.getAttribute('id')) .then(function(todo) { return databaseTodosDelete(todo); }) .then(refreshView); } } function onSubmit(e) { e.preventDefault(); var todo = { text: input.value, _id: String(Date.now()) }; databaseTodosPut(todo) .then(function() { input.value = ''; }) .then(refreshView); } function refreshView() { return databaseTodosGet().then(renderAllTodos); } function renderAllTodos(todos) { var html = ''; todos.forEach(function(todo) { html += todoToHtml(todo); }); ul.innerHTML = html; } function todoToHtml(todo) { return '
  • '+todo.text+'
    • '; } function databaseOpen() { return new Promise(function(resolve, reject) { var version = 1; var request = indexedDB.open('todos', version); request.onupgradeneeded = function(e) { db = e.target.result; e.target.transaction.onerror = reject; db.createObjectStore('todo', { keyPath: '_id' }); }; request.onsuccess = function(e) { db = e.target.result; resolve(); }; request.onerror = reject; }); } function databaseTodosPut(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.put(todo); transaction.oncomplete = resolve; request.onerror = reject; }); } function databaseTodosGet() { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readonly'); var store = transaction.objectStore('todo'); // Get everything in the store var keyRange = IDBKeyRange.lowerBound(0); var cursorRequest = store.openCursor(keyRange); // This fires once per row in the store, so for simplicity collect the data // in an array (data) and send it pass it in the resolve call in one go var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; // If there's data, add it to array if (result) { data.push(result.value); result.continue(); // Reach the end of the data } else { resolve(data); } }; }); } function databaseTodosGetById(id) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.get(id); request.onsuccess = function(e) { var result = e.target.result; resolve(result); }; request.onerror = reject; }); } function databaseTodosDelete(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.delete(todo._id); transaction.oncomplete = resolve; request.onerror = reject; }); } }()); ================================================ FILE: 03-offline-todo/09-deleting-data/index.html ================================================

    Example: Todo

    ================================================ FILE: 03-offline-todo/09-deleting-data/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j) storeNames, optional IDBTransactionMode mode = "readonly"); ``` - `storeNames` - an array of object stores that your transaction needs to access. - `mode` - there are two modes for transactions: `readwrite` or `readonly`. #### Warning Use `transaction.oncomplete` and not `request.onsuccess` as the point where you know the data has been stored. ### Object Stores You can think of Object Stores as similar to tables in a SQL database. You can have many object stores per database, and they can contain many objects - indexed by at least one index. Object Stores are accessed via transaction objects. For example:- ```js var transaction = db.transaction['todo'], 'readonly'); var store = transaction.objectStore('todo'); ``` ### Requests Requests are the only objects that you can use to _read_ or _write_ data with IndexedDB. You can get to the results of each request using event handlers on the request objects. Results are created by the methods on object stores such as `IDBObjectStore#add`, `IDBObjectStore#delete` , `IDBObjectStore#createIndex`, `IDBObjectStore#openCursor`, etc. A special type of `IDBRequest` called `IDBOpenDBRequest` that we covered in the previous review is created by `IDBFactory#open` and `IDBFactory#deleteDatabase`. ## Deleting data ```js var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.delete(todo._id); transaction.oncomplete = resolve; ``` We can now talk through each line and explain what it does:- - Create an `IDBTransaction` object with `readwrite` access to the `todo` object store, assign it to `transaction` - Get an `IDBStore` object that represents the object store `todo` and assign it to `store` - Create a request to `delete` object with ID `todo._id` from `store` - When complete, `resolve` the promise ## Getting data ```js var transaction = db.transaction(['todo'], 'readonly'); var store = transaction.objectStore('todo'); var keyRange = IDBKeyRange.lowerBound(0); var cursorRequest = store.openCursor(keyRange); var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; if (result) { data.push(result.value); result.continue(); } else { resolve(data); } }; ``` - As we're only getting data we only need to create a `readonly` transaction. - Then we get an `IDBStore` object that represents the object store `todo` and assigns it to `store` - in the same way deleting works above. - The next two lines are concerned with creating a cursor (an object that allows us to walk through the database). The line `IDBKeyRange.lowerBound(0)` simply says start from the beginning of the table. - As data is pulled out of the table `success` events are fired on the cursor. In each callback we must call `continue` to proceed to the next `result`, and if that `result` is empty we know we've reached the end of the dataset. #### Cursors [`IDBCursors`](https://developer.mozilla.org/en-US/docs/Web/API/IDBCursor) can be used to look up data by their index as well as sort data within an object store. Usage of them is normally out of scope for this course - but here are a few links that introduce the key ideas: - [`IDBCursor` documentation on MDN](https://developer.mozilla.org/en-US/docs/Web/API/IDBCursor) - [How to do some magic with indexedDB](http://www.codeproject.com/Articles/744986/How-to-do-some-magic-with-indexedDB) ### Exercises - Try changing `transaction.oncomplete` to `request.onsuccess` inside `databaseTodosPut` in your todo app. What happens when you create todos? Why? - Reverse the order of the todos so that they added to the top, not the bottom, in our demo. - What would be different if we'd have used `add` instead of `put`? What are the benefits of one over the other? - Aside from `cursorRequest.lowerBound` what other approaches could you take to retrieving all the data from a table? - _Bonus:_ Write code that selects items from an object store that instead of waiting for all of them to come back before rendering those items onto the page, immediately render each one. What approaches or browser APIs instead of Promises could be more appropriate to implement this? - _Bonus:_ Support the ability to update existing items on our todo list. --- [← Back to *deleting todos*](../09-deleting-todos) | [Continue to *truly offline* →](../11-appcache) ================================================ FILE: 03-offline-todo/11-appcache/README.md ================================================ # Truly offline Have we actually built an offline-first to-do app? Almost, but not quite. While we can now store all data offline, if you switch off your device’s Internet connection and try loading the application, it won’t open. To fix this, we need to use the **HTML5 Application Cache**. ## Warning While HTML5 Application Cache works reasonably well for a simple single-page application like this, it doesn’t always. Thoroughly research how it works before considering whether to apply it to your website. - Service Worker might soon replace HTML5 Application Cache, although it is not currently usable in any browser, and neither Apple nor Microsoft have publicly committed to supporting it. - To enable the application cache, we’ll add a `manifest` attribute to the `html` element of the web page. We will cover the essential hacks needed to get AppCache to behave itself later in the course. ##### `/index.html` ```html […] ``` Then, we’ll create a manifest file, which is a simple text file in which we crudely specify the files to make available offline and how we want the cache to behave. ##### `/offline.appcache` ``` CACHE MANIFEST ./styles.css ./indexeddb.shim.min.js ./promise.js ./application.js NETWORK: * ``` The section that begins `CACHE MANIFEST` tells the browser the following: - When the application is first accessed, download each of those files and store them in the application cache. - Any time any of those files are needed from then on, load the cached versions of the files, rather than redownload them from the Internet. The section that begins `NETWORK` tells the browser that all other files must be downloaded fresh from the Internet every time they are needed. (Yes, the default behaviour is to say any file not listed in the AppCache should never be downloaded - even if the device is online!?) ## Check Verify the application works offline by loading it up and then switching off the static web server (or if you're using a webserver disconnect your device from the internet) and press refresh. --- [← Back to *review: `window.indexedDB`*](../11-review-requests-transactions) | [Continue to *first appcache gotcha* →](../12-appcache-gotcha-1) ================================================ FILE: 03-offline-todo/11-appcache/application.js ================================================ (function() { var db, input, ul; databaseOpen() .then(function() { input = document.querySelector('input'); ul = document.querySelector('ul'); document.body.addEventListener('submit', onSubmit); document.body.addEventListener('click', onClick); }) .then(refreshView); function onClick(e) { e.preventDefault(); if (e.target.hasAttribute('id')) { databaseTodosGetById(e.target.getAttribute('id')) .then(function(todo) { return databaseTodosDelete(todo); }) .then(refreshView); } } function onSubmit(e) { e.preventDefault(); var todo = { text: input.value, _id: String(Date.now()) }; databaseTodosPut(todo) .then(function() { input.value = ''; }) .then(refreshView); } function refreshView() { return databaseTodosGet().then(renderAllTodos); } function renderAllTodos(todos) { var html = ''; todos.forEach(function(todo) { html += todoToHtml(todo); }); ul.innerHTML = html; } function todoToHtml(todo) { return '
  • '+todo.text+'
    • '; } function databaseOpen() { return new Promise(function(resolve, reject) { var version = 1; var request = indexedDB.open('todos', version); request.onupgradeneeded = function(e) { db = e.target.result; e.target.transaction.onerror = reject; db.createObjectStore('todo', { keyPath: '_id' }); }; request.onsuccess = function(e) { db = e.target.result; resolve(); }; request.onerror = reject; }); } function databaseTodosPut(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.put(todo); transaction.oncomplete = resolve; request.onerror = reject; }); } function databaseTodosGet() { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readonly'); var store = transaction.objectStore('todo'); // Get everything in the store var keyRange = IDBKeyRange.lowerBound(0); var cursorRequest = store.openCursor(keyRange); // This fires once per row in the store, so for simplicity collect the data // in an array (data) and send it pass it in the resolve call in one go var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; // If there's data, add it to array if (result) { data.push(result.value); result.continue(); // Reach the end of the data } else { resolve(data); } }; }); } function databaseTodosGetById(id) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.get(id); request.onsuccess = function(e) { var result = e.target.result; resolve(result); }; request.onerror = reject; }); } function databaseTodosDelete(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.delete(todo._id); transaction.oncomplete = resolve; request.onerror = reject; }); } }()); ================================================ FILE: 03-offline-todo/11-appcache/index.html ================================================

    Example: Todo

    ================================================ FILE: 03-offline-todo/11-appcache/offline.appcache ================================================ CACHE MANIFEST ./styles.css ./indexeddb.shim.min.js ./promise.js ./application.js NETWORK: * ================================================ FILE: 03-offline-todo/11-appcache/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j - run on the first- and second-most recent versions of all major desktop and mobile browsers. ## Which technologies to use In an ideal world, we’d use just one client database technology. Unfortunately, we’ll have to use two: - **IndexedDB** This is the standard for client-side storage and the [only option available on Firefox and Internet Explorer](http://caniuse.com/indexeddb). - **WebSQL** This is the deprecated predecessor to WebSQL and the [only option available on current versions of iOS](http://caniuse.com/sql-storage) (although iOS 8 will finally give us IndexedDB). Veterans of the offline-first world might now be thinking, “But we could just use [localStorage](http://caniuse.com/namevalue-storage), which has the benefits of a much simpler API, and we wouldn’t need to worry about the complexity of using both IndexedDB and WebSQL.” While that is technically true, [localStorage has number of problems](https://hacks.mozilla.org/2012/03/there-is-no-simple-solution-for-local-storage/), the most important of which is that the amount of storage space available with it is significantly less than IndexedDB and WebSQL. Luckily, while we’ll need to use both, we’ll only need to think about IndexedDB. To support WebSQL, we’ll use an [IndexedDB polyfill](https://github.com/axemclion/IndexedDBShim). This will keep our code clean and easy to maintain, and once all browsers that we care about support IndexedDB natively, we can simply delete the polyfill. **Note:** If you’re starting a new project and are deciding whether to use IndexedDB or WebSQL, I strongly advocate using IndexedDB and the polyfill. In my opinion, there is no reason to write any new code that integrates with WebSQL directly. I’ll go through all of the steps using Google Chrome (and its developer tools), but there’s no reason why you couldn’t develop this application using any other modern browser. --- [← Back to *Fetch*](../02-new-apis/fetch.md) | [Start by *scaffolding the application* →](./01-scaffolding) ================================================ FILE: 04-offline-todo-with-sync/01-architecture/README.md ================================================ # Architecture As with the simple offline todo app, we're going to take lots of shortcuts so that we can focus on covering the key ideas. ## Synchronisation There are many ways to store data so that it can be synchronized and merged in with other changes that have happened since the last synchronisation. For example, **git** stores changes made 'offline' as sequences of **diffs**. For our todo application we are going to take the simplest possible route use the following algorithm for synchronising with the server: - download all todos from the server, load all todos from the local database - loop through all local todos - if a todo has been deleted locally, delete it from the server - if a todo isn't in the array of todos returned from the server, assume it's new and create it on the server - loop through all the remote todos - if a todo isn't in the array of todos loaded from the local database, assume it's been created by another client since the last synchronisation and create it locally. This algorithm is clearly extremely inefficient as it requires downloading all the todos in one go, it requires holding all the todos that exist in memory at once, and it requires walking through all the data to figure out what needs updating. We get away with this because the size of the data we expect for our todos is quite small. ### Delete Corollary 1: we can't just delete todos anymore By choosing this approach to synchronisation if we continue to directly delete todos from the local database as we are at the moment, the synchronisation algorithm isn't going to be able to distinguish todos that have been previously synced with the server but deleted locallly from todos that have been added by other clients but haven't yet been downloaded. To work around this instead of deleting todos we will *mark todos for deletion* and only delete them once we are sure that the server has successfully deleted them. ### Delete Corollary 2: the server can never delete todos Again because of the choice to use this approach to synchronisation in order for clients to be able to distinguish between todos that have been just created locally and not synced (and so exist locally but not on the server) and those that have been deleted by other clients (and so exist locally but not on the server) the server can never *actually* delete todos - only mark them for deletion. Luckily the API we've chosen to use already [does this already](https://github.com/matthew-andrews/offline-todo-api#delete-todosid---delete-a-todo) and returns a `410 Gone` response for all `POST` and `GET` requests keyed for todos with `_id`s that have already been deleted. ## Appropriate `_id`s With distributed applications where it is possible to create records on clients independently from the server choosing a sensible way to uniquely identify records turns out to be challenging. ### Why you can't just use auto-increment to create a local ID and let the server fill in a remote ID later: - Say you create a new todo and it was allocated an ID of 2 whilst your browser is offline. - You have the application open in two tabs when the browser connects to the internet and both begin syncing at the same time. - Both simulataneously `POST` the new todo to the server. - Because the server hasn't yet allocated the todo its permanent unique identifier, it cannot distinguish the two todos from each other and so creates it twice. You could work around this by requiring all todo text to be unique but that would probably be undesireable for a todo app. #### Other alternatives - **Hash the todo and use it as an ID.** Also requires todo text to be unique with the additional problem that once you've created and deleted a todo item you can never re-add a todo item with the same text again. - **Generate a UUID.** The FT solved a similar problem to solve in their article authoring tool by generating a 36 character unique identifier for all articles with a one-in-a-million-like chance of colliding with another. This would work well but adds complexity to the application. - **Only support the creation of todos online.** Worthy of a mention and useful for existing systems. Afterall, *some* offline behaviour is a huge improvement on *none*. - Use the **timestamp** at the point the todo was created. Although there are likely to be bugs collisions if two clients create a todo at precisely the same moment. Also relies on the device's clock being set to some extent. ### Conclusion There is no simple solution that works in all cases. For real world applications (that use this synchronisation algorithm) I would either choose to use some sort of uuid or to try to find a way to make local ids with remote ids lazily filled tab-safe. For the purpose of this demonstration prototype we will we use the **timestamp** option because it's by far the simplest. --- [← Back to *offline todo with IndexedDB and sync*](../) | [Continue to *mark for deletion* →](../02-mark-for-deletion) ================================================ FILE: 04-offline-todo-with-sync/02-mark-for-deletion/README.md ================================================ # Marking items for deletion Rather than directly deleting todo items we need to change our client side code to *mark todos for deletion*. To achieve this we are going to need to make the following changes to `application.js`: - change the `onClick` handler, which deletes individual todos, from directly deleting the todo to setting a new `deleted` property to `true` - enhance the `databaseTodosGet` method so that todos can be filtered by their deleted status. - update `refreshView`'s use of `databaseTodosGet` so that it only renders undeleted todos. - we won't delete the `databaseTodosDelete` method, even though it is now unused as the synchronisation logic that we will implement later will make use of it. ##### `application.js` ```js […] function onClick(e) { e.preventDefault(); if (e.target.hasAttribute('id')) { databaseTodosGetById(e.target.getAttribute('id')) .then(function(todo) { todo.deleted = true; return databaseTodosPut(todo); }) .then(refreshView); } } […] function refreshView() { return databaseTodosGet({ deleted: false }).then(renderAllTodos); } […] function databaseTodosGet(query) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readonly'); var store = transaction.objectStore('todo'); var keyRange = IDBKeyRange.lowerBound(0); var cursorRequest = store.openCursor(keyRange); var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; if (result) { if (!query || (query.deleted === true && result.value.deleted) || (query.deleted === false && !result.value.deleted)) { data.push(result.value); } result.continue(); } else { resolve(data); } }; }); } […] ``` Assuming nothing has broken the application should continue working in exactly the same way it did before - you should be able to create and delete todos. The difference is an implementation detail that can be checked by opening up dev tools - where you will see that todos don't actually get deleted any more - they only have a `deleted` flag set to `true` against them. See below: !['Hallo Welt' has been flagged for deletion](./screenshot.png) --- [← Back to *architecture*](../01-architecture) | [Continue to *adding ajax* →](../03-adding-ajax) ================================================ FILE: 04-offline-todo-with-sync/02-mark-for-deletion/application.js ================================================ (function() { var db, input, ul; databaseOpen() .then(function() { input = document.getElementsByTagName('input')[0]; ul = document.getElementsByTagName('ul')[0]; document.body.addEventListener('submit', onSubmit); document.body.addEventListener('click', onClick); }) .then(refreshView); function onClick(e) { e.preventDefault(); if (e.target.hasAttribute('id')) { databaseTodosGetById(e.target.getAttribute('id')) .then(function(todo) { todo.deleted = true; return databaseTodosPut(todo); }) .then(refreshView); } } function onSubmit(e) { e.preventDefault(); var todo = { text: input.value, _id: String(Date.now()) }; databaseTodosPut(todo) .then(function() { input.value = ''; }) .then(refreshView); } function refreshView() { return databaseTodosGet({ deleted: false }).then(renderAllTodos); } function renderAllTodos(todos) { var html = ''; todos.forEach(function(todo) { html += todoToHtml(todo); }); ul.innerHTML = html; } function todoToHtml(todo) { return '
  • '+todo.text+'
    • '; } function databaseOpen() { return new Promise(function(resolve, reject) { var version = 1; var request = indexedDB.open('todos', version); request.onupgradeneeded = function(e) { db = e.target.result; e.target.transaction.onerror = reject; db.createObjectStore('todo', { keyPath: '_id' }); }; request.onsuccess = function(e) { db = e.target.result; resolve(); }; request.onerror = reject; }); } function databaseTodosPut(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.put(todo); transaction.oncomplete = resolve; request.onerror = reject; }); } function databaseTodosGet(query) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readonly'); var store = transaction.objectStore('todo'); // Get everything in the store var keyRange = IDBKeyRange.lowerBound(0); var cursorRequest = store.openCursor(keyRange); // This fires once per row in the store, so for simplicity collect the data // in an array (data) and send it pass it in the resolve call in one go var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; // If there's data, add it to array if (result) { if (!query || (query.deleted === true && result.value.deleted) || (query.deleted === false && !result.value.deleted)) { data.push(result.value); } result.continue(); // Reach the end of the data } else { resolve(data); } }; }); } function databaseTodosGetById(id) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readonly'); var store = transaction.objectStore('todo'); var request = store.get(id); request.onsuccess = function(e) { var result = e.target.result; resolve(result); }; request.onerror = reject; }); } function databaseTodosDelete(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.delete(todo._id); transaction.oncomplete = resolve; request.onerror = reject; }); } }()); ================================================ FILE: 04-offline-todo-with-sync/02-mark-for-deletion/index.html ================================================

    Example: Todo

    ================================================ FILE: 04-offline-todo-with-sync/02-mark-for-deletion/offline.appcache ================================================ CACHE MANIFEST ./styles.css ./indexeddb.shim.min.js ./promise.js ./application.js NETWORK: * ================================================ FILE: 04-offline-todo-with-sync/02-mark-for-deletion/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j

    Example: Todo

    ``` ##### `offline.appcache` ``` CACHE MANIFEST ./styles.css ./indexeddb.shim.min.js ./promise.js ./fetch.js ./application.js NETWORK: * ``` ##### `fetch.js` Download the contents of the [Fetch API polyfill](https://raw.githubusercontent.com/github/fetch/master/fetch.js), and put it in this file. ## Api wrapper methods Now we have an ajax library installed and available to us in our application we are next going to implement a few helper methods that mirror the API of the database methods: ##### `application.js` ```js (function() { var api = 'https://offline-todo-api.herokuapp.com/todos'; […] function serverTodosGet(_id) { return fetch(api + '/' + (_id ? _id : '')) .then(function(response) { return response.json(); }); } function serverTodosPost(todo) { return fetch(api, { method: 'post', headers: { 'Accept': 'application/json', 'Content-Type': 'application/json' }, body: JSON.stringify(todo) }) .then(function(response) { if (response.status === 410) throw new Error(response.statusText); return response; }); } function serverTodosDelete(todo) { return fetch(api + '/' + todo._id, { method: 'delete' }) } }()); ``` --- [← Back to *mark for deletion*](../02-mark-for-deletion) | [Continue to *synchronize* →](../04-synchronize) ================================================ FILE: 04-offline-todo-with-sync/03-adding-ajax/application.js ================================================ (function() { var api = 'https://offline-todo-api.herokuapp.com/todos'; var db, input, ul; databaseOpen() .then(function() { input = document.getElementsByTagName('input')[0]; ul = document.getElementsByTagName('ul')[0]; document.body.addEventListener('submit', onSubmit); document.body.addEventListener('click', onClick); }) .then(refreshView); function onClick(e) { e.preventDefault(); if (e.target.hasAttribute('id')) { databaseTodosGetById(e.target.getAttribute('id')) .then(function(todo) { todo.deleted = true; return databaseTodosPut(todo); }) .then(refreshView); } } function onSubmit(e) { e.preventDefault(); var todo = { text: input.value, _id: String(Date.now()) }; databaseTodosPut(todo) .then(function() { input.value = ''; }) .then(refreshView); } function refreshView() { return databaseTodosGet({ deleted: false }).then(renderAllTodos); } function renderAllTodos(todos) { var html = ''; todos.forEach(function(todo) { html += todoToHtml(todo); }); ul.innerHTML = html; } function todoToHtml(todo) { return '
  • '+todo.text+'
    • '; } function databaseOpen() { return new Promise(function(resolve, reject) { var version = 1; var request = indexedDB.open('todos', version); request.onupgradeneeded = function(e) { db = e.target.result; e.target.transaction.onerror = reject; db.createObjectStore('todo', { keyPath: '_id' }); }; request.onsuccess = function(e) { db = e.target.result; resolve(); }; request.onerror = reject; }); } function databaseTodosPut(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.put(todo); transaction.oncomplete = resolve; request.onerror = reject; }); } function databaseTodosGet(query) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readonly'); var store = transaction.objectStore('todo'); // Get everything in the store var keyRange = IDBKeyRange.lowerBound(0); var cursorRequest = store.openCursor(keyRange); // This fires once per row in the store, so for simplicity collect the data // in an array (data) and send it pass it in the resolve call in one go var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; // If there's data, add it to array if (result) { if (!query || (query.deleted === true && result.value.deleted) || (query.deleted === false && !result.value.deleted)) { data.push(result.value); } result.continue(); // Reach the end of the data } else { resolve(data); } }; }); } function databaseTodosGetById(id) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readonly'); var store = transaction.objectStore('todo'); var request = store.get(id); request.onsuccess = function(e) { var result = e.target.result; resolve(result); }; request.onerror = reject; }); } function databaseTodosDelete(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.delete(todo._id); transaction.oncomplete = resolve; request.onerror = reject; }); } function serverTodosGet(_id) { return fetch(api + '/' + (_id ? _id : '')) .then(function(response) { return response.json(); }); } function serverTodosPost(todo) { return fetch(api, { method: 'post', headers: { 'Accept': 'application/json', 'Content-Type': 'application/json' }, body: JSON.stringify(todo) }) .then(function(response) { if (response.status === 410) throw new Error(response.statusText); return response; }); } function serverTodosDelete(todo) { return fetch(api + '/' + todo._id, { method: 'delete' }); } }()); ================================================ FILE: 04-offline-todo-with-sync/03-adding-ajax/fetch.js ================================================ (function() { 'use strict'; if (window.fetch) { return } function Headers(headers) { this.map = {} var self = this if (headers instanceof Headers) { headers.forEach(function(name, values) { values.forEach(function(value) { self.append(name, value) }) }) } else if (headers) { Object.getOwnPropertyNames(headers).forEach(function(name) { self.append(name, headers[name]) }) } } Headers.prototype.append = function(name, value) { var list = this.map[name] if (!list) { list = [] this.map[name] = list } list.push(value) } Headers.prototype['delete'] = function(name) { delete this.map[name] } Headers.prototype.get = function(name) { var values = this.map[name] return values ? values[0] : null } Headers.prototype.getAll = function(name) { return this.map[name] || [] } Headers.prototype.has = function(name) { return this.map.hasOwnProperty(name) } Headers.prototype.set = function(name, value) { this.map[name] = [value] } // Instead of iterable for now. Headers.prototype.forEach = function(callback) { var self = this Object.getOwnPropertyNames(this.map).forEach(function(name) { callback(name, self.map[name]) }) } function consumed(body) { if (body.bodyUsed) { return Promise.reject(new TypeError('Body already consumed')) } body.bodyUsed = true } function Body() { this.body = null this.bodyUsed = false this.arrayBuffer = function() { throw new Error('Not implemented yet') } this.blob = function() { var rejected = consumed(this) return rejected ? rejected : Promise.resolve(new Blob([this.body])) } this.formData = function() { return Promise.resolve(decode(this.body)) } this.json = function() { var rejected = consumed(this) if (rejected) { return rejected } var body = this.body return new Promise(function(resolve, reject) { try { resolve(JSON.parse(body)) } catch (ex) { reject(ex) } }) } this.text = function() { var rejected = consumed(this) return rejected ? rejected : Promise.resolve(this.body) } return this } function Request(url, options) { options = options || {} this.url = url this.body = options.body this.credentials = options.credentials || null this.headers = new Headers(options.headers) this.method = options.method || 'GET' this.mode = options.mode || null this.referrer = null } function encode(params) { return Object.getOwnPropertyNames(params).filter(function(name) { return params[name] !== undefined }).map(function(name) { var value = (params[name] === null) ? '' : params[name] return encodeURIComponent(name) + '=' + encodeURIComponent(value) }).join('&').replace(/%20/g, '+') } function decode(body) { var form = new FormData() body.trim().split('&').forEach(function(bytes) { if (bytes) { var split = bytes.split('=') var name = split.shift().replace(/\+/g, ' ') var value = split.join('=').replace(/\+/g, ' ') form.append(decodeURIComponent(name), decodeURIComponent(value)) } }) return form } function isObject(value) { try { return Object.getPrototypeOf(value) === Object.prototype } catch (ex) { // Probably a string literal. return false } } function headers(xhr) { var head = new Headers() var pairs = xhr.getAllResponseHeaders().trim().split('\n') pairs.forEach(function(header) { var split = header.trim().split(':') var key = split.shift().trim() var value = split.join(':').trim() head.append(key, value) }) return head } Request.prototype.fetch = function() { var self = this return new Promise(function(resolve, reject) { var xhr = new XMLHttpRequest() xhr.onload = function() { var options = { status: xhr.status, statusText: xhr.statusText, headers: headers(xhr) } resolve(new Response(xhr.responseText, options)) } xhr.onerror = function() { reject() } xhr.open(self.method, self.url) self.headers.forEach(function(name, values) { values.forEach(function(value) { xhr.setRequestHeader(name, value) }) }) var body = self.body if (isObject(self.body)) { xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded; charset=UTF-8') body = encode(self.body) } xhr.send(body) }) } Body.call(Request.prototype) function Response(body, options) { this.body = body this.type = 'default' this.url = null this.status = options.status this.statusText = options.statusText this.headers = options.headers } Body.call(Response.prototype) window.fetch = function (url, options) { return new Request(url, options).fetch() } })(); ================================================ FILE: 04-offline-todo-with-sync/03-adding-ajax/index.html ================================================

    Example: Todo

    ================================================ FILE: 04-offline-todo-with-sync/03-adding-ajax/offline.appcache ================================================ CACHE MANIFEST ./styles.css ./indexeddb.shim.min.js ./promise.js ./fetch.js ./application.js NETWORK: * ================================================ FILE: 04-offline-todo-with-sync/03-adding-ajax/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j'+todo.text+''; } function databaseOpen() { return new Promise(function(resolve, reject) { var version = 1; var request = indexedDB.open('todos', version); request.onupgradeneeded = function(e) { db = e.target.result; e.target.transaction.onerror = reject; db.createObjectStore('todo', { keyPath: '_id' }); }; request.onsuccess = function(e) { db = e.target.result; resolve(); }; request.onerror = reject; }); } function databaseTodosPut(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.put(todo); transaction.oncomplete = resolve; request.onerror = reject; }); } function databaseTodosGet(query) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readonly'); var store = transaction.objectStore('todo'); // Get everything in the store var keyRange = IDBKeyRange.lowerBound(0); var cursorRequest = store.openCursor(keyRange); // This fires once per row in the store, so for simplicity collect the data // in an array (data) and send it pass it in the resolve call in one go var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; // If there's data, add it to array if (result) { if (!query || (query.deleted === true && result.value.deleted) || (query.deleted === false && !result.value.deleted)) { data.push(result.value); } result.continue(); // Reach the end of the data } else { resolve(data); } }; }); } function databaseTodosGetById(id) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readonly'); var store = transaction.objectStore('todo'); var request = store.get(id); request.onsuccess = function(e) { var result = e.target.result; resolve(result); }; request.onerror = reject; }); } function databaseTodosDelete(todo) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['todo'], 'readwrite'); var store = transaction.objectStore('todo'); var request = store.delete(todo._id); transaction.oncomplete = resolve; request.onerror = reject; }); } function serverTodosGet(_id) { return fetch(api + '/' + (_id ? _id : '')) .then(function(response) { return response.json(); }); } function serverTodosPost(todo) { return fetch(api, { method: 'post', headers: { 'Accept': 'application/json', 'Content-Type': 'application/json' }, body: JSON.stringify(todo) }) .then(function(response) { if (response.status === 410) throw new Error(response.statusText); return response; }); } function serverTodosDelete(todo) { return fetch(api + '/' + todo._id, { method: 'delete' }); } }()); ================================================ FILE: 04-offline-todo-with-sync/04-synchronize/fetch.js ================================================ (function() { 'use strict'; if (window.fetch) { return } function Headers(headers) { this.map = {} var self = this if (headers instanceof Headers) { headers.forEach(function(name, values) { values.forEach(function(value) { self.append(name, value) }) }) } else if (headers) { Object.getOwnPropertyNames(headers).forEach(function(name) { self.append(name, headers[name]) }) } } Headers.prototype.append = function(name, value) { var list = this.map[name] if (!list) { list = [] this.map[name] = list } list.push(value) } Headers.prototype['delete'] = function(name) { delete this.map[name] } Headers.prototype.get = function(name) { var values = this.map[name] return values ? values[0] : null } Headers.prototype.getAll = function(name) { return this.map[name] || [] } Headers.prototype.has = function(name) { return this.map.hasOwnProperty(name) } Headers.prototype.set = function(name, value) { this.map[name] = [value] } // Instead of iterable for now. Headers.prototype.forEach = function(callback) { var self = this Object.getOwnPropertyNames(this.map).forEach(function(name) { callback(name, self.map[name]) }) } function consumed(body) { if (body.bodyUsed) { return Promise.reject(new TypeError('Body already consumed')) } body.bodyUsed = true } function Body() { this.body = null this.bodyUsed = false this.arrayBuffer = function() { throw new Error('Not implemented yet') } this.blob = function() { var rejected = consumed(this) return rejected ? rejected : Promise.resolve(new Blob([this.body])) } this.formData = function() { return Promise.resolve(decode(this.body)) } this.json = function() { var rejected = consumed(this) if (rejected) { return rejected } var body = this.body return new Promise(function(resolve, reject) { try { resolve(JSON.parse(body)) } catch (ex) { reject(ex) } }) } this.text = function() { var rejected = consumed(this) return rejected ? rejected : Promise.resolve(this.body) } return this } function Request(url, options) { options = options || {} this.url = url this.body = options.body this.credentials = options.credentials || null this.headers = new Headers(options.headers) this.method = options.method || 'GET' this.mode = options.mode || null this.referrer = null } function encode(params) { return Object.getOwnPropertyNames(params).filter(function(name) { return params[name] !== undefined }).map(function(name) { var value = (params[name] === null) ? '' : params[name] return encodeURIComponent(name) + '=' + encodeURIComponent(value) }).join('&').replace(/%20/g, '+') } function decode(body) { var form = new FormData() body.trim().split('&').forEach(function(bytes) { if (bytes) { var split = bytes.split('=') var name = split.shift().replace(/\+/g, ' ') var value = split.join('=').replace(/\+/g, ' ') form.append(decodeURIComponent(name), decodeURIComponent(value)) } }) return form } function isObject(value) { try { return Object.getPrototypeOf(value) === Object.prototype } catch (ex) { // Probably a string literal. return false } } function headers(xhr) { var head = new Headers() var pairs = xhr.getAllResponseHeaders().trim().split('\n') pairs.forEach(function(header) { var split = header.trim().split(':') var key = split.shift().trim() var value = split.join(':').trim() head.append(key, value) }) return head } Request.prototype.fetch = function() { var self = this return new Promise(function(resolve, reject) { var xhr = new XMLHttpRequest() xhr.onload = function() { var options = { status: xhr.status, statusText: xhr.statusText, headers: headers(xhr) } resolve(new Response(xhr.responseText, options)) } xhr.onerror = function() { reject() } xhr.open(self.method, self.url) self.headers.forEach(function(name, values) { values.forEach(function(value) { xhr.setRequestHeader(name, value) }) }) var body = self.body if (isObject(self.body)) { xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded; charset=UTF-8') body = encode(self.body) } xhr.send(body) }) } Body.call(Request.prototype) function Response(body, options) { this.body = body this.type = 'default' this.url = null this.status = options.status this.statusText = options.statusText this.headers = options.headers } Body.call(Response.prototype) window.fetch = function (url, options) { return new Request(url, options).fetch() } })(); ================================================ FILE: 04-offline-todo-with-sync/04-synchronize/index.html ================================================

    Example: Todo

    ================================================ FILE: 04-offline-todo-with-sync/04-synchronize/offline.appcache ================================================ CACHE MANIFEST ./styles.css ./indexeddb.shim.min.js ./promise.js ./fetch.js ./application.js NETWORK: * ================================================ FILE: 04-offline-todo-with-sync/04-synchronize/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j> test.json # Neat trick if you're using *nix npm install --save express cookie-parser es6-promise isomorphic-fetch ``` ## Online only To begin with we'll make the application work online only (ie. a normal website) and then discuss the changes that we will have to make to make it work offline. As building normal websites is an assumed prerequisite of this course heavy use of copy-paste is encouragedhere unless anything is unclear: ##### [`public/styles.css`](./public/styles.css) ```css body { margin: 0; padding: 0; font-family: helvetica, sans-serif; } * { box-sizing: border-box; } h1 { padding: 14px 0 14px 0; margin: 0; font-size: 44px; border-bottom: solid 1px #DDD; line-height: 1em; } nav { padding: 14px 0 14px 0; } main { padding: 0 14px; } ul { padding: 0; margin: 0; list-style: none; } li { padding: 20px 0 20px 0; border-bottom: solid 1px #DDD; } ``` Nothing too surprising should jump out here - it's just plain CSS. ##### [`public/templates.js`](./public/templates.js) ```js (function() { var exports = { list: list, article: article }; function list(data) { data = data || []; var ul = ''; data.forEach(function(story) { ul += '
  • '+story.title+'
    • '; }); return '

    FT Tech Blog

      '+ul+'
    '; } function article(data) { return '

    '+data.title+'

    '+data.body; } if (typeof module == 'object') { module.exports = exports; } else { window.templates = exports; } }()); ``` As we discussed above, these functions will eventually be used on the client side - which is why they need to go inside `public`. `list` and `article` are functions that take a JavaScript object containing data that represent a list of stories and a single story, respectively. The last few lines are potentially a little confusing:- ```js if (typeof module == 'object') { module.exports = exports; } else { window.templates = exports; } ``` `if (typeof module == 'object')` is just a way of saying "am I running on the server" - and if that is the case this module will expose its functions via `module.exports`, otherwise it will add them to the `window` object. ##### [`index.js`](./index.js) ```js require('es6-promise').polyfill(); require('isomorphic-fetch'); var port = Number(process.env.PORT || 8080); var api = 'https://offline-news-api.herokuapp.com/stories'; var cookieParser = require('cookie-parser'); var express = require('express'); var path = require('path'); var templates = require('./public/templates'); var app = express(); app.use(cookieParser()); app.use(express.static(path.join(__dirname, 'public'))); // Manifest returns a 400 unless the AppCache cookie is set app.get('/offline.appcache', function(req, res) { if (req.cookies.up) { res.set('Content-Type', 'text/cache-manifest'); res.send('CACHE MANIFEST' + '\n./appcache.js' + '\n./application.js' + '\n./iframe.js' + '\n./indexeddb.shim.min.js' + '\n./promise.js' + '\n./styles.css' + '\n./fetch.js' + '\n./templates.js' + '\n' + '\nFALLBACK:' + '\n/ /' + '\n' + '\nNETWORK:' + '\n*'); } else { res.status(400).end(); } }); // Add middleware to send this when the appcache update cookie is set app.get('/', offlineMiddleware); app.get('/article/:guid', offlineMiddleware); function offlineMiddleware(req, res, next) { if (req.cookies.up) res.send(layoutShell()); else next(); } app.get('/fallback.html', function(req, res) { res.send(layoutShell()); }); app.get('/article/:guid', function(req, res) { fetch(api+'/'+req.params.guid) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.article(data) })); }, function(err) { res.status(404); res.send(layoutShell({ main: templates.article({ title: 'Story cannot be found', body: '

    Please try another

    ' }) })); }); }); app.get('/', function(req, res) { fetch(api) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.list(data) })); }, function(err) { res.status(404).end(); }); }); function layoutShell(data) { data = { title: data && data.title || 'FT Tech News', main: data && data.main || '' }; return '' + '\n' + '\n ' + '\n '+data.title+'' + '\n ' + '\n ' + '\n ' + '\n
    mattandre.ws | @andrewsmatt
    ' + '\n
    '+data.main+'
    ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n'; } app.listen(port); console.log('listening on '+port); ``` Now run the application in your favourite web browser and check that both views work by running:- ```js node index.js ``` And opening `http://localhost:8080` with your favourite browser. --- [← Back to *building and offline news app, FT style*](../) | [Continue to *single/multi-page app* →](../02-single-multi-page) ================================================ FILE: 05-offline-news/01-scaffolding/index.js ================================================ require('es6-promise').polyfill(); require('isomorphic-fetch'); var api = 'https://offline-news-api.herokuapp.com/stories'; var port = 8080; var express = require('express'); var path = require('path'); var templates = require('./public/templates'); var app = express(); app.use(express.static(path.join(__dirname, 'public'))); app.get('/article/:guid', function(req, res) { fetch(api+'/'+req.params.guid) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.article(data) })); }, function(err) { res.status(404); res.send(layoutShell({ main: templates.article({ title: 'Story cannot be found', body: '

    Please try another

    ' }) })); }); }); app.get('/', function(req, res) { fetch(api) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.list(data) })); }, function(err) { res.status(404).end(); }); }); function layoutShell(data) { data = { title: data && data.title || 'FT Tech News', main: data && data.main || '' }; return '' + '\n' + '\n ' + '\n '+data.title+'' + '\n ' + '\n ' + '\n ' + '\n
    '+data.main+'
    ' + '\n ' + '\n'; } app.listen(port); console.log('listening on port', port); ================================================ FILE: 05-offline-news/01-scaffolding/package.json ================================================ { "dependencies": { "cookie-parser": "^1.3.3", "es6-promise": "^2.0.0", "express": "^4.8.8", "isomorphic-fetch": "^1.0.0" } } ================================================ FILE: 05-offline-news/01-scaffolding/public/styles.css ================================================ body { margin: 0; padding: 0; font-family: helvetica, sans-serif; } * { box-sizing: border-box; } h1 { padding: 14px 0 14px 0; margin: 0; font-size: 44px; border-bottom: solid 1px #DDD; line-height: 1em; } nav { padding: 14px 0 14px 0; } main { padding: 0 14px; } ul { padding: 0; margin: 0; list-style: none; } li { padding: 20px 0 20px 0; border-bottom: solid 1px #DDD; } ================================================ FILE: 05-offline-news/01-scaffolding/public/templates.js ================================================ (function() { var exports = { list: list, article: article }; function list(data) { data = data || []; var ul = ''; data.forEach(function(story) { ul += '
  • '+story.title+'
    • '; }); return '

    FT Tech Blog

      '+ul+'
    '; } function article(data) { return '

    '+data.title+'

    '+data.body; } if (typeof module == 'object') { module.exports = exports; } else { window.templates = exports; } }()); ================================================ FILE: 05-offline-news/02-single-multi-page/README.md ================================================ # Single/Multi page app Back in the introduction we laid down some basic ground rules about which offline technologies we were going to use to store the different kinds of data we need to. - We will use **Application Cache** to store application assets, such as CSS, JavaScript and a basic HTML shell. - We will use **IndexedDB** to store content - in this case, articles. Because content stored in IndexedDB is not accessible to the Application Cache, in order to display pages offline we the website will need to transform into a single page app - whilst still being a normal website (multi-page app?) for the initial load. In this step we will implement synchronisation and client-side and server-side rendering. ##### [`/index.js`](./index.js) On the server side we will need to add a few more JavaScript files to the `layoutShell` function. ```js […] + '\n ' + '\n ' + '\n
    '+data.main+'
    ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n'; […] ``` ##### Add libraries and polyfills Copy over the polyfills for [**IndexedDB**](./public/indexeddb.shim.min.js), [**Promises**](./public/promise.js) and the [**Fetch API polyfill**](./public/fetch.js) library from our previous prototypes and place the JavaScript files in `public`. ##### [`/application.js`](./public/application.js) 1. Opens a database 2. Synchronises with the `/stories`, adding stories that have been added and deleting stories that have been removed 3. Uses `window.templates.list()` and `window.templates.article()` to render those articles. 4. Uses `history.pushState` to update the URL when a user clicks on an article or when they click to view list of articles and refreshes the content on screen with that of the requested page. 5. Bonus: integrate this with Google analytics so that link click that is handled on the client side is turned into a `pageview` event and tracked. Try to do use the work we have done already in previous prototypes copying the solution. ```js (function() { var api = 'https://offline-news-api.herokuapp.com/stories'; var synchronizeInProgress; var db, main; databaseOpen() .then(function() { main = document.querySelector('main'); document.body.addEventListener('click', onClick); window.addEventListener('popstate', refreshView); // Only refresh the view if the view is empty if (main.innerHTML === '') return refreshView(); }) .then(synchronize); function onClick(e) { if (e.target.classList.contains('js-link')) { e.preventDefault(); history.pushState({}, '', e.target.getAttribute('href')); refreshView(); } } function refreshView() { var guidMatches = location.pathname.match(/^\/article\/([0-9]+)/); if (!guidMatches) { renderAllStories(); return databaseStoriesGet().then(renderAllStories); } renderOneStory(); return databaseStoriesGetById(guidMatches[1]).then(renderOneStory); } function renderAllStories(stories) { main.innerHTML = templates.list(stories); } function renderOneStory(story) { if (!story) story = { title: 'Story cannot be found', body: '

    Please try another

    ' }; main.innerHTML = templates.article(story); } function synchronize() { if (synchronizeInProgress) return synchronizeInProgress; synchronizeInProgress = Promise.all([serverStoriesGet(), databaseStoriesGet()]) .then(function(results) { var promises = []; var remoteStories = results[0]; var localStories = results[1]; // Add new stories downloaded from server to the database promises = promises.concat(remoteStories.map(function(story) { if (!arrayContainsStory(localStories, story)) { return databaseStoriesPut(story); } })); // Delete stories that are no longer on the server from the database promises = promises.concat(localStories.map(function(story) { if (!arrayContainsStory(remoteStories, story)) { return databaseStoriesDelete(story); } })); return promises; }) // Only refresh the view if it's listing page .then(function(results) { if (location.pathname === '/') { return refreshView(); } }) .then(function() { synchronizeInProgress = undefined; }); } function arrayContainsStory(array, story) { return array.some(function(arrayStory) { return arrayStory.guid === story.guid; }); } function databaseOpen() { return new Promise(function(resolve, reject) { var version = 1; var request = indexedDB.open('news-server-rendered', version); request.onupgradeneeded = function(e) { db = e.target.result; e.target.transaction.onerror = reject; db.createObjectStore('stories', { keyPath: 'guid' }); }; request.onsuccess = function(e) { db = e.target.result; resolve(); }; request.onerror = reject; }); } function databaseStoriesPut(story) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var request = store.put(story); request.onsuccess = resolve; request.onerror = reject; }); } function databaseStoriesGet() { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readonly'); var store = transaction.objectStore('stories'); var keyRange = IDBKeyRange.lowerBound(0); // Using reverse direction because the index being sorted on // ends with a numerical incrementing ID so to get newest news // first you need to sort by largest first. var cursorRequest = store.openCursor(keyRange, 'prev'); var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; if (result) { data.push(result.value); result.continue(); } else { resolve(data); } }; }); } function databaseStoriesGetById(guid) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readonly'); var store = transaction.objectStore('stories'); var request = store.get(guid); request.onsuccess = function(e) { var result = e.target.result; resolve(result); }; request.onerror = reject; }); } function databaseStoriesDelete(story) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var request = store.delete(story.guid); request.onsuccess = resolve; request.onerror = reject; }); } function serverStoriesGet(guid) { return fetch(api + '/' + (guid ? guid : '')) .then(function(response) { return response.json(); }); } })(); ``` --- [← Back to *scaffolding*](../01-scaffolding) | [Continue to *hacking appcache* →](../03-hacking-appcache) ================================================ FILE: 05-offline-news/02-single-multi-page/index.js ================================================ require('es6-promise').polyfill(); require('isomorphic-fetch'); var api = 'https://offline-news-api.herokuapp.com/stories'; var port = 8080; var express = require('express'); var path = require('path'); var templates = require('./public/templates'); var app = express(); app.use(express.static(path.join(__dirname, 'public'))); app.get('/article/:guid', function(req, res) { fetch(api+'/'+req.params.guid) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.article(data) })); }, function(err) { res.status(404); res.send(layoutShell({ main: templates.article({ title: 'Story cannot be found', body: '

    Please try another

    ' }) })); }); }); app.get('/', function(req, res) { fetch(api) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.list(data) })); }, function(err) { res.status(404).end(); }); }); function layoutShell(data) { data = { title: data && data.title || 'FT Tech News', main: data && data.main || '' }; return '' + '\n' + '\n ' + '\n '+data.title+'' + '\n ' + '\n ' + '\n ' + '\n
    '+data.main+'
    ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n'; } app.listen(port); console.log('listening on port', port); ================================================ FILE: 05-offline-news/02-single-multi-page/package.json ================================================ { "dependencies": { "cookie-parser": "^1.3.3", "es6-promise": "^2.0.0", "express": "^4.8.8", "isomorphic-fetch": "^1.0.0" } } ================================================ FILE: 05-offline-news/02-single-multi-page/public/application.js ================================================ (function() { var api = 'https://offline-news-api.herokuapp.com/stories'; var synchronizeInProgress; var db, main; databaseOpen() .then(function() { main = document.querySelector('main'); document.body.addEventListener('click', onClick); window.addEventListener('popstate', refreshView); // Only refresh the view if the view is empty if (main.innerHTML === '') return refreshView(); }) .then(synchronize); function onClick(e) { if (e.target.classList.contains('js-link')) { e.preventDefault(); history.pushState({}, '', e.target.getAttribute('href')); refreshView(); } } function refreshView() { var guidMatches = location.pathname.match(/^\/article\/([0-9]+)/); if (!guidMatches) { renderAllStories(); return databaseStoriesGet().then(renderAllStories); } renderOneStory(); return databaseStoriesGetById(guidMatches[1]).then(renderOneStory); } function renderAllStories(stories) { main.innerHTML = templates.list(stories); } function renderOneStory(story) { if (!story) story = { title: 'Story cannot be found', body: '

    Please try another

    ' }; main.innerHTML = templates.article(story); } function synchronize() { if (synchronizeInProgress) return synchronizeInProgress; synchronizeInProgress = Promise.all([serverStoriesGet(), databaseStoriesGet()]) .then(function(results) { var promises = []; var remoteStories = results[0]; var localStories = results[1]; // Add new stories downloaded from server to the database promises = promises.concat(remoteStories.map(function(story) { if (!arrayContainsStory(localStories, story)) { return databaseStoriesPut(story); } })); // Delete stories that are no longer on the server from the database promises = promises.concat(localStories.map(function(story) { if (!arrayContainsStory(remoteStories, story)) { return databaseStoriesDelete(story); } })); return promises; }) // Only refresh the view if it's listing page .then(function(results) { if (location.pathname === '/') { return refreshView(); } }) .then(function() { synchronizeInProgress = undefined; }); } function arrayContainsStory(array, story) { return array.some(function(arrayStory) { return arrayStory.guid === story.guid; }); } function databaseOpen() { return new Promise(function(resolve, reject) { var version = 1; var request = indexedDB.open('news-server-rendered', version); request.onupgradeneeded = function(e) { db = e.target.result; e.target.transaction.onerror = reject; db.createObjectStore('stories', { keyPath: 'guid' }); }; request.onsuccess = function(e) { db = e.target.result; resolve(); }; request.onerror = reject; }); } function databaseStoriesPut(story) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var request = store.put(story); request.onsuccess = resolve; request.onerror = reject; }); } function databaseStoriesGet() { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var keyRange = IDBKeyRange.lowerBound(0); // Using reverse direction because the index being sorted on // ends with a numerical incrementing ID so to get newest news // first you need to sort by largest first. var cursorRequest = store.openCursor(keyRange, 'prev'); var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; if (result) { data.push(result.value); result.continue(); } else { resolve(data); } }; }); } function databaseStoriesGetById(guid) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var request = store.get(guid); request.onsuccess = function(e) { var result = e.target.result; resolve(result); }; request.onerror = reject; }); } function databaseStoriesDelete(story) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var request = store.delete(story.guid); request.onsuccess = resolve; request.onerror = reject; }); } function serverStoriesGet(guid) { return fetch(api + '/' + (guid ? guid : '')) .then(function(response) { return response.json(); }); } })(); ================================================ FILE: 05-offline-news/02-single-multi-page/public/fetch.js ================================================ (function() { 'use strict'; if (window.fetch) { return } function Headers(headers) { this.map = {} var self = this if (headers instanceof Headers) { headers.forEach(function(name, values) { values.forEach(function(value) { self.append(name, value) }) }) } else if (headers) { Object.getOwnPropertyNames(headers).forEach(function(name) { self.append(name, headers[name]) }) } } Headers.prototype.append = function(name, value) { var list = this.map[name] if (!list) { list = [] this.map[name] = list } list.push(value) } Headers.prototype['delete'] = function(name) { delete this.map[name] } Headers.prototype.get = function(name) { var values = this.map[name] return values ? values[0] : null } Headers.prototype.getAll = function(name) { return this.map[name] || [] } Headers.prototype.has = function(name) { return this.map.hasOwnProperty(name) } Headers.prototype.set = function(name, value) { this.map[name] = [value] } // Instead of iterable for now. Headers.prototype.forEach = function(callback) { var self = this Object.getOwnPropertyNames(this.map).forEach(function(name) { callback(name, self.map[name]) }) } function consumed(body) { if (body.bodyUsed) { return Promise.reject(new TypeError('Body already consumed')) } body.bodyUsed = true } function Body() { this.body = null this.bodyUsed = false this.arrayBuffer = function() { throw new Error('Not implemented yet') } this.blob = function() { var rejected = consumed(this) return rejected ? rejected : Promise.resolve(new Blob([this.body])) } this.formData = function() { return Promise.resolve(decode(this.body)) } this.json = function() { var rejected = consumed(this) if (rejected) { return rejected } var body = this.body return new Promise(function(resolve, reject) { try { resolve(JSON.parse(body)) } catch (ex) { reject(ex) } }) } this.text = function() { var rejected = consumed(this) return rejected ? rejected : Promise.resolve(this.body) } return this } function Request(url, options) { options = options || {} this.url = url this.body = options.body this.credentials = options.credentials || null this.headers = new Headers(options.headers) this.method = options.method || 'GET' this.mode = options.mode || null this.referrer = null } function encode(params) { return Object.getOwnPropertyNames(params).filter(function(name) { return params[name] !== undefined }).map(function(name) { var value = (params[name] === null) ? '' : params[name] return encodeURIComponent(name) + '=' + encodeURIComponent(value) }).join('&').replace(/%20/g, '+') } function decode(body) { var form = new FormData() body.trim().split('&').forEach(function(bytes) { if (bytes) { var split = bytes.split('=') var name = split.shift().replace(/\+/g, ' ') var value = split.join('=').replace(/\+/g, ' ') form.append(decodeURIComponent(name), decodeURIComponent(value)) } }) return form } function isObject(value) { try { return Object.getPrototypeOf(value) === Object.prototype } catch (ex) { // Probably a string literal. return false } } function headers(xhr) { var head = new Headers() var pairs = xhr.getAllResponseHeaders().trim().split('\n') pairs.forEach(function(header) { var split = header.trim().split(':') var key = split.shift().trim() var value = split.join(':').trim() head.append(key, value) }) return head } Request.prototype.fetch = function() { var self = this return new Promise(function(resolve, reject) { var xhr = new XMLHttpRequest() xhr.onload = function() { var options = { status: xhr.status, statusText: xhr.statusText, headers: headers(xhr) } resolve(new Response(xhr.responseText, options)) } xhr.onerror = function() { reject() } xhr.open(self.method, self.url) self.headers.forEach(function(name, values) { values.forEach(function(value) { xhr.setRequestHeader(name, value) }) }) var body = self.body if (isObject(self.body)) { xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded; charset=UTF-8') body = encode(self.body) } xhr.send(body) }) } Body.call(Request.prototype) function Response(body, options) { this.body = body this.type = 'default' this.url = null this.status = options.status this.statusText = options.statusText this.headers = options.headers } Body.call(Response.prototype) window.fetch = function (url, options) { return new Request(url, options).fetch() } })(); ================================================ FILE: 05-offline-news/02-single-multi-page/public/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j'+story.title+''; }); return '

    FT Tech Blog

      '+ul+'
    '; } function article(data) { return '

    '+data.title+'

    '+data.body; } if (typeof module == 'object') { module.exports = exports; } else { window.templates = exports; } }()); ================================================ FILE: 05-offline-news/03-hacking-appcache/README.md ================================================ # Hacking AppCache Try to implement AppCache using the same approach we tried for our previous prototypes. Use `chrome://appcache-internals` to see which files have been saved. What happens? Use this snippet to serve an AppCache manifest from express:- ```js app.get('/offline.appcache', function(req, res) { res.set('Content-Type', 'text/cache-manifest'); res.send('CACHE MANIFEST' + '\n./application.js' + '\n./indexeddb.shim.min.js' + '\n./promise.js' + '\n./styles.css' + '\n./fetch.js' + '\n./templates.js' + '\n' + '\nFALLBACK:' + '\n/ /' + '\n' + '\nNETWORK:' + '\n*'); }); ``` (We could do this by creating a static file, `/public/offline.appcache`, but you'll soon see why we've taken this approach) ## AppCache leaks ![Firefox IndexedDB Dev Tools](./multi-masters.png) You will notice that each time you load a URL from the server that page gets _implicitly added_ to the AppCache. This causes a number of problems:- - Every time your user visits a new page on your website, it gets added to the AppCache, endlessly - until the device runs out of space. Possibly fine for a small website, but for most websites (think Wikipedia or FT.com) this would be very, very bad. Remember the browser will treat pages that have different query parameters separately. `/article/11?something=true` is different to `/article/11?something=false` and **both** will get added to the application cache. Forever. - All resources added to the AppCache is then frozen until the next time the AppCache manifest is changed. Your application probably doesn't change as often as your content. At the FT, for example, we publish new content or updates to existing content every few minutes - but only ever update our application every few days. Remember that every time we change the application cache manifest we must re-download every item in the application cache - even if only one file has changed. This is going to use up our customer's data allowances and hit our servers hard. This is really bad. Luckily someone has come up with a workaround. ## IFRAMES to the rescue Rather than referencing your AppCache manifest on every page on your site, you can instead include an iframe that points to a single page that the user never sees that connects to that iframe. ##### [`public/iframe.html`](./public/iframe.html) ```html FT Tech News ``` (Note: you'll probably want to hide `./iframe.html` from search bots) ##### [`index.js`](./index.js) Within the `layoutShell` function add the iframe like this:- ``` + '\n ' ``` Now as you use the web application no unwanted files will be added to AppCache. Except one. --- [← Back to *Single/Multi page app*](../02-single-multi-page) | [Continue to *more hacking appcache* →](../04-more-hacking-appcache) ================================================ FILE: 05-offline-news/03-hacking-appcache/index.js ================================================ require('es6-promise').polyfill(); require('isomorphic-fetch'); var api = 'https://offline-news-api.herokuapp.com/stories'; var port = 8080; var express = require('express'); var path = require('path'); var templates = require('./public/templates'); var app = express(); app.use(express.static(path.join(__dirname, 'public'))); app.get('/offline.appcache', function(req, res) { res.set('Content-Type', 'text/cache-manifest'); res.send('CACHE MANIFEST' + '\n./application.js' + '\n./indexeddb.shim.min.js' + '\n./promise.js' + '\n./styles.css' + '\n./fetch.js' + '\n./templates.js' + '\n' + '\nFALLBACK:' + '\n/ /' + '\n' + '\nNETWORK:' + '\n*'); }); app.get('/article/:guid', function(req, res) { fetch(api+'/'+req.params.guid) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.article(data) })); }, function(err) { res.status(404); res.send(layoutShell({ main: templates.article({ title: 'Story cannot be found', body: '

    Please try another

    ' }) })); }); }); app.get('/', function(req, res) { fetch(api) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.list(data) })); }, function(err) { res.status(404).end(); }); }); function layoutShell(data) { data = { title: data && data.title || 'FT Tech News', main: data && data.main || '' }; return '' + '\n' + '\n ' + '\n '+data.title+'' + '\n ' + '\n ' + '\n ' + '\n
    '+data.main+'
    ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n'; } app.listen(port); console.log('listening on port', port); ================================================ FILE: 05-offline-news/03-hacking-appcache/package.json ================================================ { "dependencies": { "cookie-parser": "^1.3.3", "es6-promise": "^2.0.0", "express": "^4.8.8", "isomorphic-fetch": "^1.0.0" } } ================================================ FILE: 05-offline-news/03-hacking-appcache/public/application.js ================================================ (function() { var api = 'https://offline-news-api.herokuapp.com/stories'; var synchronizeInProgress; var db, main; databaseOpen() .then(function() { main = document.querySelector('main'); document.body.addEventListener('click', onClick); window.addEventListener('popstate', refreshView); // Only refresh the view if the view is empty if (main.innerHTML === '') return refreshView(); }) .then(synchronize); function onClick(e) { if (e.target.classList.contains('js-link')) { e.preventDefault(); history.pushState({}, '', e.target.getAttribute('href')); refreshView(); } } function refreshView() { var guidMatches = location.pathname.match(/^\/article\/([0-9]+)/); if (!guidMatches) { renderAllStories(); return databaseStoriesGet().then(renderAllStories); } renderOneStory(); return databaseStoriesGetById(guidMatches[1]).then(renderOneStory); } function renderAllStories(stories) { main.innerHTML = templates.list(stories); } function renderOneStory(story) { if (!story) story = { title: 'Story cannot be found', body: '

    Please try another

    ' }; main.innerHTML = templates.article(story); } function synchronize() { if (synchronizeInProgress) return synchronizeInProgress; synchronizeInProgress = Promise.all([serverStoriesGet(), databaseStoriesGet()]) .then(function(results) { var promises = []; var remoteStories = results[0]; var localStories = results[1]; // Add new stories downloaded from server to the database promises = promises.concat(remoteStories.map(function(story) { if (!arrayContainsStory(localStories, story)) { return databaseStoriesPut(story); } })); // Delete stories that are no longer on the server from the database promises = promises.concat(localStories.map(function(story) { if (!arrayContainsStory(remoteStories, story)) { return databaseStoriesDelete(story); } })); return promises; }) // Only refresh the view if it's listing page .then(function(results) { if (location.pathname === '/') { return refreshView(); } }) .then(function() { synchronizeInProgress = undefined; }); } function arrayContainsStory(array, story) { return array.some(function(arrayStory) { return arrayStory.guid === story.guid; }); } function databaseOpen() { return new Promise(function(resolve, reject) { var version = 1; var request = indexedDB.open('news-server-rendered', version); request.onupgradeneeded = function(e) { db = e.target.result; e.target.transaction.onerror = reject; db.createObjectStore('stories', { keyPath: 'guid' }); }; request.onsuccess = function(e) { db = e.target.result; resolve(); }; request.onerror = reject; }); } function databaseStoriesPut(story) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var request = store.put(story); request.onsuccess = resolve; request.onerror = reject; }); } function databaseStoriesGet() { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var keyRange = IDBKeyRange.lowerBound(0); // Using reverse direction because the index being sorted on // ends with a numerical incrementing ID so to get newest news // first you need to sort by largest first. var cursorRequest = store.openCursor(keyRange, 'prev'); var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; if (result) { data.push(result.value); result.continue(); } else { resolve(data); } }; }); } function databaseStoriesGetById(guid) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var request = store.get(guid); request.onsuccess = function(e) { var result = e.target.result; resolve(result); }; request.onerror = reject; }); } function databaseStoriesDelete(story) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var request = store.delete(story.guid); request.onsuccess = resolve; request.onerror = reject; }); } function serverStoriesGet(guid) { return fetch(api + '/' + (guid ? guid : '')) .then(function(response) { return response.json(); }); } })(); ================================================ FILE: 05-offline-news/03-hacking-appcache/public/fetch.js ================================================ (function() { 'use strict'; if (window.fetch) { return } function Headers(headers) { this.map = {} var self = this if (headers instanceof Headers) { headers.forEach(function(name, values) { values.forEach(function(value) { self.append(name, value) }) }) } else if (headers) { Object.getOwnPropertyNames(headers).forEach(function(name) { self.append(name, headers[name]) }) } } Headers.prototype.append = function(name, value) { var list = this.map[name] if (!list) { list = [] this.map[name] = list } list.push(value) } Headers.prototype['delete'] = function(name) { delete this.map[name] } Headers.prototype.get = function(name) { var values = this.map[name] return values ? values[0] : null } Headers.prototype.getAll = function(name) { return this.map[name] || [] } Headers.prototype.has = function(name) { return this.map.hasOwnProperty(name) } Headers.prototype.set = function(name, value) { this.map[name] = [value] } // Instead of iterable for now. Headers.prototype.forEach = function(callback) { var self = this Object.getOwnPropertyNames(this.map).forEach(function(name) { callback(name, self.map[name]) }) } function consumed(body) { if (body.bodyUsed) { return Promise.reject(new TypeError('Body already consumed')) } body.bodyUsed = true } function Body() { this.body = null this.bodyUsed = false this.arrayBuffer = function() { throw new Error('Not implemented yet') } this.blob = function() { var rejected = consumed(this) return rejected ? rejected : Promise.resolve(new Blob([this.body])) } this.formData = function() { return Promise.resolve(decode(this.body)) } this.json = function() { var rejected = consumed(this) if (rejected) { return rejected } var body = this.body return new Promise(function(resolve, reject) { try { resolve(JSON.parse(body)) } catch (ex) { reject(ex) } }) } this.text = function() { var rejected = consumed(this) return rejected ? rejected : Promise.resolve(this.body) } return this } function Request(url, options) { options = options || {} this.url = url this.body = options.body this.credentials = options.credentials || null this.headers = new Headers(options.headers) this.method = options.method || 'GET' this.mode = options.mode || null this.referrer = null } function encode(params) { return Object.getOwnPropertyNames(params).filter(function(name) { return params[name] !== undefined }).map(function(name) { var value = (params[name] === null) ? '' : params[name] return encodeURIComponent(name) + '=' + encodeURIComponent(value) }).join('&').replace(/%20/g, '+') } function decode(body) { var form = new FormData() body.trim().split('&').forEach(function(bytes) { if (bytes) { var split = bytes.split('=') var name = split.shift().replace(/\+/g, ' ') var value = split.join('=').replace(/\+/g, ' ') form.append(decodeURIComponent(name), decodeURIComponent(value)) } }) return form } function isObject(value) { try { return Object.getPrototypeOf(value) === Object.prototype } catch (ex) { // Probably a string literal. return false } } function headers(xhr) { var head = new Headers() var pairs = xhr.getAllResponseHeaders().trim().split('\n') pairs.forEach(function(header) { var split = header.trim().split(':') var key = split.shift().trim() var value = split.join(':').trim() head.append(key, value) }) return head } Request.prototype.fetch = function() { var self = this return new Promise(function(resolve, reject) { var xhr = new XMLHttpRequest() xhr.onload = function() { var options = { status: xhr.status, statusText: xhr.statusText, headers: headers(xhr) } resolve(new Response(xhr.responseText, options)) } xhr.onerror = function() { reject() } xhr.open(self.method, self.url) self.headers.forEach(function(name, values) { values.forEach(function(value) { xhr.setRequestHeader(name, value) }) }) var body = self.body if (isObject(self.body)) { xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded; charset=UTF-8') body = encode(self.body) } xhr.send(body) }) } Body.call(Request.prototype) function Response(body, options) { this.body = body this.type = 'default' this.url = null this.status = options.status this.statusText = options.statusText this.headers = options.headers } Body.call(Response.prototype) window.fetch = function (url, options) { return new Request(url, options).fetch() } })(); ================================================ FILE: 05-offline-news/03-hacking-appcache/public/iframe.html ================================================ FT Tech News ================================================ FILE: 05-offline-news/03-hacking-appcache/public/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j'+story.title+''; }); return '

    FT Tech Blog

      '+ul+'
    '; } function article(data) { return '

    '+data.title+'

    '+data.body; } if (typeof module == 'object') { module.exports = exports; } else { window.templates = exports; } }()); ================================================ FILE: 05-offline-news/04-more-hacking-appcache/README.md ================================================ # More hacking AppCache We've solved the AppCache problem for all pages except for the index page. The home page of our application (`http://localhost:8080/`) is still cached in the AppCache, which means in order for us to download a new version we need to change the AppCache manifest. There are a few hacks we could do here:- We could add a timestamp to the AppCache manifest that changes whenever we make a change to our website. This is hacky and would mean every content change would require our users to download our entire application every time we publish some new content. Also this approach would duplicate the data we *already have stored offline* in IndexedDB. This is not only wasteful but could mean that the articles stored in IndexedDB and the list of articles stored in AppCache could drift out of date with each other. It violates our basic principle to only store application files in AppCache - and store only content in the client side database. More hacking is required. ## Getting more control over AppCache What if we could stored an empty shell of the application in AppCache and only use the server-side rendered pages for the initial load? It's hacky, but possible:- - Change the endpoints that return pages (either `/` or `/article/:guid` in our express app to return an empty shell if a special cooke (we'll call it `up`). - Don't include the AppCache iframe loader by default - instead use JavaScript to insert it after setting the `up` cookie. Remove the iframe once the AppCache update process is complete. There's one more complication:- - When a web app is loaded from the Application Cache, it will implicitly try to do AppCache update - even if that page itself doesn't have a `manifest` attribute. Therefore if the `up` cookie is not set and `/offline.appcache` is requested we will return a `400` response. Warning: do not return a 410 because that will cause the user's device to delete the AppCache and the web app will no longer work offline. ## Reacting to the `up` cookie in express. - Add the express [cookie-parser](https://github.com/expressjs/cookie-parser) middleware. - Remove the iframe from `layoutShell`. - Respond to `GET /` and `GET /articles/:guid` with an empty shell of the application if the `up` cookie is set - Respond to `GET /offline.appcache` with `400 Bad Request` if the `up` cookie is not set - Add JavaScript to add the AppCache iframe when the application starts and remove it again once the AppCache update is complete. ##### [`/index.js`](./index.js) ```js require('es6-promise').polyfill(); require('isomorphic-fetch'); var api = 'https://offline-news-api.herokuapp.com/stories'; var port = 8080; var cookieParser = require('cookie-parser'); var express = require('express'); var path = require('path'); var templates = require('./public/templates'); var app = express(); app.use(cookieParser()); app.use(express.static(path.join(__dirname, 'public'))); app.get('/offline.appcache', function(req, res) { if (req.cookies.up) { res.set('Content-Type', 'text/cache-manifest'); res.send('CACHE MANIFEST' + '\n./appcache.js' + '\n./application.js' + '\n./iframe.js' + '\n./indexeddb.shim.min.js' + '\n./promise.js' + '\n./styles.css' + '\n./fetch.js' + '\n./templates.js' + '\n' + '\nFALLBACK:' + '\n/ /' + '\n' + '\nNETWORK:' + '\n*'); } else { res.status(400).end(); } }); // Add middleware to send this when the appcache update cookie is set app.get('/', offlineMiddleware); app.get('/article/:guid', offlineMiddleware); function offlineMiddleware(req, res, next) { if (req.cookies.up) res.send(layoutShell()); else next(); } app.get('/article/:guid', function(req, res) { fetch(api+'/'+req.params.guid) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.article(data) })); }, function(err) { res.status(404); res.send(layoutShell({ main: templates.article({ title: 'Story cannot be found', body: '

    Please try another

    ' }) })); }); }); app.get('/', function(req, res) { fetch(api) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.list(data) })); }, function(err) { res.status(404).end(); }); }); function layoutShell(data) { data = { title: data && data.title || 'FT Tech News', main: data && data.main || '' }; return '' + '\n' + '\n ' + '\n '+data.title+'' + '\n ' + '\n ' + '\n ' + '\n
    '+data.main+'
    ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n'; } app.listen(port); console.log('listening on port', port); ``` ##### [`public/iframe.html`](./public/iframe.html) ```html FT Tech News ``` ##### [`public/iframe.js`](./public/iframe.js) ```js (function() { "use strict"; var checkTimer = null, ac = window.applicationCache, status = null, hasChecked = false, loopMax = 60; function checkNow() { if (ac.status === ac.CHECKING || ac.status === ac.DOWNLOADING || ac.status === ac.UPDATEREADY) { hasChecked = true; } if (ac.status !== status) { status = ac.status; trigger(status, hasChecked); } if (loopMax--) { checkIn(1000); } else { trigger(-1, hasChecked); } } function checkIn(ms) { if (checkTimer) clearTimeout(checkTimer); checkTimer = setTimeout(checkNow, ms); } function trigger(evt, hasChecked) { if (parent && parent.window) { parent.window.postMessage({ type: 'appcache:event', args: [evt, hasChecked] }, '*'); } } ac.addEventListener('updateready', checkNow); ac.addEventListener('cached', checkNow); ac.addEventListener('checking', checkNow); ac.addEventListener('downloading', checkNow); ac.addEventListener('error', checkNow); ac.addEventListener('noupdate', checkNow); ac.addEventListener('obsolete', checkNow); ac.addEventListener('progress', checkNow); checkIn(250); }()); ``` ##### [`public/appcache.js`](./public/appcache.js) ```js (function() { var cookie = 'up'; var statuses = { "-1": 'timeout', "0": 'uncached', "1": 'idle', "2": 'checking', "3": 'downloading', "4": 'updateready', "5": 'obsolete' }; // Start the AppCache loading process when this file executes load(); function onMessage(event) { if (event.data && event.data.type && event.data.type === 'appcache:event') { onEvent.apply(window, event.data.args || []); } } function load() { window.addEventListener("message", onMessage, false); // HACK: Set a cookie so that the application // root returns a Javascript bootstrap rather // than content. var cookieExpires = new Date(new Date().getTime() + 60 * 5 * 1000); document.cookie = cookie + "=1;path=/;expires=" + cookieExpires.toGMTString(); var iframe = document.createElement('IFRAME'); iframe.setAttribute('style', 'width:0px; height:0px; visibility:hidden; position:absolute; border:none'); iframe.setAttribute('src', '/iframe.html'); iframe.setAttribute('id', 'appcache'); document.body.appendChild(iframe); } function onEvent(eventCode) { var s = statuses[eventCode], loaderEl, cookieExpires; if (s === 'uncached' || s === 'idle' || s === 'obsolete' || s === 'timeout' || s === 'updateready') { loaderEl = document.getElementById('appcache'); loaderEl.parentNode.removeChild(loaderEl); // Remove appcacheUpdate cookie cookieExpires = new Date(new Date().getTime() - 60 * 5 * 1000); document.cookie = cookie + "=;path=/;expires=" + cookieExpires.toGMTString(); // Remove message listener window.removeEventListener("message", onMessage); } } }()); ``` ## Exercises - If you work on a reasonably sized website that uses a CDN to cache content what side effects could this have? --- [← Back to *Hacking AppCache*](../03-hacking-appcache) | [Continue to *Success!* →](../05-success) ================================================ FILE: 05-offline-news/04-more-hacking-appcache/index.js ================================================ require('es6-promise').polyfill(); require('isomorphic-fetch'); var api = 'https://offline-news-api.herokuapp.com/stories'; var port = 8080; var cookieParser = require('cookie-parser'); var express = require('express'); var path = require('path'); var templates = require('./public/templates'); var app = express(); app.use(cookieParser()); app.use(express.static(path.join(__dirname, 'public'))); app.get('/offline.appcache', function(req, res) { if (req.cookies.up) { res.set('Content-Type', 'text/cache-manifest'); res.send('CACHE MANIFEST' + '\n./appcache.js' + '\n./application.js' + '\n./iframe.js' + '\n./indexeddb.shim.min.js' + '\n./promise.js' + '\n./styles.css' + '\n./fetch.js' + '\n./templates.js' + '\n' + '\nFALLBACK:' + '\n/ /' + '\n' + '\nNETWORK:' + '\n*'); } else { res.status(400).end(); } }); // Add middleware to send this when the appcache update cookie is set app.get('/', offlineMiddleware); app.get('/article/:guid', offlineMiddleware); function offlineMiddleware(req, res, next) { if (req.cookies.up) res.send(layoutShell()); else next(); } app.get('/article/:guid', function(req, res) { fetch(api+'/'+req.params.guid) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.article(data) })); }, function(err) { res.status(404); res.send(layoutShell({ main: templates.article({ title: 'Story cannot be found', body: '

    Please try another

    ' }) })); }); }); app.get('/', function(req, res) { fetch(api) .then(function(response) { return response.json(); }) .then(function(data) { res.send(layoutShell({ main: templates.list(data) })); }, function(err) { res.status(404).end(); }); }); function layoutShell(data) { data = { title: data && data.title || 'FT Tech News', main: data && data.main || '' }; return '' + '\n' + '\n ' + '\n '+data.title+'' + '\n ' + '\n ' + '\n ' + '\n
    '+data.main+'
    ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n ' + '\n'; } app.listen(port); console.log('listening on port', port); ================================================ FILE: 05-offline-news/04-more-hacking-appcache/package.json ================================================ { "dependencies": { "cookie-parser": "^1.3.3", "es6-promise": "^2.0.0", "express": "^4.8.8", "isomorphic-fetch": "^1.0.0" } } ================================================ FILE: 05-offline-news/04-more-hacking-appcache/public/appcache.js ================================================ (function() { var cookie = 'up'; var statuses = { "-1": 'timeout', "0": 'uncached', "1": 'idle', "2": 'checking', "3": 'downloading', "4": 'updateready', "5": 'obsolete' }; // Start the AppCache loading process when this file executes load(); function onMessage(event) { if (event.data && event.data.type && event.data.type === 'appcache:event') { onEvent.apply(window, event.data.args || []); } } function load() { window.addEventListener("message", onMessage, false); // HACK: Set a cookie so that the application // root returns a Javascript bootstrap rather // than content. var cookieExpires = new Date(new Date().getTime() + 60 * 5 * 1000); document.cookie = cookie + "=1;path=/;expires=" + cookieExpires.toGMTString(); var iframe = document.createElement('IFRAME'); iframe.setAttribute('style', 'width:0px; height:0px; visibility:hidden; position:absolute; border:none'); iframe.setAttribute('src', '/iframe.html'); iframe.setAttribute('id', 'appcache'); document.body.appendChild(iframe); } function onEvent(eventCode) { var s = statuses[eventCode], loaderEl, cookieExpires; if (s === 'uncached' || s === 'idle' || s === 'obsolete' || s === 'timeout' || s === 'updateready') { loaderEl = document.getElementById('appcache'); loaderEl.parentNode.removeChild(loaderEl); // Remove appcacheUpdate cookie cookieExpires = new Date(new Date().getTime() - 60 * 5 * 1000); document.cookie = cookie + "=;path=/;expires=" + cookieExpires.toGMTString(); // Remove message listener window.removeEventListener("message", onMessage); } } }()); ================================================ FILE: 05-offline-news/04-more-hacking-appcache/public/application.js ================================================ (function() { var api = 'https://offline-news-api.herokuapp.com/stories'; var synchronizeInProgress; var db, main; databaseOpen() .then(function() { main = document.querySelector('main'); document.body.addEventListener('click', onClick); window.addEventListener('popstate', refreshView); // Only refresh the view if the view is empty if (main.innerHTML === '') return refreshView(); }) .then(synchronize); function onClick(e) { if (e.target.classList.contains('js-link')) { e.preventDefault(); history.pushState({}, '', e.target.getAttribute('href')); refreshView(); } } function refreshView() { var guidMatches = location.pathname.match(/^\/article\/([0-9]+)/); if (!guidMatches) { renderAllStories(); return databaseStoriesGet().then(renderAllStories); } renderOneStory(); return databaseStoriesGetById(guidMatches[1]).then(renderOneStory); } function renderAllStories(stories) { main.innerHTML = templates.list(stories); } function renderOneStory(story) { if (!story) story = { title: 'Story cannot be found', body: '

    Please try another

    ' }; main.innerHTML = templates.article(story); } function synchronize() { if (synchronizeInProgress) return synchronizeInProgress; synchronizeInProgress = Promise.all([serverStoriesGet(), databaseStoriesGet()]) .then(function(results) { var promises = []; var remoteStories = results[0]; var localStories = results[1]; // Add new stories downloaded from server to the database promises = promises.concat(remoteStories.map(function(story) { if (!arrayContainsStory(localStories, story)) { return databaseStoriesPut(story); } })); // Delete stories that are no longer on the server from the database promises = promises.concat(localStories.map(function(story) { if (!arrayContainsStory(remoteStories, story)) { return databaseStoriesDelete(story); } })); return promises; }) // Only refresh the view if it's listing page .then(function(results) { if (location.pathname === '/') { return refreshView(); } }) .then(function() { synchronizeInProgress = undefined; }); } function arrayContainsStory(array, story) { return array.some(function(arrayStory) { return arrayStory.guid === story.guid; }); } function databaseOpen() { return new Promise(function(resolve, reject) { var version = 1; var request = indexedDB.open('news-server-rendered', version); request.onupgradeneeded = function(e) { db = e.target.result; e.target.transaction.onerror = reject; db.createObjectStore('stories', { keyPath: 'guid' }); }; request.onsuccess = function(e) { db = e.target.result; resolve(); }; request.onerror = reject; }); } function databaseStoriesPut(story) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var request = store.put(story); request.onsuccess = resolve; request.onerror = reject; }); } function databaseStoriesGet() { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var keyRange = IDBKeyRange.lowerBound(0); // Using reverse direction because the index being sorted on // ends with a numerical incrementing ID so to get newest news // first you need to sort by largest first. var cursorRequest = store.openCursor(keyRange, 'prev'); var data = []; cursorRequest.onsuccess = function(e) { var result = e.target.result; if (result) { data.push(result.value); result.continue(); } else { resolve(data); } }; }); } function databaseStoriesGetById(guid) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var request = store.get(guid); request.onsuccess = function(e) { var result = e.target.result; resolve(result); }; request.onerror = reject; }); } function databaseStoriesDelete(story) { return new Promise(function(resolve, reject) { var transaction = db.transaction(['stories'], 'readwrite'); var store = transaction.objectStore('stories'); var request = store.delete(story.guid); request.onsuccess = resolve; request.onerror = reject; }); } function serverStoriesGet(guid) { return fetch(api + '/' + (guid ? guid : '')) .then(function(response) { return response.json(); }); } })(); ================================================ FILE: 05-offline-news/04-more-hacking-appcache/public/fetch.js ================================================ (function() { 'use strict'; if (window.fetch) { return } function Headers(headers) { this.map = {} var self = this if (headers instanceof Headers) { headers.forEach(function(name, values) { values.forEach(function(value) { self.append(name, value) }) }) } else if (headers) { Object.getOwnPropertyNames(headers).forEach(function(name) { self.append(name, headers[name]) }) } } Headers.prototype.append = function(name, value) { var list = this.map[name] if (!list) { list = [] this.map[name] = list } list.push(value) } Headers.prototype['delete'] = function(name) { delete this.map[name] } Headers.prototype.get = function(name) { var values = this.map[name] return values ? values[0] : null } Headers.prototype.getAll = function(name) { return this.map[name] || [] } Headers.prototype.has = function(name) { return this.map.hasOwnProperty(name) } Headers.prototype.set = function(name, value) { this.map[name] = [value] } // Instead of iterable for now. Headers.prototype.forEach = function(callback) { var self = this Object.getOwnPropertyNames(this.map).forEach(function(name) { callback(name, self.map[name]) }) } function consumed(body) { if (body.bodyUsed) { return Promise.reject(new TypeError('Body already consumed')) } body.bodyUsed = true } function Body() { this.body = null this.bodyUsed = false this.arrayBuffer = function() { throw new Error('Not implemented yet') } this.blob = function() { var rejected = consumed(this) return rejected ? rejected : Promise.resolve(new Blob([this.body])) } this.formData = function() { return Promise.resolve(decode(this.body)) } this.json = function() { var rejected = consumed(this) if (rejected) { return rejected } var body = this.body return new Promise(function(resolve, reject) { try { resolve(JSON.parse(body)) } catch (ex) { reject(ex) } }) } this.text = function() { var rejected = consumed(this) return rejected ? rejected : Promise.resolve(this.body) } return this } function Request(url, options) { options = options || {} this.url = url this.body = options.body this.credentials = options.credentials || null this.headers = new Headers(options.headers) this.method = options.method || 'GET' this.mode = options.mode || null this.referrer = null } function encode(params) { return Object.getOwnPropertyNames(params).filter(function(name) { return params[name] !== undefined }).map(function(name) { var value = (params[name] === null) ? '' : params[name] return encodeURIComponent(name) + '=' + encodeURIComponent(value) }).join('&').replace(/%20/g, '+') } function decode(body) { var form = new FormData() body.trim().split('&').forEach(function(bytes) { if (bytes) { var split = bytes.split('=') var name = split.shift().replace(/\+/g, ' ') var value = split.join('=').replace(/\+/g, ' ') form.append(decodeURIComponent(name), decodeURIComponent(value)) } }) return form } function isObject(value) { try { return Object.getPrototypeOf(value) === Object.prototype } catch (ex) { // Probably a string literal. return false } } function headers(xhr) { var head = new Headers() var pairs = xhr.getAllResponseHeaders().trim().split('\n') pairs.forEach(function(header) { var split = header.trim().split(':') var key = split.shift().trim() var value = split.join(':').trim() head.append(key, value) }) return head } Request.prototype.fetch = function() { var self = this return new Promise(function(resolve, reject) { var xhr = new XMLHttpRequest() xhr.onload = function() { var options = { status: xhr.status, statusText: xhr.statusText, headers: headers(xhr) } resolve(new Response(xhr.responseText, options)) } xhr.onerror = function() { reject() } xhr.open(self.method, self.url) self.headers.forEach(function(name, values) { values.forEach(function(value) { xhr.setRequestHeader(name, value) }) }) var body = self.body if (isObject(self.body)) { xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded; charset=UTF-8') body = encode(self.body) } xhr.send(body) }) } Body.call(Request.prototype) function Response(body, options) { this.body = body this.type = 'default' this.url = null this.status = options.status this.statusText = options.statusText this.headers = options.headers } Body.call(Response.prototype) window.fetch = function (url, options) { return new Request(url, options).fetch() } })(); ================================================ FILE: 05-offline-news/04-more-hacking-appcache/public/iframe.html ================================================ FT Tech News ================================================ FILE: 05-offline-news/04-more-hacking-appcache/public/iframe.js ================================================ (function() { "use strict"; var checkTimer = null, ac = window.applicationCache, status = null, hasChecked = false, loopMax = 60; function checkNow() { if (ac.status === ac.CHECKING || ac.status === ac.DOWNLOADING || ac.status === ac.UPDATEREADY) { hasChecked = true; } if (ac.status !== status) { status = ac.status; trigger(status, hasChecked); } if (loopMax--) { checkIn(1000); } else { trigger(-1, hasChecked); } } function checkIn(ms) { if (checkTimer) clearTimeout(checkTimer); checkTimer = setTimeout(checkNow, ms); } function trigger(evt, hasChecked) { if (parent && parent.window) { parent.window.postMessage({ type: 'appcache:event', args: [evt, hasChecked] }, '*'); } } ac.addEventListener('updateready', checkNow); ac.addEventListener('cached', checkNow); ac.addEventListener('checking', checkNow); ac.addEventListener('downloading', checkNow); ac.addEventListener('error', checkNow); ac.addEventListener('noupdate', checkNow); ac.addEventListener('obsolete', checkNow); ac.addEventListener('progress', checkNow); checkIn(250); }()); ================================================ FILE: 05-offline-news/04-more-hacking-appcache/public/promise.js ================================================ !function(){var a,b,c,d;!function(){var e={},f={};a=function(a,b,c){e[a]={deps:b,callback:c}},d=c=b=function(a){function c(b){if("."!==b.charAt(0))return b;for(var c=b.split("/"),d=a.split("/").slice(0,-1),e=0,f=c.length;f>e;e++){var g=c[e];if(".."===g)d.pop();else{if("."===g)continue;d.push(g)}}return d.join("/")}if(d._eak_seen=e,f[a])return f[a];if(f[a]={},!e[a])throw new Error("Could not find module "+a);for(var g,h=e[a],i=h.deps,j=h.callback,k=[],l=0,m=i.length;m>l;l++)"exports"===i[l]?k.push(g={}):k.push(b(c(i[l])));var n=j.apply(this,k);return f[a]=g||n}}(),a("promise/all",["./utils","exports"],function(a,b){"use strict";function c(a){var b=this;if(!d(a))throw new TypeError("You must pass an array to all.");return new b(function(b,c){function d(a){return function(b){f(a,b)}}function f(a,c){h[a]=c,0===--i&&b(h)}var g,h=[],i=a.length;0===i&&b([]);for(var j=0;j'+story.title+''; }); return '

    FT Tech Blog

      '+ul+'
    '; } function article(data) { return '

    '+data.title+'

    '+data.body; } if (typeof module == 'object') { module.exports = exports; } else { window.templates = exports; } }()); ================================================ FILE: 05-offline-news/05-success/README.md ================================================ # Success! We have achieved the goal of creating an online-first news app that works on all major browsers and has a unique URL for every page that can render both on the server and the client. But that success is bittersweet. We've had to add hacks into every layer of our application - and we won't stop facing reprocussions for adding those hacks. We need to be careful any time we make changes to any part of the application - even our CDN will need to be aware of what the `up` cookie means. This is not a good place to be. Although we've unlocked the ultimate offline user experience maintaining this complex solution is almost guaranteed to cause us problems. Luckily a better way is coming. --- [← Back to *More hacking AppCache*](../04-more-hacking-appcache) | [Continue to *Offline news with Service Worker* →](../../06-offline-news-with-service-worker) ================================================ FILE: 05-offline-news/README.md ================================================ # Offline news website We’re going to make a simple offline-first to-do application with HTML5 technology. Here is what the app will do: - store data offline and load without an Internet connection; - automatically download the latest stories when there is a good connection; - run on the first- and second-most recent versions of all major desktop and mobile browsers; - list the latest news headlines as well as allow users to click into an article; - **have a unique URL for each of the articles, and the list of articles page.** The last point is highlighted because it's the key distinguishing feature that makes making a website work offline different, and substantially harder, (with AppCache) than "single page apps" like the one described in the previous section. ## Why news? Content-based websites (which includes websites other than news sites, such as Wikipedia, where the focus is on **reading** rather than **doing** make up a huge proportion of the world wide web) but are a use case that stretches AppCache to its limits. This makes building a simple offline news website a good example to explore AppCache and its replacement, Service Worker, in depth. Also it will be possible to takes these tips and tricks and apply them **immediately** to most websites. --- [← Back to *success*](../04-offline-todo-with-sync/05-success) | [Continue to *scaffolding* →](01-scaffolding) ================================================ FILE: 06-offline-news-with-service-worker/01-scaffolding/README.md ================================================ # Scaffolding We're going to go back to the simple implementation of the online only news app we built [at the beginning of part 5](../05-offline-news/01-scaffolding#indexjs). (Well almost, we're going to need to be able to render the entire page - on the client so we'll move `layoutShell` into [`public/templates.js`](./public/templates.js). ## Set up a quick *Express* server In a new folder, create some files and a directory:- - [`/public`](./public) - a new directory - [`/public/styles.css`](./public/style.css) - some simple styles - [`/public/templates.js`](./public/templates.js) - templating functions that will be shared between the server and the client - [`/index.js`](./index.js) - this will be our express server - [`/package.json`](./package.json) - this will be for listing our dependencies, initially all this will need to contain is `{}`. ``` echo {} >> package.json # Neat trick if you're using *nix npm install --save express isomorphic-fetch es6-promise ``` ##### [`public/styles.css`](./public/styles.css) ```css body { margin: 0; padding: 0; font-family: helvetica, sans-serif; } * { box-sizing: border-box; } h1 { padding: 14px 0 14px 0; margin: 0; font-size: 44px; border-bottom: solid 1px #DDD; line-height: 1em; } nav { padding: 14px 0 14px 0; } main { padding: 0 14px; } ul { padding: 0; margin: 0; list-style: none; } li { padding: 20px 0 20px 0; border-bottom: solid 1px #DDD; } ``` This should be very familiar by now. ##### [`public/templates.js`](./public/templates.js) ```js (function() { var exports = { list: list, article: article }; function list(data) { var ul = ''; data.forEach(function(story) { ul += '
  • '+story.title+'
    • '; }); return shell({ main: '

    FT Tech Blog

      '+ul+'
    ' }); } function article(data) { return shell({ title: data.title, main: '

    '+data.title+'

    '+data.body }); } function shell(data) { data = { title: data && data.title || 'FT Tech News', main: data && data.main || '' }; return '' + '\n' + '\n ' + '\n '+data.title+'' + '\n ' + '\n ' + '\n ' + '\n
    '+data.main+'
    ' + '\n ' + '\n ' + '\n ' + '\n'; } if (typeof module == 'object') { module.exports = exports; } else { this.templates = exports; } }()); ``` This is a little different - the `shell` function has been moved to `./public/templates.js` from `./index.js` as the Service Worker will now need to use it on the client side. We've also taken the opportunity to simplify the way the templating functions work. Now when you call `list` or `article` you get an entire HTML page. ##### [`index.js`](./index.js) ```js var port = Number(process.env.PORT || 8080); var api = 'https://offline-news-api.herokuapp.com/stories'; var express = require('express'); var path = require('path'); require('es6-promise').polyfill(); require('isomorphic-fetch'); var templates = require('./public/templates'); var app = express(); app.use(express.static(path.join(__dirname, 'public'))); app.get('/article/:guid', function(req, res) { fetch(api+'/'+req.params.guid) .then(function(response) { return response.json(); }) .then(function(article) { res.send(templates.article({ title: article.title, body: article.body })); }, function(err) { res.status(404); res.send(templates.article({ title: 'Story cannot be found', body: '

    Please try another

    ' })); }); }); app.get('/', function(req, res) { fetch(api) .then(function(response) { return response.json(); }) .then(function(data) { res.send(templates.list(data)); }, function(err) { res.status(404).end(); }); }); app.listen(port); console.log('listening on '+port); ``` --- [← Back to *Offline news with Service Worker*](../) | [Continue to *Registering a Service Worker* →](../02-registering-a-service-worker) ================================================ FILE: 06-offline-news-with-service-worker/01-scaffolding/index.js ================================================ var port = Number(process.env.PORT || 8080); var api = 'https://offline-news-api.herokuapp.com/stories'; var express = require('express'); var path = require('path'); require('es6-promise').polyfill(); require('isomorphic-fetch'); var templates = require('./public/templates'); var app = express(); app.use(express.static(path.join(__dirname, 'public'))); app.get('/article/:guid', function(req, res) { fetch(api+'/'+req.params.guid) .then(function(response) { return response.json(); }) .then(function(article) { res.send(templates.article({ title: article.title, body: article.body })); }, function(err) { res.status(404); res.send(templates.article({ title: 'Story cannot be found', body: '

    Please try another

    ' })); }); }); app.get('/', function(req, res) { fetch(api) .then(function(response) { return response.json(); }) .then(function(data) { res.send(templates.list(data)); }, function(err) { res.status(404).end(); }); }); app.listen(port); console.log('listening on '+port); ================================================ FILE: 06-offline-news-with-service-worker/01-scaffolding/package.json ================================================ { "dependencies": { "es6-promise": "^2.0.0", "express": "^4.10.0", "isomorphic-fetch": "^1.0.0" } } ================================================ FILE: 06-offline-news-with-service-worker/01-scaffolding/public/styles.css ================================================ body { margin: 0; padding: 0; font-family: helvetica, sans-serif; } * { box-sizing: border-box; } h1 { padding: 14px 0 14px 0; margin: 0; font-size: 44px; border-bottom: solid 1px #DDD; line-height: 1em; } nav { padding: 14px 0 14px 0; } main { padding: 0 14px; } ul { padding: 0; margin: 0; list-style: none; } li { padding: 20px 0 20px 0; border-bottom: solid 1px #DDD; } ================================================ FILE: 06-offline-news-with-service-worker/01-scaffolding/public/templates.js ================================================ (function() { var exports = { list: list, article: article }; function list(data) { var ul = ''; data.forEach(function(story) { ul += '
  • '+story.title+'
    • '; }); return shell({ main: '

    FT Tech Blog

      '+ul+'
    ' }); } function article(data) { return shell({ title: data.title, main: '

    '+data.title+'

    '+data.body }); } function shell(data) { data = { title: data && data.title || 'FT Tech News', main: data && data.main || '' }; return '' + '\n' + '\n ' + '\n '+data.title+'' + '\n ' + '\n ' + '\n ' + '\n
    '+data.main+'
    ' + '\n ' + '\n ' + '\n ' + '\n'; } if (typeof module == 'object') { module.exports = exports; } else { this.templates = exports; } }()); ================================================ FILE: 06-offline-news-with-service-worker/02-registering-a-service-worker/README.md ================================================ # Registering a Service Worker ## Enabling Service Workers Service Workers are not enabled by default in web browsers right now. In order to use them in Chrome you must first switch on a feature flag. To do this go to:- ``` chrome://flags ``` … and search for **Enable experimental Web Platform features** and click **Enable**. ![Experimental Web Platform features](https://cloud.githubusercontent.com/assets/825088/4304757/597f461c-3e73-11e4-836f-b44e1da67056.png) ## Registering Service Workers To get a Service Worker attached to our web page we need to use `navigator.serviceWorker.register`. It's API is this:- ```js Promise register(scriptURL, options); ``` All asynchronous APIs related to Service Workers, including `serviceWorker.register`, return [native browser Promises](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) - in fact we can throw away our Promise polyfill because browsers must support Promises before supporting Service Workers. Once you've called `serviceWorker.register` the browser will load the Service Worker at the URL specified by `scriptURL` and fire the `install` event on it. You can listen to that event within the Service Worker like this:- ```js this.oninstall = function(e) { e.waitUntil(/* a promise */); }; ``` Passing a Promise into the `waitUntil` method on the event object tells the browser to `waitUntil` that Promise resolves - and once that promise resolves the Promise returned from `serviceWorker.register` on the page that registered the Service Worker will resolve. ## Accessing Dev Tools for Service Worker ### Chrome ![Service Worker Dev Tools in Chrome](./service-worker-dev-tools.png) ``` chrome://serviceworker-internals ``` Then click **Inspect** ### Firefox Not yet :cry:. https://jakearchibald.github.io/isserviceworkerready/#debugging ## Exercises - Register a Service Worker that listens to the `install` event on our news application. - Find out how to open Dev Tools in a Service Worker and experiment with setting breakpoints. ## Implementing Service Worker in the news app ##### [`public/application.js`](./public/application) ```js (function() { if ('serviceWorker' in navigator) { navigator.serviceWorker.register('/service-worker.js'); } }()); ``` ##### [`public/service-worker.js`](./public/service-worker.js) ```js console.log("I am a Service Worker"); this.oninstall = function(e) { // TODO: Something interesting console.log("The Service Worker has been installed"); }; ``` ## Other Service Worker events There are other events that you can listen to from within Service Worker, the most useful of which is `fetch`. This event allows you to hook into and/or respond to requests for pages on your website. You can even set breakpoints from within Dev Tools and respond to requests manually from the console. ```js this.onfetch = function(e) { debugger; event.waitUntil(new Response("Hello world!")); }; ``` ![The power of fetch](./breakpoint.gif) --- [← Back to *Scaffolding*](../01-scaffolding) | [Continue to *Service Worker Cache* →](../03-service-worker-caches) ================================================ FILE: 06-offline-news-with-service-worker/02-registering-a-service-worker/index.js ================================================ var port = Number(process.env.PORT || 8080); var api = 'https://offline-news-api.herokuapp.com/stories'; var express = require('express'); var path = require('path'); require('es6-promise').polyfill(); require('isomorphic-fetch'); var templates = require('./public/templates'); var app = express(); app.use(express.static(path.join(__dirname, 'public'))); app.get('/article/:guid', function(req, res) { fetch(api+'/'+req.params.guid) .then(function(response) { return response.json(); }) .then(function(article) { res.send(templates.article({ title: article.title, body: article.body })); }, function(err) { res.status(404); res.send(templates.article({ title: 'Story cannot be found', body: '

    Please try another

    ' })); }); }); app.get('/', function(req, res) { fetch(api) .then(function(response) { return response.json(); }) .then(function(data) { res.send(templates.list(data)); }, function(err) { res.status(404).end(); }); }); app.listen(port); console.log('listening on '+port); ================================================ FILE: 06-offline-news-with-service-worker/02-registering-a-service-worker/package.json ================================================ { "dependencies": { "es6-promise": "^2.0.0", "express": "^4.10.0", "isomorphic-fetch": "^1.0.0" } } ================================================ FILE: 06-offline-news-with-service-worker/02-registering-a-service-worker/public/application.js ================================================ (function() { if ('serviceWorker' in navigator) { navigator.serviceWorker.register('/service-worker.js'); } }()); ================================================ FILE: 06-offline-news-with-service-worker/02-registering-a-service-worker/public/service-worker.js ================================================ this.oninstall = function(event) { // TODO: Something interesting console.log("The Service Worker has been installed"); }; this.onfetch = function(event) { debugger; }; ================================================ FILE: 06-offline-news-with-service-worker/02-registering-a-service-worker/public/styles.css ================================================ body { margin: 0; padding: 0; font-family: helvetica, sans-serif; } * { box-sizing: border-box; } h1 { padding: 14px 0 14px 0; margin: 0; font-size: 44px; border-bottom: solid 1px #DDD; line-height: 1em; } nav { padding: 14px 0 14px 0; } main { padding: 0 14px; } ul { padding: 0; margin: 0; list-style: none; } li { padding: 20px 0 20px 0; border-bottom: solid 1px #DDD; } ================================================ FILE: 06-offline-news-with-service-worker/02-registering-a-service-worker/public/templates.js ================================================ (function() { var exports = { list: list, article: article }; function list(data) { var ul = ''; data.forEach(function(story) { ul += '
  • '+story.title+'
    • '; }); return shell({ main: '

    FT Tech Blog

      '+ul+'
    ' }); } function article(data) { return shell({ title: data.title, main: '

    '+data.title+'

    '+data.body }); } function shell(data) { data = { title: data && data.title || 'FT Tech News', main: data && data.main || '' }; return '' + '\n' + '\n ' + '\n '+data.title+'' + '\n ' + '\n ' + '\n ' + '\n
    '+data.main+'
    ' + '\n ' + '\n ' + '\n ' + '\n'; } if (typeof module == 'object') { module.exports = exports; } else { this.templates = exports; } }()); ================================================ FILE: 06-offline-news-with-service-worker/02-registering-a-service-worker/styles.css ================================================ body { margin: 0; padding: 0; font-family: helvetica, sans-serif; } * { box-sizing: border-box; } h1 { padding: 14px 0 14px 0; margin: 0; font-size: 44px; border-bottom: solid 1px #DDD; line-height: 1em; } nav { padding: 14px 0 14px 0; } main { padding: 0 14px; } ul { padding: 0; margin: 0; list-style: none; } li { padding: 20px 0 20px 0; border-bottom: solid 1px #DDD; } ================================================ FILE: 06-offline-news-with-service-worker/02-registering-a-service-worker/templates.js ================================================ (function() { var exports = { list: list, article: article }; function list(data) { var ul = ''; data.forEach(function(story) { ul += '
  • '+story.title+'
    • '; }); return shell({ main: '

    FT Tech Blog

      '+ul+'
    ' }); } function article(data) { return shell({ title: data.title, main: '

    '+data.title+'

    '+data.body }); } function shell(data) { data = { title: data && data.title || 'FT Tech News', main: data && data.main || '' }; return '' + '\n' + '\n ' + '\n '+data.title+'' + '\n ' + '\n ' + '\n ' + '\n
    '+data.main+'
    ' + '\n ' + '\n ' + '\n ' + '\n'; } if (typeof module == 'object') { module.exports = exports; } else { this.templates = exports; } }()); ================================================ FILE: 06-offline-news-with-service-worker/03-service-worker-caches/README.md ================================================ # Sevice Worker Caches ## One tiny problem Service Workers will be really helpful for creating offline apps. Unfortunately they're not yet implemented any browser, even the very latest Firefox Nightlies or Chrome Canaries. A [polyfill (using IndexedDB) has been created](https://github.com/jeffposnick/service-worker-cache), which we will use instead. Due to a bug in Chrome the polyfill object is named `cachesPolyfill` (when Service Workers are released publicly this will change to simply `caches`. All these code samples will use `cachesPolyfill` as that is the syntax that is currently needed. ## API ### `open` Like IndexedDB to create a cache for storing data offline you must call `open` and give it a name. This will return an ES6 promise. ```js cachesPolyfill.open('my-first-sw-cache'); ``` ### `addAll` To cache some URLs use the `addAll` method on the *object that the `cachesPolyfill` promise resolves with*. ```js cachesPolyfill.open('my-first-sw-cache') .then(function(myCache) { myCache.addAll([ '/scripts.js', '/styles.css', '/pic.png' ]); }); ``` ### `match` As we saw in the previous part Service Workers can respond to `fetch` events so all that remains is to demonstrate how to respond to `fetch` events with appropriate content from `caches` and we will have made it work offline. `caches` has a method `match` that will look through **all** the current Service Worker's caches, looking for a piece of content that **matches** the requested URL, method, vary headers. (It will also ignore any cache headers - which is useful as often we would prefer the user to receive *something* even if that resource has technically expired if the alternative is for the user to get *nothing*). ```js this.onfetch = function(event) { event.respondWith( caches.match(event.request).then(function(response) { return response || fetch(event.request); }) ); }; ``` You can also customise how the matching works, [discounting things](https://slightlyoff.github.io/ServiceWorker/spec/service_worker/#cache-query-options-dictionary) such as query string, methods, and vary headers. ## Exercise - Create a version of the offline news application that works offline with Service Worker. [← Back to *Registering a Service Worker*](../02-registering-a-service-worker) | [Continue to *Success* →](../04-success) ================================================ FILE: 06-offline-news-with-service-worker/04-success/README.md ================================================ # Success! ## Exercises - Rebuild the todo app (either with or without sync) with Service Worker. ================================================ FILE: 06-offline-news-with-service-worker/README.md ================================================ # Offline news with Service Worker To get AppCache to work with our offline news website required us to write so many awful confusing hacks:- - Varying the response of every page based on whether a cookie is set; - Returning an HTTP error on the manifest when a cookie is not set; - Created an iframe to load a page that loads the manifest rather than referencing it directly; - Without the `NETWORK: *` hack in your AppCache manifest you will break all non-offline'd resources; - If we deployed this as a real app on to a CDN we would also have to add hacks into the CDN too to vary its cache by cooke. Luckily, a better way is coming. ## Service Workers Service Workers are a new web browser feature that enable developers to write scripts that run independently from web pages, similar to shared workers, and are shared between web pages on the same domain name. - They run **independently** from web pages, in the background - They are **shared** between different URLs on the same domain. - A domain can have **multiple** Service Workers. - They can be **shut down** at the end of events. - They are able to **intercept** and **manipulate** HTTP requests between the web application and the internet. - They have their own mechanism to **cache requests** (in addition to the existing browser cache). - They can enable websites to **work offline** because they are able to respond to requests and respond from a cache without an internet connection. - They will probably [only be permitted on websites served over **https**](https://github.com/slightlyoff/ServiceWorker/issues/199). Service Workers are meant to replace the HTML5 Application Cache. Unlike AppCache, which is controlled by [adding specific entries into a manifest file](https://developer.mozilla.org/en/docs/HTML/Using_the_application_cache), Service Workers are written in Javascript. Or, in visual form:- ![Service Worker Explained](./service-worker-explained.png) ## Architecture For our first Service Worker prototype we're going to take the simplest possible approach. - The back-end will only contain routes for rendering the article list page and the article page. Unlike the AppCache example, it won't be oblivious to the fact the app works offline. - The front-end won't contain any JavaScript (except a line to install the Service Worker) - Which leaves the Service Worker (the middle-end, if you like) to do the following things:- - Download and store the latest news. - Download and store the JavaScript and CSS files needed to run the application. - Respond to requests for either the application files or application pages and render them locally with local data - enabling the application to work offline. --- [← Back to *success*](../05-offline-news/05-success) | [Continue to *scaffolding* →](01-scaffolding) ================================================ FILE: 07-dexie/README.md ================================================ # Introducing [Dexie.js](https://github.com/dfahlander/Dexie.js) [Dexie.js](https://github.com/dfahlander/Dexie.js) is a wrapper around IndexedDB that I believe will become the jQuery of IndexedDB. ## Advantages - Simple **Promise-based** API - Human readable queries `db.friends.where('lastName').anyOf('Helenius', 'Fahlander').each(function(friends) { … })` - Error handling - Supports search - Only one layer away from actual IndexedDB objects - not difficult to migrate code from manual IndexedDB integration to Dexie. - Cross-browser (but you'll still need to polyfill IndexedDB on WebSQL browsers) ## Disadvantages - Slightly funky object store key syntax, but I think we can forgive them for that… ## Key ideas - You no longer need to wait for the database to be opened - start using it as if it were already opened immediately and the internals will take care of the rest - No need to think about transactions (unless you want to) - it's all handled internally ## Todo app IndexedDB integration in Dexie.js ```js […] db.version(1).stores({ todo: '_id' }); db.open() .then(refreshView); function onClick(e) { e.preventDefault(); if (e.target.hasAttribute('id')) { db.todo.where('_id').equals(e.target.getAttribute('id')).delete() .then(refreshView); } } function onSubmit(e) { e.preventDefault(); db.todo.put({ text: input.value, _id: String(Date.now()) }) .then(function() { input.value = ''; }) .then(refreshView); } function refreshView() { return db.todo.toArray() .then(renderAllTodos); } […] ``` ================================================ FILE: 08-success/README.md ================================================ # Success ## Summary Over the course of today we've covered **IndexedDB**, **AppCache** and **ServiceWorker** in depth. - Stop implementing WebSQL code. It's deprecated and Safari is getting it soon. Use the IndexedDB polyfill instead. - Offline applications are possible **today** [across 83% of the web](http://caniuse.com/#search=appcache) with AppCache. - If you're using AppCache I highly recommend only storing **application files** in AppCache and only storing **content** in client database. - Service Worker is coming, [track its status](https://github.com/jakearchibald/isserviceworkerready) and prepare your websites. ================================================ FILE: README.md ================================================ Making it work offline ====================== Making it work offline, the workshop. I ran this workshop at [SmashingConf in Freiburg](http://smashingconf.com/freiburg-2014/workshops/matthew-andrews), at [Imperial College in London](http://www.whiteoctoberevents.co.uk/event/javascript-workshops/making-it-work-offline/) and internally for the [Financial Times](http://ft.com) also in London in Autumn 2014. If you are interested in having a similar workshop run at your company or conference or would like to use these materials to run a workshop yourselves please email me: matt@mattandre.ws / [mattandre.ws](https://mattandre.ws). Contents -------- 1. [Introduction](01-introduction) - [Why?](01-introduction/why.md) - [How?](01-introduction/how.md) - [Client storage technologies](01-introduction/dysfunctional-family.md) 2. [New APIs: Promises & Fetch](02-new-apis) 3. [Offline Todo with IndexedDB](03-offline-todo) - [Scaffolding the application](03-offline-todo/01-scaffolding) - [Opening an IndexedDB database](03-offline-todo/02-opening-a-database) - [Using Dev Tools with Safari, Chrome, IE and Firefox](03-offline-todo/03-using-dev-tools) - [Creating Object Stores](03-offline-todo/04-creating-object-stores) - [Review: `window.indexedDB`](03-offline-todo/05-review-window-indexeddb) - [Adding data](03-offline-todo/06-adding-data) - [Getting data](03-offline-todo/07-getting-data) - [Rendering todos](03-offline-todo/08-rendering-todos) - [Deleting data](03-offline-todo/09-deleting-data) - [Review: `IDBRequest` and `IDBTransaction`](03-offline-todo/10-review-requests-transactions) - [AppCache for full offline experience](03-offline-todo/11-appcache) - [AppCache Gotchas](03-offline-todo/12-appcache-gotcha-1) - [Success!](03-offline-todo/13-success) 4. [Offline Todo with IndexedDB with sync](04-offline-todo-with-sync) - [Architecture](04-offline-todo-with-sync/01-architecture) - [Mark for deletion](04-offline-todo-with-sync/02-mark-for-deletion) - [Adding ajax](04-offline-todo-with-sync/03-adding-ajax) - [Synchronize](04-offline-todo-with-sync/04-synchronize) - [Success!](04-offline-todo-with-sync/05-success) 5. [Building an offline news app, FT style](05-offline-news) - [Scaffolding the application](05-offline-news/01-scaffolding) - [Single/Multi-page app](05-offline-news/02-single-multi-page) - [More hacking AppCache](05-offline-news/03-hacking-appcache) - [Even more hacking AppCache](05-offline-news/04-more-hacking-appcache) - [Success!](05-offline-news/05-success) 6. [Building an offline news app with Service Worker](06-offline-news-with-service-worker) - [Scaffolding](06-offline-news-with-service-worker/01-scaffolding) - [Registering a Service Worker](06-offline-news-with-service-worker/02-registering-a-service-worker) - [Service Worker Cache](06-offline-news-with-service-worker/03-service-worker-caches) - [Success!](06-offline-news-with-service-worker/04-success) 7. [Dexie.js - IndexedDB without the pain](07-dexie) 8. [Success!](08-success) © ================================================ FILE: package.json ================================================ { "name": "workshop-making-it-work-offline", "version": "0.0.0", "description": "Making it work offline ======================", "main": "index.js", "scripts": { "test": "npm run jshint && npm run lintspaces", "jshint": "jshint `npm run js-files | tail -n +5`", "lintspaces": "lintspaces -ntd tabs -i js-comments `npm run js-files | tail -n +5`", "js-files": "find . -name '*.js' ! -path './node_modules/*' ! -name 'fetch.js' ! -name 'promise.js' ! -name 'indexeddb.shim.min.js'" }, "repository": { "type": "git", "url": "git://github.com/matthew-andrews/workshop-making-it-work-offline.git" }, "author": "Matt Andrews ", "license": "proprietary", "bugs": { "url": "https://github.com/matthew-andrews/workshop-making-it-work-offline/issues" }, "homepage": "https://github.com/matthew-andrews/workshop-making-it-work-offline", "private": true, "devDependencies": { "jshint": "^2.5.5", "lintspaces-cli": "0.0.4" } }