Generation of site pages by means of service-riveters

Generation of site pages by means of service-riveters  
( C )
 
 
From this article you will learn how to create a page with a list of previously cached materials on the mobile device, in the browser, so that the conditional, stuck in the elevator user, did not miss the Internet. As we approach the goal, we will touch on the following topics:
 
 
 
caching site pages for offline access;
 
keeping records of pages available offline, transferring necessary data;
 
monitor network status, online or offline;
 
communicating service-vorker with the pages and tabs that it serves;
 
interception of the service-vorkerom request to open the address /offline / and generating a new page directly on the device, without requesting to the server.  

 
If the topic is service-vorkers and Progressive Web Apps (PWA) for you is new, then before reading this article you need to get to know them better.
 
 
My name is Rybin Pavel, I work in front-development of Media Projects Mail.Ru Group. This guide helped me write rake, stuffed cones and pitfalls that I got when implementing PWA for mobile version Auto Mail.Ru .
 
 
In the text there will be small examples of code that illustrate the story. An extended demo version can be look at GitHub .
 
cache on event install , the first of a chain of events in the life cycle.
 
 
    //service-worker.js
//Files that are required offline
const dependencies =[
'/css/app.css',
'/js/offline_page.js',
'/img/logo.png',
'/img/default_thumb.png'
];
//The installation phase, service-vorker is not active yet
self.addEventListener ('install', event => {
//Load all the files that are required for offline mode
) const loadDependencies = self.caches.open ('myApp')
.then (cache = > cache.addAll (dependencies))
//The service-vorcher will proceed to the next stage of its cycle,
//when all necessary files are downloaded and cached
event.waitUntil (loadDependencies);
}) ;

 
The next event is activate . It is useful for us in order to clear the old cache and records in the database. In our example, a simple helper is used to work with IndexedDB. idb-keyval . He and his more pumped brother idb are convenient wrappers, promicifying the work with obsolete morally API IndexedDB .
 
 
    //service-worker.js
import {clear} from 'idb-keyval';
//Files that are required offline
const dependencies =[/* */];
//Activation of
self.addEventListener ('activate', event => {
//clean the records in IndexedDB
.constantClearIDB = clear ();
//clean old cache
.conferenceClearCache = self.caches.open (cacheName)
.then ((cache) => cache.keys ()
.then ((cacheKeys) => Promise.all (cacheKeys.map ((request) => {
//Delete everything except resources from the list of files,
//which are required offline
.constantDelete =! dependencies.includes (request.url);
return canDelete? cache.delete (request, {ignoreVary: true})
: Promise.resolve ();
))));
constantClearAll = Promise.all ([promiseClearIDB, promiseClearCache])
catch (err => console.error (error));
//Life Cycle Service the store manager
//when the cache and IndexedDB
are cleared. event.waitUntil (promiseClearAll);
});

 
After activation, the service-vorker is ready to work. He will be able to process network requests and receive messages from all pages of our site that were opened by after its activation. You just need to add the appropriate event handlers. It is here that we will catch the request for the page /offline / .
 
 
    //service-worker.js
//Process outgoing network requests
self.addEventListener ('fetch', event => {
? const {request} = event;
const url = new URL (request.url);
.
.if (url.origin! == self.location. origin) {
//Alien domain, do not process this request
return fetch (request) .catch (err => console.log (err));
}
//Check if the page was requested /offline /
const isOfflineListRequested = /^/offline//.test(url.pathname);
const response = isOfflineListRequested
//Create a custom response with the page
//the list of materials available offline
? createOfflineListResponse ()
.
//We do a normal query,
//Here depending on the URL, we can apply
//different caching strategies, something to save "for ages",
//something to update each time etc.
: FetchWithOneOrAnotherCacheStrategy (event.request);
event.respondWith (response);
});

 
What are "caching strategies" and why are they needed?
 
 
The resources that we load play a different role on the page. It can be an image with a logo or some kind of shared JS library, which most likely will never change. It can be JSON with comments, which are updated every five minutes.
 
 
The files and documents involved in the construction of the page, in its life cycle, depending on the purpose, can be conditionally divided into groups:
 
 
  •  
  • can be cached "forever";  
  • can not be cached for long;  
  • can be cached, but if possible, update;  
  • and so on, this list is limited only by your imagination and business objectives.  

 
If you implement the filtering of such groups at the address, file type, anything, you can apply to each of them the logic of the mutual work of network requests and the local cache. Several examples of different caching strategies you can Look in the repository of example .
 
 
Now we already have support for offline mode for cached pages. They will open when the phone is activated in flight mode. Now you need to collect them all in one place, on a separate page.
 
 

Registration of pages available offline


 
To draw a page of the list of materials available for viewing offline, this list must first be created, and then, with each opening of a new page, updated. The logic for registering pages will be as follows:
 
 
  •  
  • When the page is opened, we will create the data object describing this page (address, title, preview address to display in the list).  
  • After the formation of the data, we will send them to the service-vorker through postMessage .  
  • The service-vorker will receive data and add them to the general list.  

 
A script that collects data about a page will be executed on it, being a part of the page. So it is more convenient. This will allow you to throw the necessary information, for example, through the block head and get it from the layout. Or in any other way that suits you.
 
 
Let's use the micro-markup Open Graph . Today it is difficult to imagine a site without it. In addition, with its help you can transfer all the necessary information in our case:
 
 

 
Why transfer the page address through the layout? Why not get it in JS via the location object?
 
 
Today, most sites use analytics for all possible get-parameters, marking, for example, the source of traffic. As a result, it turns out that the addresses are /homer.html , /homer.html?utm_source=vk and /homer.html?utm_source=email in fact they lead to the same page, which means that they must be registered in the list once. Here we will be helped by the "canonical" address transmitted through og: url , it will always be the same. Most likely all the necessary og-markup you already have, you can check its completeness using extensions for Google Chrome .
 
 
So, let's teach the page to tell the service-richer that it's loaded. Finish the function registerServiceWorker (see above).
 
 
    //app.js - runs on the site page
function registerServiceWorker () {
navigator.serviceWorker.register ('/service-worker.js')
.then (registration => {
if if (! registration.active) {
//Not yet activated
return;
}
//Service-vorker is activated, it is possible to work with it
//We inform, that the current page is now available offline
registerPageAsCached ();
});
}
/**
* Registers the current page of the site, as available offline
* /
function registerPageAsCached () {
//We will not parse the getPageInfoFromHtml function,
//the main thing is that it should return an object with fields:
//url - the "canonical" address of the page
//title - the name of the page
//thumb - the address of the page thumbnail
const page = getPageInfoFromHtml ();
//Send the data to the service-riveter
postMessage ({
action: 'registerPage',
page
});
}
/**
* Send a message to the service worker
* @param {object} message
* /
function postMessage (message) {
const {controller} = navigator.serviceWorker;
if (controller) {
controller.postMessage (message);
}
}

 
Please note: in the message in addition to the data about the page, we pass the field action , which describes the type of message. This will allow us to transmit different data for different purposes in the future.
 
 
Someone will ask, but how do we know that the page is cached?
 
 
All requests from our site are processed through one of the caching strategies that we introduced earlier, and so we accept that everything that was displayed in the browser passed through the cache.
 
 
We receive the data from the page in the service-rarer:
 
 
    //service-worker.js
import {get, set} from 'idb-keyval';
/*
* Processing of messages from pages
* /
self.addEventListener ('message', event => {
.stat {data = {}} = event;
.const {page} = data;
//Messages can be different,
//split, using action
switch (data.action) {
case 'registerPage':
addToOfflineList (page);
break;
}
});
/**
* Registers the page as available offline
* @param {object} pageInfo
* @return {Promise}
* /
export function addToOfflineList (pageInfo) {
//cache the preview of the page using the appropriate strategy,
//the image is useful in offline mode
if (pageInfo.thumb) {
fetchWithOneOrAnotherCacheStrategy (pageInfo.thumb);
}
//add information about the page in IndexedDB
return get ('cachedPages')
.then ((pages = {}) => set ('cachedPages', {
pages,
.[pageInfo.url]: pageInfo
}));
}

 
The page is registered.
 
 
In this example, we used the header, address, and image to describe the page, but the list of data can be expanded. For example, it makes sense to specify the timestamp of the last update. This will sort the articles by the download date, and also remove old materials from the cache.
 
 

Monitoring the status of the connection to the network


 
While the page is open, we will teach it to monitor the status of the network, or rather, the availability of our server. When the server stops responding, a corresponding message appears with a link to /offline / , which we will do later. Also for convenience we will highlight the available links directly on the page.
 
 
You can make inaccessible materials dim, visually highlighting cached:
 
 

 
 
And you can, on the contrary, highlight the articles stored in the cache with a special icon, as is done on Auto Mail.Ru :
 
 

 
 
In the script running on the page, create the function ping , which will be periodically called through the specified interval and send a message to the service-vorker.
 
 
    //app.js - runs on the site page
const PING_INTERVAL = 10000; //10 seconds
function registerServiceWorker () {
navigator.serviceWorker.register ('/service-worker.js')
. .then (registration => {
if (registration.active) {
//Not activated
return;!
}
//service worker is activated, you can work with him
registerPageAsCached ().; //see above
.
//run the ping
ping ();
});
}
/**
* Periodic verification of the availability of the network (our server)
* /
function ping () {
postMessage ({
action: 'ping',
});
setTimeout (ping, PING_INTERVAL);
}
/**
* Send a message to the service worker
* @param {object} message
* /
function postMessage (message) {
const {controller} = navigator.serviceWorker;
if (controller) {
controller.postMessage (message);
}
}

 
On the service-vorker side, we will receive a message, check the availability of the server and send back the report. For verification, you can request any URL, it's better if it is some kind of statics, for example, a traditional pixel.
 
 
    //service-worker.js
import {get} from 'idb-keyval';
/*
* Processing of messages from pages
* /
self.addEventListener ('message', event => {
.stat {data = {}} = event;
.const {page} = data;
//Messages can be different,
//split, using action
switch (data.action) {
case 'ping':
ping ();
break;
}
});
/**
* Check the availability of the server
* /
export function ping () {
fetch ('/ping.gif'). then (
() => pingHandler (true),
() => pingHandler (false)
);
}
/**
* Logs and sends a message to page
* about the success or failure of ping
* @param {boolean} isOnline
* /
function pingHandler (isOnline) {
postMessage ({
action: 'ping',
online: isOnline,
});
}
/**
* Sends data to all pages and tabs,
* Serviced by a service-riveter
* @param {object} message
* /
function postMessage (message) {
//Find all the open pages and tabs of our site
self.clients.matchAll (). then (clients => {
//If there is no network added to the message
//list of cached pages
const offlinePagesPromise = message.online?
Promise.resolve ()
.: get ( 'cachedPages');
offlinePagesPromise.then (offlinePages => {
if (offlinePages) {
message.offlinePages = offlinePages;
}
clients.forEach (client => {
//The client can disappear, we do check
If (client) {
Client.postMessage (message);
}
});
});
});
}

 
In the browser can be opened several tabs with our site. Each of them will call its own method ping . Therefore, it is better to load a pixel not from the page, but through a service-vorker, which can control the frequency of the verification network requests, for example, through throttle microtemperature . Also, knowledge of the status can be useful to the service-vorker himself.
 
 
The page, after receiving the report, produces the necessary manipulations with its contents:
 
 
    //app.js - runs on the site page
let isOnline = true;
function registerServiceWorker () {
navigator.serviceWorker.register ('/service-worker.js')
.!. .then (registration => {
if (registration.active) {
//Not activated
return;
}
//Podpisyvatesya to messages from service worker`s
serviceWorker.addEventListener ( 'message', handleMessage);
.
registerPageAsCached (); //see above
ping (); //see above
});
}
/**
* Processing of the message from the service worker
* @param {MessageEvent} e
* /
function handleMessage (e) {
const {data} = e;
if (data.action === 'ping' && isOnline! == data.online) {
isOnline = data.online;
toggleNetworkState (data);
}
}
/**
* Toggles the online /offline status of the
page. * @param {object} params
* /
function toggleNetworkState (params) {
const {online, offlinePages = {}} = params;
//We hang the global modifier,
//in our example it will make all links "fade"
document.body.classList.toggle ('offline',! online);
//For offline mode, highlight the cached links
if (! online) {
Array.from (document.links) .forEach (link => {
const href = link.getAttribute ('href');
Const isCached = !! offlinePages[href]|| href === '/offline /' ;
Link.classList.toggle ('cached', isCached);
});
}
}

 

Creation of the page /offline /service-grinder


 
So, we got to the main thing, before creating a page inside the service-grinder without accessing the server. We need a template that draws HTML, and data about cached pages.
 
 
I'm at demo-version used a simple template pug . However, you can use any other up to "server render" for an isomorphic application on React.
 
 
My template looks like this:
 
 
    html (lang = "en")
head
title Available offline
link (rel = "stylesheet" href = "/css /app.css")
body
section.layout
header.layout__header
a.layout__header__logo (href = "/")
h1 You can read it offline
ul.articles-list
each page in pages
li.articles-list__item
a (href = page.url)
if page.thumb
img.avatar (src = page.thumb alt = "")
else
img.avatar (src = "/img /default_thumb.png" alt = "")
span = page.title

 
In the service-broker, in the event handler fetch select the query to /offline / and return to the unsuspecting browser a freshly created page:
 
 
    //service-worker.js
import {get} from 'idb-keyval';
const template = require ('offlinePage.pug');
//Process outgoing network requests
self.addEventListener ('fetch', event => {
? const {request} = event;
const url = new URL (request.url);
.
.if (url.origin! == self.location. origin) {
//Alien domain, do not process this request
return fetch (request) .catch (err => console.log (err));
}
//Check if the page was requested /offline /
const isOfflineListRequested = /^/offline//.test(url.pathname);
const response = isOfflineListRequested
//Our case
//Create a custom response to the page
//! The list of materials available offline is
? createOfflineListResponse ()
.
//Common query
: fetchWithOneOrAnotherCacheStrategy (event.request);
3r3r 3901. event.respondWith (response);
});
/**
* Forms a response object with page information,
* Available offline
* @return {Promise }
* /
function createOfflineListResponse () {
//Get the information about the offline pages
return get ('cachedPages')
.then ((pagesList = {}) => {
//Pass in the template list of the available pages
const html = template ({
pages: Object.values ​​(pagesList)
});
//Create and return the response object
const blob = new Blob ([html] , {
type: 'text /html; charset = utf-8 '
});
return new Response (blob, {
status: 20?
statusText: 'OK'
});
}). catch (err => console.error (err));
}

 
Result:
 
 

 
 

Finally


 
In order not to inflate this manual, some topics had to be omitted. Nevertheless, they are extremely important. The most important thing is cleaning the cache. This should be done regularly and independently, otherwise the place provided by the browser will end.
 
 
It makes sense to keep resources in the cache that are required regularly: CSS, JS, images of interface elements. For the rest, one should come up with some kind of "rule of extinction". For example, delete everything that was not requested for more than three (five, ten, year?) Days.
 
 
For the convenience of debugging, detailed journaling of each stage is useful. To do this, you can create your own utility log , which inside can turn on /off the flag from the environment, and output information through it. Unlike the pages, the service-vorker continues to live between their reboots and closing, so I recommend that in the developer tools console include the checkbox Preserve log .
 
 
Thank you for reading this line. Write questions, ideas and insights from personal experience in the comments. If it turns out that the topic is in demand (I have fears that it is too narrow), I will continue it.
 
 

Useful links


 
 
Demo version of the tutorial on GitHub
 
Service Worker API
 
Cache
 
IndexedDB
 
Utilities for working with IndexedDB: idb and idb-keyval
 
Extension To test Open Graph-markup
 
Optimization micropropaths JS
 
+ 0 -

Add comment