SEO & Progressive Web Apps: Looking to the Future

Practitioners of SEO have at all times been mistrustful of JavaScript.

This is partly primarily based on expertise; the capability of search engines like google and yahoo to uncover, crawl, and precisely index content material which is closely reliant on JavaScript has traditionally been poor. But it’s additionally routine, born of a normal wariness in the direction of JavaScript in all its types that isn’t primarily based on understanding or expertise. This manifests itself as dependence on conventional SEO methods that haven’t been related for years, and a conviction that to be good at technical SEO doesn’t require an understanding of contemporary internet growth.

As Mike King wrote in his publish The Technical SEO Renaissance, these attitudes are contributing to “an ever-growing technical knowledge gap within SEO as a marketing field, making it difficult for many SEOs to solve our new problems”. They additionally put SEO practitioners susceptible to being left behind, since too many people refuse to discover – not to mention embrace – applied sciences akin to Progressive Web Apps (PWAs), trendy JavaScript frameworks, and different such developments that are more and more being seen as the way forward for the internet.

In this text, I’ll be taking a contemporary have a look at PWAs. As effectively as exploring implications for each SEO and usefulness, I’ll be showcasing some trendy frameworks and construct instruments which you’ll not have heard of, and suggesting methods wherein we want to adapt if we’re to put ourselves at the technological forefront of the internet.

1. Recap: PWAs, SPAs, and repair staff

Progressive Web Apps are basically web sites which offer a person expertise akin to that of a local app. Features like push notifications allow simple re-engagement together with your viewers, whereas customers can add their favourite websites to their house display screen with out the complication of app shops. PWAs can proceed to operate offline or on low-quality networks, and so they enable a top-level, full-screen expertise on cell gadgets which is nearer to that supplied by native iOS and Android apps.

Best of all, PWAs do that whereas retaining – and even enhancing – the essentially open and accessible nature of the internet. As urged by the identify they’re progressive and responsive, designed to operate for each person no matter their selection of browser or machine. They may also be saved up-to-date robotically and — as we will see — are discoverable and linkable like conventional web sites. Finally, it’s not all or nothing: current web sites can deploy a restricted subset of those applied sciences (utilizing a easy service employee) and begin reaping the advantages instantly.

The spec remains to be pretty younger, and naturally, there are areas which want work, however that doesn’t cease them from being one in all the largest developments in the capabilities of the internet in a decade. Adoption of PWAs is rising quickly, and organizations are discovering the myriad of real-world enterprise targets they’ll impression.

You can learn extra about the options and necessities of PWAs over on Google Developers, however two of the key applied sciences which make PWAs potential are:

  • App Shell Architecture: Commonly achieved utilizing a JavaScript framework like React or Angular, this refers to a approach of constructing single web page apps (SPAs) which separates logic from the precise content material. Think of the app shell as the minimal HTML, CSS, and JS your app wants to operate; a skeleton of your UI which could be cached.
  • Service Workers: A particular script that your browser runs in the background, separate out of your web page. It basically acts as a proxy, intercepting and dealing with community requests out of your web page programmatically.

Note that these applied sciences are usually not mutually unique; the single web page app mannequin (introduced to maturity with AngularJS in 2010) clearly predates service staff and PWAs by a while. As we will see, it’s additionally totally potential to create a PWA which isn’t constructed as a single web page app. For the functions of this text, nonetheless, we’re going to be specializing in the ‘typical’ strategy to growing trendy PWAs, exploring the SEO implications — and alternatives — confronted by groups that select to be part of the rapidly-growing variety of organizations that make use of the two applied sciences described above.

We’ll begin with the app shell structure and the rendering implications of the single web page app mannequin.

2. The app shell structure


In a nutshell, the app shell structure entails aggressively caching static belongings (the naked minimal of UI and performance) after which loading the precise content material dynamically, utilizing JavaScript. Most trendy JavaScript SPA frameworks encourage one thing resembling this strategy, and the separation of logic and content material on this approach advantages each pace and usefulness. Interactions really feel instantaneous, very similar to these on a local app, and knowledge utilization could be extremely economical.

Credit to

As I alluded to in the introduction, a heavy reliance on client-side JavaScript is an issue for SEO. Historically, many of those points centered round the proven fact that whereas search crawlers require distinctive URLs to uncover and index content material, single web page apps don’t want to change the URL for every state of the utility or web site (therefore the phrase ‘single page’). The reliance on fragment identifiers — which aren’t despatched as a part of an HTTP request — to dynamically manipulate content material with out reloading the web page was a serious headache for SEO. Legacy options concerned changing the hash with a so-called hashbang (#!) and the _escaped_fragment_ parameter, a hack which has long-since been deprecated and which we received’t be exploring at this time.

Thanks to the HTML5 historical past API and pushState methodology, we now have a greater resolution. The browser’s URL bar could be modified utilizing JavaScript with out reloading the web page, thereby retaining it in sync with the state of your utility or web site and permitting the person to make efficient use of the browser’s ‘back’ button. While this resolution isn’t a magic bullet — your server have to be configured to reply to requests for these deep URLs by loading the app in its right preliminary state — it does present us with the instruments to remedy the drawback of URLs in SPAs.

// Run this in your console to modify the URL in your 
// browser - word that the web page would not really reload. 
historical past.pushState(null, "Page 2", "/page2.html");

The larger drawback dealing with SEO at this time is definitely a lot simpler to perceive: rendering content material, particularly when and how it will get completed.

Rendering content material

Note that after I refer to rendering right here, I’m referring to the technique of setting up the HTML. We’re specializing in how the precise content material will get to the browser, not the technique of drawing pixels to the display screen.

In the early days of the internet, issues have been easier on this entrance. The server would usually return all the HTML that was needed to render a web page. Nowadays, nonetheless, many websites which make the most of a single web page app framework ship solely minimal HTML from the server and delegate the heavy lifting to the consumer (be person or a bot). Given the scale of the internet this requires rather a lot of time and computational useful resource, and as Google made clear at its I/O convention in 2018, this poses a serious drawback for search engines like google and yahoo:

“The rendering of JavaScript-powered websites in Google Search is deferred until Googlebot has resources available to process that content.”

On bigger websites, this second wave of indexation can generally be delayed for a number of days. On high of this, you’re possible to encounter a myriad of issues with essential data like canonical tags and metadata being missed fully. I’d extremely advocate watching the video of Google’s wonderful discuss on this topic for a rundown of a few of the challenges confronted by trendy search crawlers.

Google is one in all the only a few search engines like google and yahoo that renders JavaScript in any respect. What’s extra, it does so utilizing an internet rendering service that till very just lately was primarily based on Chrome 41 (launched in 2015). Obviously, this has implications outdoors of simply single web page apps, and the wider topic of JavaScript SEO is an enchanting space proper now. Rachel Costello’s current white paper on JavaScript SEO is the greatest useful resource I’ve learn on the topic, and it contains contributions from different consultants like Bartosz Góralewicz, Alexis Sanders, Addy Osmani, and a fantastic many extra.

For the functions of this text, the key takeaway right here is that in 2019 you can’t depend on search engines like google and yahoo to precisely crawl and render your JavaScript-dependent internet app. If your content material is rendered client-side, it is going to be resource-intensive for Google to crawl, and your web site will underperform in search. No matter what you’ve heard to the opposite, if natural search is a priceless channel in your web site, you want to make provisions for server-side rendering.

But server-side rendering is an idea which is regularly misunderstood…

“Implement server-side rendering”

This is a standard SEO audit suggestion which I typically hear thrown round as if it have been a self-contained, easily-actioned resolution. At greatest it’s an oversimplification of an infinite technical endeavor, and at worst it’s a misunderstanding of what’s potential/needed/useful for the web site in query. Server-side rendering is an final result of many potential setups and could be achieved in many alternative methods; finally, although, we’re involved with getting our server to return static HTML.

So, what are our choices? Let’s break down the idea of server-side rendered content material slightly and discover our choices. These are the high-level approaches which Google outlined at the aforementioned I/O convention:

  • Dynamic Rendering — Here, regular browsers get the ‘standard’ internet app which requires client-side rendering whereas bots (akin to Googlebot and social media providers) are served with static snapshots. This entails including a further step onto your server infrastructure, particularly a service which fetches your internet app, renders the content material, then returns that static HTML to bots primarily based on their person agent (i.e. UA sniffing). Historically this was completed with a service like PhantomJS (now deprecated and now not developed), whereas at this time Puppeteer (headless Chrome) can carry out an analogous operate. The primary benefit is that it will probably typically be bolted into your current infrastructure.

  • Hybrid Rendering — This is Google’s long-term suggestion, and it’s completely the approach to go for newer web site builds. In quick, everybody — bots and people — get the preliminary view served as fully-rendered static HTML. Crawlers can proceed to request URLs on this approach and can get static content material every time, whereas on regular browsers, JavaScript takes over after the preliminary web page load. This is a good resolution in concept, and comes with many different benefits for pace and usefulness too; extra on that quickly.

The latter is cleaner, doesn’t contain UA sniffing, and is Google’s long-term suggestion. It’s additionally value clarifying that ‘hybrid rendering’ shouldn’t be a single resolution — it’s an final result of many potential approaches to making static prerendered content material out there server-side. Let’s break down how a few methods such an final result could be achieved.

Isomorphic/common apps

This is a technique wherein you would possibly obtain a ‘hybrid rendering’ setup. Isomorphic purposes use JavaScript which runs on each the server and the consumer. This is made potential thanks to the creation of Node.js, which – amongst many different issues – permits builders to write code which might run on the backend in addition to in the browser.

Typically you’ll configure your framework (React, Angular Universal, no matter) to run on a Node server, prerendering some or all of the HTML earlier than it’s despatched to the consumer. Your server should, subsequently, be configured to reply to deep URLs by rendering HTML for the acceptable web page. In regular browsers, that is the level at which the client-side utility will seamlessly take over. The server-rendered static HTML for the preliminary view is ‘rehydrated’ (sensible time period) by the browser, turning it again right into a single web page app and executing subsequent navigation occasions with JavaScript.

Done effectively, this setup could be improbable because it provides the usability advantages of client-side rendering, the SEO benefits of server-side rendering, and a speedy first paint (even when Time to Interactive is commonly negatively impacted by the rehydration as JS kicks in). For worry of oversimplifying the activity, I received’t go into an excessive amount of extra element right here, however the key level is that whereas isomorphic JavaScript / true server-side rendering generally is a highly effective resolution, it’s typically enormously complicated to arrange.

So, what different choices are there? If you possibly can’t justify the time or expense of a full isomorphic setup, or if it is merely overkill for what you’re attempting to obtain, are there some other methods you possibly can reap the advantages of the single web page app mannequin — and hybrid rendering setup — with out sabotaging your SEO?


Having rendered content material out there server-side doesn’t essentially imply that the rendering course of itself wants to occur on the server. All we want is for rendered HTML to be there, prepared to serve to the consumer; the rendering course of itself can occur anyplace you want. With a JAMstack strategy, rendering of your content material into HTML occurs as a part of your construct course of.

I’ve written about the JAMstack strategy earlier than. By approach of a fast primer, the time period stands for JavaScript, APIs, and markup, and it describes a approach of constructing complicated web sites with out server-side software program. The technique of assembling a web site from front-end element components — a activity a standard web site would possibly obtain with WordPress and PHP — is executed as a part of the construct course of, whereas interactivity is dealt with client-side utilizing JavaScript and APIs.

Think of it this fashion: all the things lives in your Git repository. Your content material is saved as plain textual content markdown recordsdata (editable through a headless CMS or different API-based resolution) and your web page templates and meeting logic are written in Go, JavaScript, Ruby, or no matter language your most well-liked web site generator occurs to use. Your web site could be constructed into static HTML on any pc with the acceptable set of command line instruments earlier than it’s hosted anyplace. The ensuing set of easily-cached static recordsdata can typically be securely hosted on a CDN for subsequent to nothing.

I truthfully suppose static web site turbines – or reasonably the rules and applied sciences which underpin them — are the future. There’s each probability I’m flawed about this, however the energy and suppleness of the strategy must be clear to anybody who’s used trendy npm-based automation software program like Gulp or Webpack to writer their CSS or JavaScript. I’d problem anybody to check the deep Git integration supplied by specialist webhost Netlify in a real-world undertaking and nonetheless suppose that the JAMstack strategy is a fad.

The recognition of static web site turbines on GitHub, generated utilizing

The significance of a JAMstack setup to our dialogue of single web page apps and prerendering must be pretty apparent. If our static web site generator can assemble HTML primarily based on templates written in Liquid or Handlebars, why can’t it do the identical with JavaScript?

There is a brand new breed of static web site generator which does simply this. Frequently powered by React or Vue.js, these applications enable builders to construct web sites utilizing cutting-edge JavaScript frameworks and might simply be configured to output SEO-friendly, static HTML for every web page (or ‘route’). Each of those HTML recordsdata is absolutely rendered content material, prepared for consumption by people and bots, and serves as an entry level into an entire client-side utility (i.e. a single web page app). This is an ideal execution of what Google termed “hybrid rendering”, although the exact nature of the pre-rendering course of units it fairly other than an isomorphic setup.

An excellent instance is GatsbyJS, which is in-built React and GraphQL. I received’t go into an excessive amount of element, however I’d encourage everybody who’s learn this far to take a look at their homepage and wonderful documentation. It’s a well-supported software with an inexpensive studying curve, an energetic neighborhood (a feature-packed was launched in September), an extensible plugin-based structure, wealthy integrations with many CMSs, and it permits builders to make the most of trendy frameworks like React with out sabotaging their SEO. There’s additionally Gridsome, primarily based on VueJS, and React Static which — you guessed it — makes use of React.

Nike’s current Just Do It marketing campaign, which utilized the React-powered static web site generator GatsbyJS and is hosted on Netlify.

Enterprise-level adoption of those platforms seems to be set to develop; GatsbyJS was utilized by Nike for his or her Just Do It marketing campaign, Airbnb for his or her engineering web site, and Braun have even used it to energy a serious e-commerce web site. Finally, our buddies at SEOmonitor used it to energy their new web site.

But that’s sufficient about single web page apps and JavaScript rendering for now. It’s time we explored the second of our two key applied sciences underpinning PWAs. Promise you’ll stick with me to the finish (haha, nerd joke), as a result of it’s time to discover Service Workers.

three. Service Workers

First of all, I ought to make clear that the two applied sciences we’re exploring — SPAs and repair staff — are not mutually unique. Together they underpin what we generally refer to as a Progressive Web App, sure, nevertheless it’s additionally potential to have a PWA which isn’t an SPA. You may additionally combine a service employee into a standard static web site (i.e. one with none client-side rendered content material), which is one thing I imagine we’ll see taking place much more in the close to future. Finally, service staff function in tandem with different applied sciences like the Web App Manifest, one thing that my colleague Maria just lately explored in additional element in her wonderful information to PWAs and SEO.

Ultimately, although, it’s service staff which make the most fun options of PWAs potential. They’re one in all the most important modifications to the internet platform in its historical past, and everybody whose job entails constructing, sustaining, or auditing an internet site wants to pay attention to this highly effective new set of applied sciences. If, like me, you’ve been eagerly checking Jake Archibald’s Is Service Worker Ready web page for the final couple of years and watching as adoption by browser distributors has grown, you’ll know that the time to begin constructing with service staff is now.

We’re going to discover what they’re, what they’ll do, how to implement them, and what the implications are for SEO.

What can service staff do?

A service employee is a particular type of JavaScript file which runs outdoors of the primary browser thread. It sits in-between the browser and the community, and its powers embrace:

  • Intercepting community requests and deciding what to do with them programmatically. The employee would possibly go to community as regular, or it’d rely solely on the cache. It may even fabricate a completely new response from quite a lot of sources. That contains setting up HTML.
  • Preloading recordsdata throughout service employee set up. For SPAs this generally contains the ‘app shell’ we mentioned earlier, whereas easy static web sites would possibly choose to preload all HTML, CSS, and JavaScript, making certain primary performance is maintained whereas offline.
  • Handling push notifications, related to a local app. This means web sites can get permission from customers to ship notifications, then depend on the service employee to obtain messages and execute them even when the browser is closed.
  • Executing background sync, deferring community operations till connectivity has improved. This could be an ‘outbox’ for a webmail service or a photograph add facility. No extra “request failed, please try again later” – the service employee will deal with it for you at an acceptable time.

The advantages of those sorts of options transcend the apparent usability perks. As effectively as driving adoption of HTTPS throughout the internet (all the main browsers will solely register service staff on the safe protocol), service staff are transformative when it comes to pace and efficiency. They underpin new approaches and concepts like Google’s PRPL Pattern, since we are able to maximize caching effectivity and decrease reliance on the community. In this fashion, service staff will play a key position in making the internet quick and accessible for the subsequent billion internet customers.

So yeah, they’re an absolute powerhouse.

Implementing a service employee

Rather than doing a nasty job of writing a primary tutorial right here, I’m as an alternative going to hyperlink to some key assets. After all, you’re in the greatest place to know the way deep your understanding of service staff wants to be.

The MDN Docs are a very good place to be taught extra about service staff and their capabilities. If you’re already assured with the necessities of internet growth and luxuriate in a learn-by-doing strategy, I’d extremely advocate finishing Google’s PWA coaching course. It features a complete sensible train on service staff, which is an effective way to familiarize your self with the fundamentals. If ES6 and guarantees aren’t but part of your JavaScript repertoire, put together for a baptism of fireplace.

They key factor to perceive — and which you’ll notice in a short time when you begin experimenting — is that service staff hand over an unbelievable stage of management to builders. Unlike earlier makes an attempt to remedy the connectivity conundrum (akin to the ill-fated AppCache), service staff don’t implement any particular patterns in your work; they’re a set of instruments for you to write your personal options to the issues you’re dealing with.

One consequence of that is that they are often very complicated. Registering and putting in a service employee shouldn’t be a easy train, and any makes an attempt to cobble one collectively by copy-pasting from StackExchange are doomed to failure (severely, don’t do that). There’s no such factor as a ready-made service employee in your web site — for those who’re to writer an acceptable employee, you want to perceive the infrastructure, structure, and utilization patterns of your web site. Uncle Ben, ever the internet growth guru, mentioned it greatest: with nice energy comes nice duty.

One last item: you’ll in all probability be stunned what number of websites you go to are already utilizing a service employee. Head to chrome://serviceworker-internals/ in Chrome or about:debugging#staff in Firefox to see a listing.

Service staff and SEO

In phrases of SEO implications, the most related factor about service staff might be their capability to hijack requests and modify or fabricate responses utilizing the Fetch API. What you see in ‘View Source’ and even on the Network tab shouldn’t be essentially a illustration of what was returned from the server. It could be a cached response or one thing constructed by the service employee from quite a lot of completely different sources.


Here’s a sensible instance:

  • Head to the GatsbyJS homepage
  • Hit the hyperlink to the ‘Docs’ web page.
  • Right-click – View Source

No content material, proper? Just some inline scripts and types and empty HTML components — a basic client-side JavaScript app in-built React. Even for those who open the Network tab and refresh the web page, the Preview and Response tabs will inform the identical story. The precise content material solely seems in the Element inspector, as a result of the DOM is being assembled with JavaScript.

Now run a curl request for the identical URL (, or fetch the web page utilizing Screaming Frog. All the content material is there, together with correct title tags, canonicals, and all the things else you would possibly anticipate from a web page rendered server-side. This is what a crawler like Googlebot will see too.

This is as a result of the web site makes use of hybrid rendering and a service employee — put in in your browser — is dealing with subsequent navigation occasions. There isn’t any want for it to fetch the uncooked HTML for the Docs web page from the server as a result of the client-side utility is already up-and-running – thus, View Source reveals you what the service employee returned to the utility, not what the community returned. Additionally, these pages could be reloaded when you’re offline thanks to the service employee’s efficient use of the cache.

You can simply spot which responses got here from the service employee utilizing the Network tab — word the ‘from ServiceWorker’ line under.

On the Application tab, you possibly can see the service employee which is operating on the present web page together with the varied caches it has created. You can disable or bypass the employee and check any of the extra superior performance it could be utilizing. Learning how to use these instruments is an especially priceless train; I received’t go into particulars right here, however I’d advocate learning Google’s Web Fundamentals tutorial on debugging service staff.

I’ve made a aware effort to preserve code snippets to a naked minimal on this article, however grant me this one. I’ve put collectively an instance which illustrates how a easy service employee would possibly use the Fetch API to deal with requests and the diploma of management which we’re afforded:

The end result:

I hope that this (massively simplified and non-production prepared) instance illustrates a key level, particularly that we’ve extraordinarily granular management over how useful resource requests are dealt with. In the instance above we’ve opted for a easy try-cache-first, fall-back-to-network, fall-back-to-custom-page sample, however the prospects are countless. Developers are free to dictate how requests must be dealt with primarily based on hostnames, directories, file sorts, request strategies, cache freshness, and masses extra. Responses – together with total pages – could be fabricated by the service employee. Jake Archibald explores some frequent strategies and approaches in his Offline Cookbook.

The time to study the capabilities of service staff is now. The skillset required for contemporary technical SEO has a good diploma of overlap with that of an internet developer, and at this time, a deep understanding of the dev instruments in all main browsers – together with service employee debugging – must be considered a prerequisite.

four. Wrapping Up

SEOs want to adapt

Until just lately, it’s been too simple to get away with not understanding the penalties and alternatives posed by PWAs and repair staff.

These have been cutting-edge options which sat on the periphery of what was related to search advertising and marketing, and the aforementioned wariness of many SEOs in the direction of JavaScript did nothing to encourage experimentation. But PWAs are quickly on their approach to changing into a norm, and it’ll quickly be not possible to do an efficient job with out understanding the mechanics of how they operate. To keep related as a technical SEO (or SEO Engineer, to borrow one other time period from Mike King), it’s best to put your self at the forefront of those sorts of paradigm-shifting developments. The technical SEO who’s illiterate in internet growth is already an anachronism, and I imagine that additional divergence between the technical and content-driven points of search advertising and marketing isn’t any unhealthy factor. Specialize!

Upon studying growth staff is adopting a brand new JavaScript framework for a brand new web site construct, it’s not unusual for SEOs to react with a level of cynicism. I’m actually responsible of joking about builders being attracted to the newest shiny know-how or framework, and at how quickly the world of JavaScript growth appears to evolve, layer upon layer of abstraction and automation being added to what — from the outdoors — can typically appear to be a leaning tower of a growth stack. But it’s value taking the time to perceive why frameworks are chosen, when applied sciences are possible to begin being utilized in manufacturing, and how these selections will impression SEO.

Instead of criticizing 404 dealing with or inner linking of a single web page app framework, for instance, it could be much better to give you the option to supply significant suggestions that are grounded in an understanding of how they really work. As Jono Alderson noticed in his discuss on the Democratization of SEO, contributions to open supply tasks are extra priceless in spreading appreciation and consciousness of SEO than repeatedly fixing the identical issues on an ad-hoc foundation.

Beyond SEO

One last item I’d like to point out: PWAs are such a transformative set of applied sciences that they clearly have penalties which attain far past simply SEO. Other areas of digital advertising and marketing are instantly impacted too, and from my standpoint, one in all the most fascinating is analytics.

If your web site is partially or absolutely purposeful whereas offline, have you ever tailored your analytics setup to account for this? If push notification subscriptions are a KPI in your web site, are you monitoring this as a aim? Remembering that service staff wouldn’t have entry to the Window object, monitoring these occasions shouldn’t be potential with ‘normal’ monitoring code. Instead, it’s needed to configure your service employee to construct hits utilizing the Measurement Protocol, queue them if needed, and ship them instantly to the Google Analytics servers.

This is an enchanting space that I’ve been exploring rather a lot these days, and you may learn the first publish in my collection of articles on PWA analytics over on the Builtvisible weblog.

That’s all from me for now! Thanks for studying. If you may have any questions or feedback, please depart a message under or drop me a line on Twitter @tomcbennet.

Many thanks to Oliver Mason and Will Nye for his or her suggestions on an early draft of this text.

Source hyperlink Internet Marketing

Be the first to comment

Leave a Reply

Your email address will not be published.