SEO News & Updates

Preparing for Google’s Mobile-Only Index

Patrick Hathaway
Last Updated: Feb. 15, 2024

New to learning SEO?

Well, Google has been pushing SEOs and website owners to adopt mobile-first design for at least half a decade, from their mobile-friendly tag and ‘Mobilegeddon’ back in 2015, to their more recent move to mobile-first indexing.

After experimenting with mobile-first indexing over the last few years, Google have settled on March 2021 as the time when they will use mobile-first indexing for ALL sites (this was originally September 2020, but they pushed it back in July).

‘Mobile-first’ actually means ‘mobile-only’

Technically speaking, ‘mobile-first indexing’ simply means that Google will only use mobile Googlebot to crawl and index your website.

Their current guidance states that they perform content-parity tests, and only move sites over to mobile-first when they ‘recognize that they’re ready.’

From March 2021, they will do this whether your site is ready or not. This means that if you show some content on desktop that you do not show on mobile, it will become effectively invisible to Google for indexing purposes.

Whilst their intentions have been clear for some time, the industry has taken renewed notice following some statements John Mueller made at PubCon recently.

Image source.

To the point where John Mueller himself took to Twitter to try and clarify the meaning:

https://twitter.com/JohnMu/status/1316448858508062725

I think this would actually be a good idea. While ‘mobile-first’ implies that they may use desktop as a fallback, ‘mobile-only’ makes it much clearer that content will be indexed if, and only if, it is present on mobile.

Why is mobile-only indexing significant?

To clarify, this is not new information, although it may seem that way to anyone who has yet to consider the implications. Many sites have already been moved across to mobile-first indexing, so nothing will change for them. It’s the sites that haven’t that we need to worry about…

If your site has NOT yet moved to mobile-first indexing, then it might be because Google deems it is not ready. And if it is still not ready in March 2021, you might be in for a shock when you review your SEO KPIs.

Content that is not available to Mobile Googlebot will no longer be indexed. 

Let’s consider an example – you run a site about birds of prey, and have a page on falcons. You have a block of content about a specific type of falcon (e.g. a Hobby), which for some reason does not get displayed at all on mobile.

From March 2021, all of your Hobby related content will no longer be indexable, which will mean you will lose any rankings for Hobby related keywords. Furthermore, Google’s understanding of the page as a whole will be diminished, so it may have a knock-on effect for other keywords too. (If you’re wondering about what type of SEO software to use to monitor your target keywords, you might find this Semrush vs. Ahrefs article and our Semrush vs. Similarweb review helpful.)

This is quite a stark example, which serves to illustrate the point, but misses the nuance.

Mobile is the new source of truth

In essence, with mobile-first, Google has been moving from your desktop site as the ‘source of truth’ and instead making indexing/ranking decisions based on your mobile site.

The core consideration is that mobile and desktop content should be the same (and both should follow Google’s E-E-A-T and YMYL guidelines as well). In particular, the content that can have an impact upon crawling, indexing and ranking:

  • Internal links
  • On page content
  • Titles and descriptions
  • Robots.txt
  • Page resources
  • Meta robots
  • Canonicals
  • Hreflang
  • Structured data

In short, this is both ‘content for the user’ and ‘signals for the search engines.’ For example, if you use link-building tools for outreach, the recipient of your backlink request might view the page on your site on mobile. So, naturally, it should load fast and look correct. We’ll dig more into why each is important further on.

What do we even mean by ‘mobile site’?

There’s essentially 3 different ways that you can have a mobile site:

  • Responsive design (most common) – you serve a single page, which responsively adapts the design to fit the device size. The same code is used, whether it is mobile or not.
  • Dynamic serving – the server checks the User Agent, and depending on the value (mobile, tablet or desktop), sends out different HTML, CSS and JS code, on the same URL.
  • Separate mobile site – the ‘old’ way to do it, you maintain a separate mobile website, often on an ‘m-dot’ subdomain (e.g. m.example.com).

In terms of maintaining content parity between desktop and mobile, this is typically easier with responsive design, as you are only serving one consistent ‘version’ of the page code.

You also need to pay attention to how Google renders the page content. At Google I/O 2018 they shared details on the two-phase indexing process they use:

  1. The first wave of indexing is carried out using the source HTML.
  2. The page is sent off for rendering.
  3. The second wave of indexing is carried out using the rendered HTML.

So the index is updated based on the rendered HTML. Now, in a mobile-only world, we can add some more details to this process:

  1. The first wave of indexing is carried out using the source HTML, as requested by the Mobile Googlebot user-agent.
  2. The page is sent off for rendering, using the Mobile Googlebot user-agent.
  3. The second wave of indexing is carried out using the rendered HTML.

Depending on what kind of a mobile site you have (responsive, dynamic or m-dot), this will affect you to varying degrees. If the underlying code you serve to mobile User Agents is fundamentally different to the desktop, this might undermine how Google understands your website.

This is one of the reasons why Google has been pushing responsive design as the preferred solution, as the underlying HTML is the same, even if CSS changes the layout. This does not mean that responsive sites can just sleep on this, as JavaScript can fire during rendering which changes core elements of the content or metadata – which would then kick-in during the second wave of indexing.

All this to say, if you’re ranking for some terms with high keyword difficulty and Google traffic is important to you, it is worth paying attention to the mobile-only changes – whatever kind of mobile site you have.

Ensuring content parity between mobile and desktop

In terms of mobile-only, the phrase ‘content parity’ does not necessarily mean that ALL the content must be identical on both desktop and mobile – just the bits that are important for site visitors and search engines.

It boils down to two core recommendations:

  1. If you have content that you want the user to see, make sure it is visible on the (mobile) page.
  2. If you have HTML that you want search engines to see, make sure it is present in the (mobile rendered) HTML. 

We’ll run through the specific factors themselves, and clarify why parity is important for each (although some of them are hopefully obvious!).

Internal links

Internal links are probably the biggest banana skin when it comes to mobile-only, as they are one of the biggest levers when it comes to on-site SEO. With mobile-only, Google will build the internal link graph based only on the links found in the mobile version.

It is not uncommon to see a mega menu on desktop pared down to a much simpler user interface on mobile. If your mobile implementation fundamentally changes the links present in navigation elements, this could meaningfully affect the distribution of link equity in your site.

This could mean that sets of pages which were receiving lots of link equity will now receive very little, which could have a profoundly negative impact upon rankings for those pages.

On page content

By this we mean on page textual content, as well as any images and videos, along with semantic markup like header tags. On page content forms the basis of Google’s understanding of what your content actually means and represents, so it is extremely important.

It is worth noting that Google has clarified that mobile UX instruments such as accordions or burger menus will not be downgraded, even though they require the user to ‘open’ tabs to see the content. 

Per John Mueller at BrightonSEO, “On mobile it’s a lot harder to have everything included on one page without using things like tabs or dividers that expand… those kind of things. So specifically for mobile we want to make it so that you can use these elements and that you don’t have any disadvantage for using them.”

Titles and descriptions

As we know, both page titles and meta descriptions can have a huge impact upon click-through-rate (CTR), and the page title is considered a relevance factor in terms of ranking.

If you have spent time and effort optimizing titles and descriptions, you will of course want them to be the ones used by search engines. 

Page resources

As we have already mentioned, rendering is an important part of determining how search engines understand your content. This is where robots.txt comes into play, because you can accidentally block search engines from accessing page resources, which they need in order to render the page.

For example, these disallow rules would block Googlebot from accessing CSS or image files on our website:

Additionally, ensure that accessibility features, such as alt text on images, do not suddenly disappear on mobile, as “Google’s mobile-first team has noticed many sites don’t include alt text for images on mobile sites.”

Meta robots 

Possibly the most important signal of them all! If your mobile version of the page has a noindex, when one was not present on desktop, you could be in for a big shock!

The majority of web pages you will encounter do not even include meta robots, since the default assumption by search engines is “index,follow”. So if it is present, AND has a different value on mobile, you will want to correct this.

Canonicals

Similar to meta robots, canonical tags impact the core kpi of ranking – whether a page is even indexed in the first place. Although canonicals are not as strong of a signal as robots directives, since they could impact indexing, they should definitely be taken seriously.

Hreflang

As your hreflang tags allow you to specify translated content for different regions and languages, it can have a huge impact upon indexing and ranking in different regional SERPs. As with other indexing signals, you want your mobile version to include all the right hreflang.

Hreflang is a particular case where a separate m-dot site appears to be a demonstrably worse implementation. From Pubcon 2020;

“Mueller said that for some cases, particularly m-dot sites that use hreflang attributes, that Google may not be able to send desktop users from the search engine results pages (SERPs) to the desktop version. In those cases, Google will end up sending desktop users to the m-dot mobile version.”

This is likely due to the conflict of ‘mobile-only’ with previous ‘mobile-first’ hreflang advice, which has no mechanism for including desktop URLs into the hreflang cluster:

Structured Data

Your structured data markup is your opportunity to explicitly tell Google what your content represents, without ambiguity. Since some structured data markup can enable rich results like review stars or FAQ snippets in the SERPs, it can have a meaningful impact upon click-through-rate. 

Presuming the page content is the same as on desktop, there is no reason why you should make the structured data any different (since it describes the same page content).

Methods for checking content parity

Practically, there are a number of things you can do to check differences between the desktop version of your site and the mobile one. Essentially, you want to be able to ‘check off’ all of the different factors above, to ensure parity between the two versions.

Conduct a desktop-mobile parity audit

What you need to do here is perform a very specific type of technical SEO audit. Instead of looking for problems or opportunities (like you might look for during a backlink audit), you are simply looking for differences – and hoping not to find any!

Essentially, all you need to do is fire up your favourite crawling tool, crawl the site once with desktop Googlebot, and once again with mobile Googlebot, then compare the results.

For your first audit, you’ll want to crawl with regular/desktop Googlebot, and then for your second audit, switch over to Googlebot Smartphone. All crawlers allow you to easily adjust the User Agent:


Once your second audit is complete, you should be able to compare a lot of the main metrics of interest. 

In general, you are looking for big differences in things like the total number of:

  • Internal URLs
  • Internal indexable URLs
  • Internal links (particularly in-content vs navigation)
  • Status codes (do 404s or redirects increase?)
  • Page resources (CSS, JavaScript, images, videos etc…)
  • Duplicate content
  • Errors and warnings (do any of these go up/down?)
  • Structured data found (and any errors/warnings)
  • Hreflang found (and any errors/warnings)

Theoretically, all these values should be identical from your desktop crawl to your mobile one, if the content is supposed to be equivalent. Of course you are looking for the examples where they are not.

Practically, this process can be as easy as examining the ‘change’ trend from one crawl to the next:


Working through the audit, you can check through the various reports and highlighted errors/hints. You might even find things have changed in a good way:


Or in ways you did not expect:

It is important to remember that small fluctuations are normal from one crawl to the next. The numbers can get slightly thrown off if a single page is temporarily inaccessible during the crawl (e.g. transient server error) or if a few page resource files timed out. So although identical is preferred, it is not a practical expectation, and it is good enough if the numbers are pretty much the same.

It is also helpful to explore and compare the various graphs and charts which crawlers produce, as they can very quickly show stark differences if they are present. For example, a page depth chart, which indicates the site structure:


If this changes dramatically, this likely reflects a fundamental change to how the site hangs together.


You can get a similar impression if the Crawl Map dramatically changes, as this reflects the core architecture of the site:


Fundamentally, you should just not see dramatic differences between the Crawl Map for the desktop site and the one for the mobile site. If you do, it is almost definitely worth exploring further.


Beyond the wider trends, you will also want to check that the data itself is the same. So, not just that a page title was found, but that the page title is the same on each page for desktop on mobile.

Once both audits are complete, perform an export of the on-page data from each audit, and add them both to a spreadsheet, in different worksheets.

Then, for each value you wish to compare, add 2 new columns, 1 to perform a simple VLOOKUP, and the other to compare the two values.


If you are unfamiliar with how to write these formulas, check out this article (which makes for an excellent bookmark).

With a relatively small amount of data work in spreadsheets, you could use this methodology to tick off anything where you are comparing a single ‘cell’ value, i.e.

  • Page titles
  • H1 tags
  • Meta descriptions
  • Meta robots
  • Canonicals
  • Word counts
  • Alt attributes on images

One page-level metric to pay particular attention to is the internal ‘link equity’ score. At Sitebulb we call this ‘URL Rank’, but most other crawlers have different names for it (e.g. DeepCrawl have ‘DeepRank’ and Screaming Frog have a ‘Link Score’). Whatever it is called, this value reflects site-wide internal linking practices and their impact upon each URL. Again, you want these values to be pretty much the same on both desktop and mobile.

Semrush gives you access to impressive on-page tools that can help with these types of evaluations. You can read more about essential SEO software in Nick Eubanks’ Semrush review.)

Conduct a rendering parity audit

That’s half the job done! Assuming you are convinced that the content is not really changing from one User Agent to another, you now need to take into account the impact of rendering.

In terms of process, this is much the same as the above. You need to set up a 3rd audit, and this time keep Googlebot Mobile as the User Agent, but switch on JavaScript rendering.


Keep the other settings the same, and set the audit running. Once complete, you’ll need to perform all the same comparisons as you did for the desktop-mobile parity audit. Again, you are looking for the big differences in data/trends, and point differences between the ‘single cell’ data.

One important factor to remember is that, when rendering, the crawler will almost always find more page resource files, as scripts are fired. This is normal and should not be considered a ‘big difference’.

Explore the differences between audits

The point of auditing the entire site (multiple times) is to scale up your investigation, and allow you to evaluate all the pages on the site. 

If you do find some big differences, you really want to drill down into them and try to isolate what is causing the change.

  • Does the change occur only during rendering? (which indicates that JavaScript is modifying the content)
  • Can you identify specific page templates that exhibit the change (e.g. only on product pages)?
  • Can you identify where in the HTML the change is occurring (e.g. hreflang/canonical in the <head>, or structured data in the <body>)?
  • If the difference relates to internal links, can you spot any sitewide/navigational differences in terms of linking structure?

Identifying these sort of patterns can help isolate the underlying issue, particularly if you need the help of a developer at this point. Going in with a clear understanding of the symptoms will make it much easier to find the cause (e.g. ‘on mobile, the sidebar navigation element is missing on the subcategory page template’).

If you can’t identify any large differences at this point, you most likely have nothing to worry about, particularly if you are dealing with a responsive site where the technology promotes content parity by default. However it is still worth confirming via Google’s tools below, to be completely thorough.

Mobile-Friendly Test

While mobile-friendliness is independent of mobile-first indexing, it is undoubtedly important for performance in the mobile SERPs.

The Mobile-Friendly Test will also allow you to see how the page renders to Google’s smartphone User Agent, which might itself be…illuminating.


It will also notify you if there were any problems loading the page;


And in particular, if any page resources were disallowed by robots.txt;


One thing to note here is that only testing the homepage is not really sufficient. You really want to test at least one URL per page template you identify (e.g. Home, About, Category, Subcategory, Product, Blog Post, etc…).

Google’s Rich Result Test

If you employ structured data markup on your website, you’ll want to check Google’s Rich Result Test (RRT).


You’ll want to compare the desktop/mobile results for both rich results eligibility AND the specific items ‘detected’ on the page.


Again, you’ll want to test your main page templates to ensure nothing is being missed.

Google Search Console

Google Search Console is a great place to dig around if you want to better understand how Google views your website.

If you have concerns about particular URLs, make use of the ‘URL Inspection’ tool, which includes details on indexing and mobile usability.


You can also explore the Mobile Usability report itself, to see any sitewide errors:


You can also dig into the search performance data itself. In particular, you want to search out pages where the mobile/desktop/tablet split bucks the sitewide trend, in terms of clicks/impressions.

PageSpeed Insights

We’ve all heard about Google’s upcoming algorithm updates, where they will take into account ‘page experience’ metrics. Since page experience is particularly important on mobile, performance on mobile vs desktop certainly bears consideration.

Conclusion

Before even reading this article, you probably had a pretty good idea of whether mobile-first is something you should be worrying about or not. Some sites had the notification years ago in Google Search Console that they had been moved to mobile-first indexing, and noticed no ill effects.

Most sites will probably just be fine, and could get away with ignoring the practical advice in this article.

However, consider the way that John Mueller clarified how Google have been determining when sites are ready for them to make the switch;

“We essentially need to be sure that the mobile version, when we index it with mobile is equivalent to the desktop version so that more or less we can shift these over without any problems.”

But Google’s definition of ‘more or less’ might be different to yours or mine, right? So I guess to some extent you need to figure out how comfortable you are just rolling with it and hoping everything is ok, or taking control yourself and making sure it is ok.


From Google’s perspective, whilst it is important to them to preserve the integrity of the search results, they are also trying to roll this out across every single website in the world.

Collateral damage is inevitable.

Make sure it isn’t you.

Resources