At Least 66.5% of Links to Sites in the Last 9 Years Are Dead (Ahrefs Study on Link Rot)

At Least 66.5% of Links to Sites in the Last 9 Years Are Dead (Ahrefs Study on Link Rot)

Patrick Stox
Patrick Stox is a Product Advisor, Technical SEO, & Brand Ambassador at Ahrefs. He was the lead author for the SEO chapter of the 2021 Web Almanac and a reviewer for the 2022 SEO chapter. He also co-wrote the SEO Book For Beginners by Ahrefs and was the Technical Review Editor for The Art of SEO 4th Edition. He’s an organizer for several groups including the Raleigh SEO Meetup (the most successful SEO Meetup in the US), the Beer and SEO Meetup, the Raleigh SEO Conference, runs a Technical SEO Slack group, and is a moderator for /r/TechSEO on Reddit.
Article Performance
  • Linking websites
    279

The number of websites linking to this post.

This post's estimated monthly organic search traffic.

    Link rot is when links stop pointing to their intended file, web page, or server. The links are dead. Link rot happens as pages on the internet are removed, redirected, or updated causing the links on these pages to go to broken pages or pages whose content has been altered.

    Since January 2013, 66.5% of the links pointing to the 2,062,173 websites we sampled have rotted. We found another 6.45% with temporary errors. We don’t know if they’re still there or not.

    This is even more complicated when it comes to SEO. Another 1.55% have other issues that prevent the links from being counted for the purposes of ranking.

    That means a total of 74.5% of the links in our study are considered lost, with at least 66.5% being rotted.

    Often, the links that no longer work are important. Check out this example of a website that was referenced in a U.S. Supreme Court case. Someone bought the domain and used it to make a statement.

    Image describing that a page referenced in a supreme court case has been removed

    In a previous study of legal journals and citations from 2014, 70% of the links within the journals and 50% of the URLs from U.S. Supreme Court decisions did not contain the originally cited material.

    Another study from 2012 found that 30% of social media links were dead within two years.

    Most of the previous studies are fairly small and contain older parts of the web. I assume a lot more of the older web is already gone, if not most of it. For example, most sites stopped using extensions like .html on URLs many years ago in favor of clean URLs. Most sites have also moved from HTTP to HTTPs.

    Considering the above, we decided to do the largest link rot study ever. And it’s one of the only ones that cover the more recent version of the web.

    Let’s dig into the data.

    About the data

    Ahrefs has been crawling the web since 2010. But for the purpose of this study, we’re only looking at the data from January 2013.

    You can use the Backlinks report in Ahrefs’ Site Explorer to check the data for your own site. For Ahrefs, 26.9 million out of 174.3 million links have been lost. Just compare the numbers with the “Lost” filter applied vs. the numbers with the “All” filter applied.

    Gif showing how to check for lost backlinks in Ahrefs

    There are a few cases we tag as lost that we don’t count as link rot. I’ll cover that below.

    As I mentioned in the intro, at least 66.5% of links to the sampled websites have rotted in the last nine years.

    The web is complex and messy, and some things change faster than others. I wanted to see how many sites have link rot—and what percentage of their links experience link rot. A huge thanks to our data scientist Loveme Felicilda for pulling this data! This is the distribution for the percentage of link rot by domain across the dataset.

    Histogram showing the link rot percentage that occurs by number of domains

    There are a lot of small sites that don’t have much link rot. If we take out the smallest sites and only look at those with more than 10 live links, you’ll see that larger sites seem to have quite a bit of link rot.

    Histogram showing the link rot percentage that occurs by number of domains, filtered to greater than 10 live links

    As I mentioned in the intro, the number of links we consider lost when it comes to SEO is even higher—percentage-wise, it’s 74.5%. I also wanted to see the distribution for these across the dataset.

    Histogram showing lost link percentage by domain

    There are a lot of small sites that don’t have many lost links. If we take out the smallest sites and only look at those with more than 10 live links, you’ll see that larger sites seem to have lost quite a lot of their links.

    Histogram showing lost link percentage by domain, filtered to greater than 10 live links

    Links can be lost for many reasons. We classify lost links in different ways at Ahrefs. Here are the most common reasons that links are lost:

    • Dropped (47.7%)
    • Link removed (34.2%)
    • Crawl error (6.45%)
    • 301/302 (5.99%)
    • Not found (4.11%)
    • Not canonical (0.82%)
    • Noindex (0.73%)
    • Broken redirect (0%)

    Pie chart showing the main reasons links are lost

    Let’s look at each of those and why they happen.

    47.7% of links are from dropped pages

    These pages are removed from our index for various reasons.

    Example of link dropped

    Pages may be dropped because they can’t be crawled or indexed. In some cases, a domain may not exist anymore.

    34.2% of links are removed

    In this case, the pages still exist; they just no longer link to you.

    Example of link removed

    It could be that someone removed the link during a content refresh, replaced your link with a different one, or removed the link due to company policies. Another possibility is that a competitor decided to no longer link to you.

    6.45% of lost links are from crawl errors

    When we encounter an error while trying to crawl a page, it will be put into this bucket.

    Link lost due to crawl error

    If the page is accessible when it’s crawled again and the link is still there, it will be counted as live. If the page continues to “error,” we may drop it from the index.

    We chose to not count crawl errors in the total for link rot. It’s likely that a portion of these links no longer exists, but others still do.

    5.99% of links are lost due to redirected pages

    The page containing the link has been redirected somewhere else.

    Link lost due to 301 redirect

    Pages change locations for all kinds of reasons. Commonly, this is the result of some kind of website migration.

    4.11% of links are pages that are not found

    In this case, the linking page has been deleted. The content, including the link, is missing.

    Page not found

    Occasionally, these pages may become live again or be redirected; in such situations, they will be added back or placed in the redirect bucket.

    0.82% of links are lost because the page they were on is no longer canonical

    The canonical specified by the page has changed.

    Page not canonical anymore

    The linking page has a “rel=canonical” tag to some other location. It could be a change from HTTP to HTTPs or some kind of standardization involving trailing slashes or parameters. This is usually nothing to be worried about. The page is simply changing how it wants to be indexed. These links have just shifted locations, going from one page to another.

    0.73% of links are lost because their pages are marked “noindex”

    The linking page is marked “noindex,” so we don’t count the links from it. 

    Page marked as noindex

    We did not count pages marked as noindex in the numbers for link rot. The link technically exists, but the page it’s on won’t be found in search engines and won’t pass any value.

    A small number of links are lost due to broken redirects

    In this case, we saw multiple redirects in a chain before. Now one of those redirects is broken. The link is, thus, kind of disconnected from the target.

    Redirect broken because destination changed

    This happens if:

    • The redirect chain is broken – If any of the pages in the redirect chain fails to respond, it gets reported as a lost link.
    • The redirect no longer exists (or is changed) – Let’s say you had a link from Site A → Site B, but the link was first redirected through one or more other URLs (e.g., Site A → Site C → Site B). If the linking site swapped this link out so that it linked directly (rather than going through a redirect chain), it would be reported as a lost link. The same applies if the final URL of the redirect is changed to redirect elsewhere.

    What can you do about link rot?

    A lot of the links you obtain may be lost over time. One way you can possibly get some of them back is with link reclamation.

    In many cases, your old URLs have links from other websites. If they’re not redirected to the current pages, then those links are lost and no longer count for your pages. It’s not too late to do these redirects, and you can quickly reclaim any lost value. Think of this as the fastest link building you will ever do.

    Here’s how to find those opportunities:

    I usually sort this by “Referring domains.”

    Best by links report filtered to 404 status code to show redirect opportunities

    You can even use link rot to your advantage. Broken link building is a tactic that involves finding resources in your niche that are no longer live, then reaching out to site owners and letting them know about a resource you have that can replace the broken link.

    Want to know how to do this for your site? Our head of content, Joshua Hardwick, has you covered with a process-oriented guide to broken link building.

    Another way to help with link rot is to fix broken links on your own website. These are easily identified in the Site Audit Links report. Just remove the links or update the reference to a relevant page that exists.

    Broken internal links

    You may also want to fix broken links from your site that point to other sites. I have trouble arguing for this for SEO and, generally, will deem it as a website health and maintenance task that is of pretty low priority.

    However, you can argue that clicking these links is bad for user experience. Accordingly, you can prioritize the links that are more often clicked.

    The list of broken links to external pages can also be found in the Links report. If you see zero broken external links as I do, it’s probably because you didn’t enable “Check HTTP status of external links” in your Site Audit crawl settings.

    Site Audit settings need to have "Check HTTP status of external links" turned on

    Final thoughts

    Some companies and technologies have tried to help with link rot. Many of these solutions don’t really solve the problem of broken links or a changing web. Instead, they rely on archiving what was on the web so it can still be seen. For example, the Internet Archive has a Chrome extension that will show archives of pages if they’re broken.

    Similarly, the CDN Cloudflare has an Always Online option that will first look for its own archived copy of a page that’s offline. But if that doesn’t exist, it will pull the most recent version from the Internet Archive.

    If you use Brave browser, a broken page will have a message that lets you check for an archived version at archive.org.

    The Law Library of Congress implemented an external archiving solution for the problem of link and reference rot in its legal research reports.

    As always, message me on Twitter if you have any questions.

    Article Performance
    • Linking websites
      279

    The number of websites linking to this post.

    This post's estimated monthly organic search traffic.