How Authority Score Resists Manipulation (Data Study)

This week Semrush updated our Authority Score. We wanted to challenge ourselves and build a metric that would be the most resistant to spam and manipulation.

We know that all domain authority metrics are inexact. Nevertheless, we tried hard to conduct a thorough study and compare our new score to similar scores on the market (Moz Domain Authority and Ahrefs Domain Rating*).

To see what we learned from this study, read on! To read more about the update, check out this post.

How We Tested It

First, we took 24,000 random domains in 10 equal groups based on their Semrush Authority Score (2,400 domains with a score 0-10, 2,400 domains with a score 11-20, etc.).

Then we checked each domain’s score on each platform (Domain Authority from Moz and Domain Rating from Ahrefs) as well as our old and new Authority Score.

Finally, we looked at how these scores correlated with organic traffic and signs of link spam. 

These were the spam signals that we checked for:

• No organic rankings on SERPs (aka 0 organic traffic)
• Unnaturally high % of dofollow domains in backlink profile
• An imbalance between links and organic traffic
• Too many referring domains with the same IP address
• Too many referring domains with the same IP network
• Presence of another domain with an identical backlink profile

While comparing our old and new scores, we found some interesting patterns:

This graph shows two things:

Area 1 (Red): Our old formula would normalize many scores at 40

Area 2 (Blue): Our old formula would overvalue a domain’s Authority Score where we now see signs of spam and decrease the score.

With our old score that didn’t check for spam, a site that built links via schemes or manipulation could see its score in the 20-40 range. 

Now, we appropriately adjust these scores so you get an accurate read on every domain’s real authority.

Comparing our new score to our competitors' scores, we found a few interesting anomalies.

In most cases, all scores are very similar. However, we also see many points where the scores differ significantly (zones A and B). 

Let’s dive deeper into each zone to see what they mean.

Zone A: Overvaluing Spammy Domains 

Since there are two comparison graphs above, there are two “zone A’s;” one for Moz and one for Ahrefs.

To define these zones, we found the domains that had both of these qualities at the same time:

• Semrush AS less than 10 AND
• Moz DA or Ahrefs DR between 15-40 

In these zones, we saw hundreds of domains where the Semrush Authority Score was significantly lower than the scores from our competitors. 

Why could that be? We believe the answer is spam.

When we notice a site with an unnatural link profile that looks duplicated or heavily involved in a link scheme, our new formula counts it against the site’s score. 

Taking a deeper look, we found spam signals on over 3/4ths of these domains that Moz and Ahrefs were overvaluing in zone A. 

That confirms our hypothesis that Semrush more often gives lower scores to sites with signs of spam. 

Specific Examples of Spam Factors Impacting Domain Scores

Want a closer look?

Here’s a selection of domains we saw in Zone A with heavy spam signals, and their scores on each platform:

While this is just a small sample, in the chart below you can see the whole picture. 

Overall Correlations with Spam Factors

For every spam signal, you can see that Semrush has the strongest negative correlation.

A strong negative correlation suggests that Semrush likely considers spam factors the most in our calculation. 

This lets us conclude that it’s harder to manipulate Semrush Authority Score with private blog networks and link schemes.

On Semrush, it’s now easy to see any site’s spam signals right in the interface when you look at Authority Score in Backlink Analytics. 

For example:

Want to see the list of all the parameters of the new Authority Score? Read Authority Score Explained.

Zone B: High Scores, Low Organic Traffic 

Another area where we saw outliers was domains with high scores, but low organic traffic. 

A high score (60-80) should describe a highly authoritative site that receives lots of traffic from Google. 

Let's take a look at all of the domains that fell within the 60-80 range for all of the providers.

Valuing Domains in the 60-80 Range

As stated above, we believe domains with a 60-80 score should be definitive authorities in their niche. To us, that means having tens of thousands of monthly visitors from organic search.

A small site might have a leading authority in its niche even with 5-10k in organic traffic, but many larger markets will have plenty of sites with traffic in that range. 

If a website has thousands of backlinks, but less than ten thousand in organic traffic, it could be a sign of the content not being engaging enough.

When we looked at the sites with scores between 60-80 on Semrush, none had less than 10k estimated monthly organic traffic.

However, 3.8% of sites rated between 60-80 by Moz and 8.12% of sites rated in this range by Ahrefs had monthly organic traffic under 10k.

Specific Examples of Overvaluing Sites with Low Traffic

Now, let’s look closer at a few more examples from Zone B where you can see Ahrefs and Moz’s overvaluing sites with little to no estimated organic traffic:

With the new Authority Score widget in the UI, you can tell exactly how much links, traffic, and spam signals contribute to a domain’s overall AS.

Overall Correlation with Organic Traffic

Although 24,000 domains are only a fraction of the web, we saw the following trend across the sample:

This data tells us that Semrush gives higher scores to domains that are prominent on the SERPs. 

Along with the negative correlations to major spam factors, we believe these findings show that Semrush Authority Score leads the way in resisting manipulation.

Checking SEO Authority with Better Data

Yes, correlations are not exhaustive evidence. We don’t have a “true source” to compare our new score to. Each provider is subjective.

But, by combining different data relevant to a site’s true SEO weight, we can say our new AS is more resistant to manipulation than the others in many cases. Thus, we believe our score is more accurate.

*Note: Majestic was not included in the comparison study because they use a different approach to calculating a domain’s authority, providing two metrics to gauge link authority (trust flow and citation flow), instead of one.