Maybe you’re reading this guide because you’ve sharpened your SEO skills, but you’ve never had the know-how to launch & measure an SEO test.
Or, maybe you’re here because you really like to hoard guides and you want to save this to your vast collection of SEO resources and TikTok shorts (hey, I’m not hatin’).
Either way, it’s my view that SEO testing is a fundamental part of everything we do as marketers. So much so that most of the activities we implement and measure are (at a core level) SEO experiments.
What is SEO testing?
SEO testing is simply the process of making material changes to a website and then measuring the impact of those changes to organic search performance. SEO tests can take the form of advanced A/B split testing, or it can take the form of simple time-based single-URL tests.
All that’s really required to start running an SEO test is the ability to make an optimization and then measure the result.
Caution: A common myth in SEO testing is the idea that a test can only be valid if statistical significance is present in the test report. Much as we love statistical significance, this goal is often unattainable in most SEO environments which is why I encourage marketers to get comfortable with directionally-informative results in the absence of statistical significance.
Why do we test?
In a word, uncertainty.
As marketers, if we knew the future outcomes of every project, then our strategies would be clear enough to only focus our 100% of efforts on success guarantees.
Crystal ball, anyone?
For the rest of us down here on earth, SEO testing is how we figure out whether or not our ideas are actually capable of driving positive marketing results. We have to try things, and we have to measure those things after we’ve tried them.
Then, by making those attempts and scaling up our testing efforts, we can reap bigger rewards and we can make smarter marketing decisions in the future based on our enhanced knowledge of past SEO learnings.
Additionally, SEO testing helps you to develop a clearer understanding of what Google’s algorithms are & aren’t rewarding, which makes you a better SEO professional.
SEO test measurement techniques
SEO testing techniques fall into two overarching measurement techniques: time-based measurement and split test measurement (a.k.a. SEO A/B testing).
Time-based measuring is a method that’s suitable for beginners, accessible in all environments, and it’s the go-to for most SEOs when it comes time to measure the outcomes of their work.
Split testing (A.k.a. SEO A/B testing) is a more advanced technique that can create statistically significant results. The main drawback to split testing for SEO is that it can only be executed in environments that have programmatic page groups that have substantial organic traffic.
To better understand the differences between time-based measuring and split test measuring, I’ve created the following table.
For further reading, check out my guide to SEO testing techniques.
What variables can you experiment on through SEO testing?
Short answer. Almost anything SEO-related can qualify for SEO testing.
The long answer gets more complicated as it all comes back to what your goals are as a business, or as an SEO professional.
If you’re testing to reach measurable growth in traffic, conversions, or other KPIs, then the question is not, “what can you test?” It’s, “what should you be testing?”
Below are some of my personal favorite SEO tests that I rely on when I’m aiming to grow organic traffic to a web page or group of web pages (in no particular order).
Some of these are explained in more detail in my Moz article, Beyond Title Tags: 5 Worthwhile SEO Tests that Seem “Untestworthy”.
How to set up, launch, and measure your first SEO test
Your first SEO test should be set up and measured as a time-based single-URL experiment. After all, it’s the easiest and most accessible way to test. I recommend doing a single title tag experiment for starters, and I’ve created two more-detailed sets of instructions which you can find linked here below:
The In-Depth Guide to SEO Title Testing
The Simplified Guide to SEO Title Testing
Step one: Identify a testing opportunity on your website
Look for a page title on your site that is either under-optimized, or that is not aligned on search intent with the page’s primary keyword(s). I like to find these opportunities in Google Search Console by looking for large gaps in available clicks (as measured by the difference between clicks and impressions).
A larger available clicks gap usually represents a stronger opportunity to grow traffic with an SEO test, especially if your primary keyword is ranking somewhere between positions 4-15.
Step two: Analyze the SERPs for your primary keyword
Once you’ve got your opportunity in-hand, do a Google search on the primary keyword to analyze the SERPs. Ask yourself some of the following questions:
- What are the predominating search intents for this keyword?
- Is my title aligned with satisfying what users want to see?
- What patterns do I see amongst the top-ranking page titles?
- Can I write a better title? Perhaps one that displays unique value?
- What would make my title more clickable, direct, and succinct?
Step three: Build a clear hypothesis
The formula for a great title testing hypothesis looks like this:
Writing your hypothesis helps you understand exactly what you’re hoping to measure. When it’s time to analyze the results at the end of your experiment, the hypothesis will make it easier to see what went positively or negatively with your new test variant.
Step four: Draft your test variant & backup a copy of the control
If you’re testing on-page content, or an element like a title tag, draft the new version of your test variable according to the hypothesis you created in step three.
Important: remember to backup a copy of your control. It’s surprisingly easy to run an experiment without backing up the control. If your test ends in traffic declines, you’ll need the control backup to revert your changes and recover lost traffic.
Step five: Publish & re-index the URL to launch your experiment
Now, go ahead and publish your test variant to set the changes live on your test URL.
Once your changes are live, head back over to your GSC account to inspect the URL and perform a re-index request (Instructions here).
Step six: Benchmark the current performance data
It’s easy to forget when you launched your experiment, what your rankings looked like, and other details about your test that you’ll need in the future. Benchmarking these pieces of information ensures that you’ll have the right information when it’s time to measure and analyze your SEO experiment.
- URL submission date: always benchmark the date that you submitted your URL to Google for reindexation.
- Traffic to the URL: If you’re planning to measure your test based only on traffic (clicks), then you may not need to benchmark the traffic because you’ll be able to access it in Google Search Console and/or Google Analytics.
- Primary keyword rankings: Any time you intend to analyze your experiment based on the impact to keyword rankings, I strongly recommend benchmarking the ranking(s) of your page’s primary keyword(s). Although Google Search Console includes position data, it’s usually more difficult to analyze ranking changes in Google Search Console than it is with benchmarked data, or keyword rank tracking software.
Step seven: Wait at least 2-3 weeks
Most SEO tests will need to run untouched for at least two weeks to generate enough data for directional accuracy. Some tests can even take 3 weeks, 4 weeks, or even longer before the results can be deemed informative.
These simple time-based experiments do not have statistical significance thresholds, which means that you’ll need to make your own judgment call about how long to let your experiment run. If your results look directionally informative after 2 weeks, then go ahead and wrap it up with a 2-week period.
If it looks too difficult to learn from at the 2-week mark, give your test more time.
Step eight: Analyze your results
Last, analyzing the results of your experiment.
In most time-based experiments, I recommend analyzing total GSC clicks to the test URL as your primary KPI. Rankings can be equally-valuable to measure, which is why your benchmarks come in handy.
Checking rankings is as simple as re-checking the primary keyword rankings and comparing them to the page’s rankings before your experiment.
Checking your Google Search Console clicks can be done by using the date range comparison in GSC’s performance report.
Pro tip: try to align the same days of the week in your before and after date ranges. Weekend traffic patterns usually differ from weekday traffic patterns, so aligning both data sets with the same days of the week is a good way to normalize your comparison.
I’ve broken down more detailed instructions on analyzing your GSC clicks before and after in my in-depth guide to SEO title testing.
Conclusion
SEO testing & experimentation isn’t only about statistical significance and controlled scientific methods. Sometimes, testing is just a matter of making a change, waiting to see what happens, and trying to figure out what insights we can learn from our actions.
Having a “beginner’s mind” and a thirst for testing out ideas to see what sticks is really the root of all experimental learning, whether we’re talking about SEO, email marketing, or any other trial-and-error activity.
If the idea of SEO testing sounds out of reach, don’t overthink it! Chances are, you’re probably already experimenting with your SEO theories and you don’t even know it.
So don’t let the complexity of advanced techniques hinder your experimental mind. Just give your ideas a try and see if they work!
