A 5-Step Guide to Diagnosing Technical SEO Problems

I’m going to keep the intro short, because I’m assuming you’re coming here amid technical SEO chaos, a fire, maybe even just a fire drill (that has a lot of executive attention).

Below is a process for fighting technical SEO flames. It may help your life, or it may not, chance are we don’t know each other, so why should I make assumptions about what’s useful to you.

Anyway, if you’re looking to a quick audit list, you should probably just skip to step 4. I won’t hold it against you.

You may miss out on some good random takeaways along the way, but most people don’t read anyways. Well, except you, since you’re still here and it turned out to be not that short of an intro after all…

Step 1: Panic Disengaged

Close your email. (This process works better if you actually follow the process).

OK, now take in a deep breath in 1.2.3.4.5… now breath out, 1.2.3.4.

Alright, now stop reading this blog. Maybe go grab some water, coffee, tea.

Take a walk outside (unless it’s frigid, then definitely don’t do that). This post will be here when you get back.

(Hopefully, you came back…)

Now to lessen your nerves – turn into a robot. Just kidding. But you get the point – nerves, anxiety, and the mental load – life can serve us a lot.

It can be hard to reach a place where emotions and nerves aren’t intertwined, especially when faced with stress. At the same time, it’s important to approach technical SEO problems with a clear, analytical, logical mindset.

Sometimes this “mode” comes with more experience, genetic, personality type, or horoscope.

For the rest of us we can:

• Breathe.
• Laugh about something (maybe watch this six-second video).
  • Seriously though, if you made it this far and didn’t just skip to step four, you have six seconds, you won’t regret it.
• Grab some water, coffee, tea, etc.
• Listen to your favorite song (spare a few minutes for yourself).
• Re-prioritize your day to deal with this issue (let go of the plans you had).
• Remember that it’s not expected that you know all the answers (but will try).

When you’re ready to tackle the issue, solve some problems, roll up your sleeves, and deep dive into the data – this post will be here for you (or on some other site that has pirated the content).

The point is, that you’ll find it somewhere online, so give yourself a minute.

Step 2: Understand the Problem

Before we can attempt to solve problems, we have to understand the problem. That means you’re going to have to face that panic-ridden email (and/or voice message and/or pager note) again.

Look it in its beady little email eyes and extrapolate all necessary information you (and your squad team) needs to find a solution.

Some questions to answer:

• What does the email say?
• What does the email really say? (are you sensing some underlying issue?)
• What do we know already about said issue?
• Do we have any data collected already?
• What are the symptoms surrounding said issue?
• Do we need to solve ASAP?
  • Or is this more of a “we have this flagged, going to check it out tomorrow morning” type of deal?

Einstein once said, “If I had an hour to solve a problem, I’d spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.”

And he had more experience problem-solving than me, so I’ll defer to his expertise.

Step 3: Make a Plan (…or Part of One)

You may be thinking: alright, now I get the problem, now is the time to attack the data. And… you could.

Or maybe…we could start with a making a plan.

Let me sell it to you.

If you have a plan – you have vision. You are mapping the path to your success (allowing room for detours).

This can save a ton of time in the end, when you are avoiding scope and the myriad of pitfalls that affect our productivity and ability to execute correctly the first (few) time(s).

Plus, making a plan includes making a hypothesis. And hypothesizing is important enough for the scientific process.

Note: making a plan doesn’t have to be a formal process. It can be some notes jotted in a .txt or .md file.

Basically, what you need:

• Create a hypothesis:
  • Is there anything that you can think of that might be causing this?
  • List out anything you would like to test.
  • Outline indicators that would be true (if the hypothesis is valid).
  • Outline where to start.
  • Estimate how much time this is going to take.

Step 4: Do Plan (+Analyzing)

Now it’s time to take action, activate your plan, fight the chaos.

This step involves genuinely looking at your hypothesis, not as an extension of yourself (or your smarts, or your ego), but as a genuine issue.

Taking that issue and trying to build a case for or against it.

For you, dear reader, I did some reflecting on where to start with diagnosing technical SEO issues and common ones.

Maybe others start somewhere different (tell me in the comments – let me learn the ways, sensei (bows)).

And, of course, (we must keep in mind) that some problems are better solved with certain tools.

But, instead of nitpick, let’s generalize all of our work, and make a set of arbitrary steps that may* not help at all (yay!).

Regardless, here is a general process that I use, when I’m handed a problem:

1. User-agent switcher to Googlebot and check the site.
2. Check DOM (that’s the DevTools, i.e., right click > inspect > element tab).
3. Throw URLs in Screaming Frog (or other equivalent crawlers) and see if the issue can be found.
4. Check analytics platform and Google Search Console (GSC) data.
   • Review relevant data in analytics platform (think: Google Analytics or Adobe) + GSC + Bing Webmaster Tools.
     • Side note: I wrote an article (as an SEO guppy) that has checklists for determining shifts in organic search data fluctuations. Check it out for ideas on deep diving into analytics shifts.
5. Check your keyword ranking tracker.
6. Check your log files (especially if it’s a crawling related issue).

Common Roots of Technical SEO Problems

Some more context around (if you’re interested):

Crawling Issues

• Not using a href links
  • Google recommends using “a href” links. JavaScript onclick functionality is cool and all (Google can crawl JS links (note the italics)); however, “a href” links are classic, consistent, and capable of get your site crawled.
• Infinite scroll
  • Infinite scroll can be set up in multiple ways, some of the infinite scroll content cannot be visible to search engines.
  • A way to start to diagnosing if they’re picking up: try to see if Google surfaces direct quotes from the content, check GSC reports on coverage, and review current page rankings.
• Lazy loaded images (i.e., images that load either when everything else has loaded or upon a specific user action, like scrolling)
  • If you want your images lazy loaded (i.e., so your site appears to load faster – get that CRP in check), you’re going to need to:
    • Add in <noscript> tags.
    • Use structured data (you’ll either have to use an ImageObject or an “image” item property within another, more specific markup).
• Poor crawl efficiency (+crawl traps)
  • Potential for poor crawl efficiency includes:
    • XL sites (think: +100k URLs).
    • Duplication issues (see a chart below on potential causes…, which are both technical and content-driven).
    • Using 302 temporary redirects in that place of 301s on a large scale.
    • Exceptionally slow load times (think: +20 seconds).
    • JavaScript rendering required for multiple experiences.

• Using CSS background images
  • They’re just that – background images. They aren’t crawled and won’t be indexed in image search. I know, it’s sad. But they’re supposed to be only decorative and purposeless.
• Inaccurate status codes
  • Nothing is worse than mixed messages. Are we 200? Is this a 404 relationship? Why are you 503ing right now?! Just be open, clear, and always send clear signals with your HTTP status.
• Robots.txt getting jacked up (or going down)
  • Nothing is more irritating than doing everything right and it all getting messed up due to a robots.txt file.
    • Contained rant: [[IT’S A .TXT FILE GOSH DARN IT?!]] End rant. Rant has been contained.
  • Tip: Keep a tracker on your robots.txt to make sure you’re notified with any changes.
  • Tip: Use Uptime Robot to know the status of the robots.txt.
• SSR version ≠ consistent
  • Your SSR or pre-rendered version must be an accurate representation of the user’s version.
  • Tip: Keep a tracker on your SSR versions. Visual PNG works great for this.
• Large resources on page (speed)
  • There are a lot of page speed tools. I like WebPageTest.org. But maybe you don’t. Well, you could also use – Page Speed Insights, GTMetrix, Lighthouse, the myriad of DevTools features, your analytics platform + navigation API stuffs, tools like SpeedCurve. The list could go on for a while, so I’ll stop at – find what works for you here.

Rendering

• JavaScript not being rendered appropriately
  • Tip: Check out the page in Google’s Mobile-Friendly Testing Tool (a tool from Google that renders the DOM).
  • Tip: Switch your user agent to Googlebot or Bingbot or whatever bot you’re interested in.
• Recently migrated site
  • Well. Migrations are a whole ‘nother thing. When the URLs change, it takes time to shift everything, especially for large sites. Just make sure you’ve used 301 redirects and followed a ton of other important steps to make the migration as smooth as possible. Consult other posts on this one.

Indexing

• Noindex HTML tag or header
  • Tip: Use Screaming Frog (or other crawler of choice) to quickly check indexability. They do good work and you don’t have to individually check every tag and network header (yay!).
• Canonical tags
  • A correct canonical may not help you much, but an incorrect canonical will jack your site performance up.
  • Tip: Also use Screaming Frog (or other crawler of choice) to pull all (or a sample of) canonical tags.

Rankings

• Structured data not validating
  • Code doesn’t work if the syntax isn’t right or the vocabulary is off. I don’t make the rules, Schema.org makes them. We live in their world and it’s organized.

New Tech

• AMP not validating
  • Tip: Google [AMP validator], the big G added one in the SERPs.

• PWA not validating
  • Side note: PWAs aren’t something that should directly affect organic search performance, because (as it stands as of today) Googlebot’s web rendering service (the evergreen one), does not process service workers (because Googlebot always acts as a first-time visitor). Check out Martin Splitt’s chart and presentation, tweet below (it rocks). Theoretically, Googlebot should just ignore service workers (like an underappreciated salary man, he’s just trying to help, but never lauded for his work). Anyway, just because it doesn’t directly support your search ranking, doesn’t mean you shouldn’t do it for your users. And… it can still be a source of problems.

Our session on #javascript and #seo at 1:30pm tomorrow will be LIT🔥🔥🔥
Come to stage 3 to see what it's all about 🙃

Amongst other things, we'll talk:
🏗️Web app architectures
🤖Googlebot
⚡️Lazy loading
🚀Web APIs
🔬Testing
📈Crawl budget & more!https://t.co/6CGsiGgg9w pic.twitter.com/b4MnvpSmWt

— Martin Splitt @ 🇦🇹 #vienna (@g33konaut) May 9, 2019

Step 5: Suggest a Solution (Or Something That Gets Closer to One) + Write a *Succinct* Response

The motto:

“…Me think why waste time say lot word when few word do trick?” –Kevin Malone, The Office

Note: If you have a minute (watch Kevin’s famous chili), it’ll make ya chuckle.

You should have at this stage:

• Analysis notes.
• Screenshots.
• Labels.
• Recommendations on how to improve the current situation.

Here is one of my (personal) favorite formats for receiving emails:

Hello {person},

Thanks for sending that over! We dug into it and here’s what we recommend and found. (Niceties not necessary but appreciated.)

Recommended solution: {insert solution}

Issue cause: {understand the when and why to prevent it from happening}

Analysis note: {insert notes, bullet list, with labeled screenshots that tell a data analysis story of a journey (maybe even links to the reports, so I can check them out myself}

Your favorite friend right now,

{signature}

Read a completed version of this email (with all of the details) aloud softly and it’s basically ASMR for a technical SEO.

A Note for People on Conference Calls Who Are Facing a Fire(Drill)

1. Don’t panic.
   • Take a long breath (on mute, not in everyone’s ear, no one wants to hear your breathing).
2. Try to ask questions that will enable you to wrap your head around the problem (and buy some brain processing time).
3. If you don’t know the answer at the moment – admit it (it’s OK to not know everything, that’s what Google’s job is) and use the power of “fast follow-ups.”
   • Tip: Be vocal about the time you need to provide quality work and what you need the time for. You may need time to:
     • Dive into the data.
     • Consult with the team (and/or) manager.
     • Consult specific subject matter experts.
     • Simply to evaluate and process the options.

4. Make sure to follow up in a timely manner (an obvious step, that I knew you would already do, because you’re awesome).

A Note for People Without Technical SEO Fires

Maybe you’ve read this far, and you’ve never had a fire(drill). I appreciate you for reading this far and truthfully, I envy, respect, and want to be you.

But I wanted to give you something too, because you deserve it. If your UX is perfect – humans and bots alike love you, you’ve been through everything on this list.

Well – you can always aim to improve site speed. It’s not a great gift, but I don’t know you, so why should I be expected to know the best gift for you.

Closing Thoughts

Thanks for sticking in there, fighting the good fight, and reading this far (or skipping to the conclusion and missing out on all the great insights in the middle!).

Here’s a summary, in case you are really in need of the quick list.

Steps for Dealing with Fire(Drill)

1. Get into the right state of mind.
2. Seek to understand the problem.
3. Make plan a (even if it’s a plan on how to analyze).
4. Execute the plan (and or analyzing).
5. Suggest a solution (or something that gets closer to one).
6. Write a succinct write-up (with detailed analysis notes).
7. Take another breather, switch out of high-focus mode and take a moment to appreciate yourself and your work (because you just rocked that analysis).
8. You maybe also find a ton of potential action items to focus on (here).
9. Question, play, learn, test, report, repeat – QPLTRR. (The acronym could use some re-work, but it’s the vibe that counts!)
10. Reach out in the comments or on Twitter @AlexisKSanders.

(yes, those are pockets on the retro mario version of myself escaping down a mario pipe… and yes, it only took me about 1.5 hours to make this….)

*or may not

More Resources:

• The 20-Minute Technical SEO Workweek
• A Technical SEO Checklist for the Non-Technical Marketer
• Advanced Technical SEO: A Complete Guide

Image Credits

Featured Image:
In-Post Images/Screenshots: Created/taken by author, August 2019
Scientific method diagram: Science Buddies