JavaScript websites can be a b**ch!

I was speaking with a colleague this week who was having trouble with an eCommerce website they were working on. 

They knew there were hundreds of pages on the website, but Screaming Frog was only showing 50! The sitemap was included in the crawl, and hundreds of pages were flagged as orphaned pages.  

This didn't make sense, as they were sure the pages were linked internally! 

So why weren't they being found in the Screaming Frog crawl?

This was becoming a real headache!

I had a look at the site and realised that it was built with JavaScript, and the category pages were using JS widgets to share the products.

When JS is turned off in the browser, these widgets disappear. The same goes for the crawler; if JavaScript isn't turned on, it's hard for them to know the links and content are there.  

My colleague and I discussed this, and after altering some settings in the crawl configuration to turn on JavaScript rendering, we re-ran the crawl and the hundreds of pages that were previously "missing" were now being found. 

Always check to see how the website you are working on is built, and make sure you adjust your crawl settings accordingly. Whether using Screaming Frog, Sitebulb, or another crawler, they all have options for crawling JS websites.  

Check the technology and stop getting frustrated and scratching your head thinking that hundreds or thousands of pages are missing from the site! 

So, yes, JavaScript can be a b**ch, but there's usually a way around it! 
Tech SEO Tuesday - Tweet of the Week 👉 Canonical tags 👈

 Actionable SEO Tips and Advice 

Search engine news as well as articles, and more, from amazing people in the industry. 

How Earned 1M+ Organic Monthly Visits Through Content Syndication [Case Study] 
Amanda Milligan - Moz

Googlebot Doesn’t Click On Buttons – What to Use Instead
Matt G. Southern - Search Engine Journal

Moneyball is the future of SEO
Will Critchlow - Search Pilot

[Video] How to SEO optimize your eCommerce website (8 Tips)
Google Search Central

 Featured Release 

Maddy Osman is releasing a new book on 18th July 2022 called "Writing for Humans and Robots: The New Rules of Content Style", which is a homage to "The Elements of Style" and the AP Stylebook.

The book provides helpful guidance for online content creators who strive to build an excellent reader experience. It includes testimonials from Andy Crestodina, Kaleigh Moore, Ryan Robinson, Mordy Oberstein, Joe Casabona, and several other mentors of Maddy's. 

Pre-order / Purchase a copy of the book yourself:

Maddy Osman is a digital native with a decade-long devotion to creating engaging, accessible, and relevant content. After teaching herself web design at age 11, she found her true passion for content creation. 

Her efforts earned her a spot in Semrush’s and BuzzSumo’s Top 100 Content Marketers and The Write Life’s 100 Best Websites for Writers. She has spoken for audiences at WordCamp US, SearchCon, and Denver Startup Week. 
Get Writing for Humans and Robots Now
Buy Me A Coffee
Do you have Feedback? 
If you'd like to give me a heads up on any SEO resources to include in the next edition, if you'd like to be featured, become a sponsor or if you have any feedback to improve Tech SEO Tips, just reply to this email and let me know. 

Nikki Halliwell
Technical SEO Consultant
Follow me on Twitter | LinkedIn | Facebook | Website 
Share Tech SEO Tips Newsletter:
Share Share
Tweet Tweet
Forward Forward
If you enjoy Tech SEO Tips, I would really appreciate you spreading the word by sharing this link:

If you're looking for SEO consultancy, audits, white label SEO or simply need a second opinion on your current strategy, I'd be happy to chat.

Drop me a message at and I'll get back to you as soon as I can.

Copyright © 2022 Nikki Halliwell SEO, All rights reserved.

Want to change how you receive these emails?
You can update your preferences or unsubscribe from this list.

Email Marketing Powered by Mailchimp