Screaming for Screaming Frog
Updated May 19, 2021
Screaming for Screaming Frog
Score 9 out of 10
Vetted Review
Verified User
Software Version
4.1
Overall Satisfaction with Screaming Frog SEO Spider
My team and I use Screaming Frog to crawl websites to better understand the site hierarchy, unravel internal linking structures and verify live links across hundreds of domains. We can also easily find and download page titles, descripts and H1 tags in addition to the canonical links a page. What you can do with Screaming Frog is really limited by your imagination and desires. Our entire SEO team uses the product with every client that we touch and as a whole, the company has integrated Screaming Frog into every phase of the sales funnel. Our sales team uses it to estimate how big a site is and how many hours might be needed while our marketing team uses it to find the best placement for creative materials. It helps us to find information on a site without manually sifting through each webpage.
- Check status codes for a set of URLs - Simply copy and paste the URLs into the open box and within minutes you are able to see the status code, response time and potential redirects for any of the URLs. While there are online versions of this, Screaming Frog has the best copy/paste option and includes any other URL data you might need as well.
- Crawl URLs for a Custom Text - Without developing your own spider, there is no other way to do this other than visit each URL individually. This has saved us countless hours verifying proper analytics code implementation across an entire website and finding client mentions across hundreds of links.
- Finding Canonical Links - Every time Screaming Frog crawls a link or site they report back the basic information which happens to include canonical links. Again, without building your own spider or reviewing hundreds of URLs it would be a lengthy task to find all of this information.
- Customizing the crawler - It takes a little bit of playing around with the settings to understand what each of them controls and what options need to be checked or unchecked to get the right amount of information.
- Cannot Crawl a large site - Screaming Frog runs out of memory when asked to crawl a large site, even after playing around with memory settings and allocating additional memory.
- Increased employee efficiency - We spend many less hours tracking down and reporting simple meta tags.
- Better Client Servicing - We are able to ensure we have a total look at a client's site before we recommend anything.
- True Pricing Structure - We are able to more accurately predict how much help a client may need based on factors such as size of site, number of redirects, proper use of meta tags, etc.
Screaming Frog cannot return specific metrics such as domain authority or majestic stats and often the number of linking domains is off, while Netpeak can return almost any metrics you would like to see. Screaming Frog is vastly more controllable and user friendly than custom spiders but does have limitations about what it can return. Screaming Frog allows you to enter URLs from different domains and crawl each one, where Link Sleuth is not able to do that (only spider a single domain). Xenu is better at crawling larger websites on a limited memory budget, but does not return as much information, or have as many customization options as Screaming Frog.