Item: Screaming Frog SEO Spider
Rating: 10
Author: Anthony Daronzio

Use Cases and Deployment Scope

Screaming Frog is a standard tool of operation since I introduced it a couple years ago. We use it to perform crawls of sites that we are auditing for SEO, crawls of sites we have built to check for errors, both in staging and development. The project management team now uses it to QA certain implementations of tags, or other aspects on the site.

Pros and Cons

Best tool to crawl a site, hands down.
Configurable for custom data extractions.
In-depth exports for reporting.
Configurable connections to Analytics and Search Console.
Wealth of online documentation.

Exporting one report at time to CSV or XSLX can be cumbersome and time consuming.
Custom extraction requires pretty in-depth knowledge of RegEx or XPath.
Not a very visual tool.
Larges sites can take up a tremendous amount of resources (RAM, space, connection).

Return on Investment

Allows for quick and easy business development audits, allowing us to generate net new income.
Saves us time in the SEO department, hours saved is money earned.
Saved a tremendous amount of time and resources across Dev and PM.

Alternatives Considered

DeepCrawl

Screaming Frog is a desktop application that uses your own resources to crawl a site (Processing, RAM, Connection speed). Deep Crawl is a cloud-based crawler, and probably the best cloud based crawler out there. DeepCrawl can gather A LOT of data, but it becomes expensive. The larger the amount of pages you need to crawl and more projects you need open, the pricier it gets. If you know what you are looking at and don't need fancy charts and graphs, then Screaming Frog is the right tool for you over DeepCrawl.

Other Software Used

SEMRush, Conductor Searchlight, Ahrefs Site Explorer

Likelihood to Recommend

Screaming Frog is the best tool to crawl any site, anywhere, hands down. We use it for all of our SEO auditing purposes. We also use it to quality check new builds in both staging and live environments, some behind password restrictions and other various logins. Even our project management team has found use with it to quickly check changes across multiple pages instead of going to each page one by one. This tool is not the tool for you if you are looking for something to generate auditing reports. This will not will not make a pretty PDF of it's findings for you to show management or send to a client. It gives your raw data which you can export into a CSV or Excel, but will not help in the reporting process if you are looking for something to auto generate a report or insights.

Users and Roles

6 - SEO Department, SEM Department, Development and Project Management

Support Headcount Required

2 - The more intricate you get with the use case of Screaming Frog the bigger the skillset it takes to support it. Myself and my SEO Manager support internal use cases, sometimes knowledge of ReGex and XPath to do custom extraction.

Business Processes Supported

SEO Auditing
Quality Assurance with Dev
Quality Assurance with Project Management

Innovative Uses

We have a wide range of development clients, we use the tool to check updates to sites that the dev makes through custom extraction. When the change affects many pages, this makes it considerably easier to check changes as opposed to hours spent going page by page.
For SEO auditing, I export my reports that I need from a site crawl and copy that data into a google sheet, I then use that sheet as a data source to visualize the data from the crawl. This allows us to easily create reports to clients.

Future Planned Uses

Allowing for easier methods of custom extraction (intsead of writing regex or XPath) would allow us to bring this out to more users in the company.
Maybe an enhanced license to create PDFs of the stuff in the audits for use for our recurring audit program. This would allow the software to stay competitive from other tools such as SEMRush and DeepCrawl.

Likelihood to Renew

The tool has become integrated into our teams daily workings and I have yet to find a tool other than scremaing frog to replicate all of our use cases for it. It's a great tool and we're sticking with it.

Products Replaced

Key Differentiators

Price
Product Features
Product Usability
Product Reputation
Prior Experience with the Product
Third-party Reviews

Product Features is the most important selection from above. We need a tool that can be versatile and used across quite a few disciplines, and not just serve and SEO function. There are no other tools out there than can accomplish all that Screaming Frog has accomplished for us. We always look forward to any new features added in product updates.

Evaluation Lessons Learned

I wouldn't change my mind. Screaming Frog is the best tool there is for us.

Implementation Rating

If the software was a little easier to use for beginners, there really wouldn't be an issue.

Implementation Details / Implementation Partner

Implemented in-house

Implementation Phases

Change Management Lessons

Change management was minimal

Implementation Issues

Training others who did not know the software how to use it. It was rather time consuming.

Support Rating

I have had very little issues with the software, most of which can be answered with the very extensive online documentation. However the few times I have had to ask for support, they have been nothing but helpful.

Screaming Frog Customer Support Pros and Cons

Pros	Cons
Quick Resolution Good followup Knowledgeable team Problems get solved Kept well informed No escalation required Immediate help available Support understands my problem Quick Initial Response	None

Premium Support

Bug Resolution

Exceptional Examples of Screaming Frog Support

I have had standard actions for support, however this also classifies for support, when my company switched over to invoice billing and no longer used credit cards, I had to reach out to screaming frog for assistance. I was answered immediately and after a brief explanation of what I needed from them, it was sent over right away. I also had a problem because of wording on the invoice from my billing department. I emailed them back and it was changed very quickly and got back to my billing department without a lapse in the software usage.

Usability

Easy for me to use because I have using the product for a long time and understand the ins and outs of it. If I was a newbie, I would be completely lost. It does take some prior knowledge to operate this software and understanding of what a crawl means.

Usability Pros and Cons

Pros	Cons
Like to use Technical support not required Well integrated Consistent Feel confident using	Slow to learn Lots to learn

Easy Tasks

A standard crawl is easy and can be accomplished by anyone
Connecting to outside APIs

Difficult Tasks

Custom extraction is not easy to use without knowledge of RegEx (which I already knew) or XPath which I had to learn.
With reporting you have to export each report separately instead of one workbook created, with multiple sheets. This can get very cumbersome when you need a lot of the reporting, especially the SERP reporting.
For a beginner, it's hard to extrapolate the most important findings from your crawl.
Getting the report to visualized.

The best tool to have in your pocket, hands down.

Overall Satisfaction with Screaming Frog SEO Spider