Screaming Frog SEO Spider

Take a websites URL address, give it to Screaming Frog SEO Spider, and hit the Start button to initiate the crawling process. Depending on how large the website is, this may take a while to complete. But once it’s done, Screaming Frog SEO Spider will bring up a detailed report that features lots of useful information: broken links, temporary or permanent redirects, loops, page titles and metadata, duplicate content, and more.

Screaming Frog SEO Spider is available for Windows, Mac, and Ubuntu. Please note that this website crawler needs Java to run. So if you install it on your PC and it refuses to start, it might be a Java-related problem and you need to make sure you have the latest version of Java on your PC. You should also note that the developer recommends using Screaming Frog SEO Spider on a 64-bit PC that has 8GB of RAM or more.

The main element on this application’s interface is the search box featured at the top, underneath the traditional File Menu. The rest of the interface won’t display anything unless you enter a URL in the search box and hit the Start button. This will tell Screaming Frog SEO Spider to crawl the website you gave it and, once the process completes, a wealth of information will be displayed on the interface.

Screaming Frog SEO Spider is available as a Free and as a Paid version. The free version will find broken links, analyze page titles and metadata, review meta robots and directives, discover duplicate pages, and generate XML sitemaps. For extra features such as crawl configuration, custom source code search, custom extraction, search console integration, custom robots.txt, and Google Analytics integration, you have to get the Paid version.

You should also note that Screaming Frog SEO Spider Free is limited to crawling 500 URLs. The paid version on the other hand will crawl an unlimited number of URLs.

Screaming Frog SEO Spider will retrieve important website information and will help you find broken links, redirects, duplicate content, and other things that might have a negative impact on your website’s traffic.