Screaming Frog SEO Spider helps you crawl a website, find technical inconsistencies and trace them to the pages that need editing. The useful outcome is a verified repair list, not the largest possible issue count. A flagged URL may be intentional, outside the crawl scope or affected by the crawler's settings.
Prepare the scope and install the current official version
Download the current version from Screaming Frog’s official site. The free SEO Spider crawls up to 500 URLs; saving crawls, advanced configuration and JavaScript rendering require a licence. Check the installed version and licence before planning a repeatable audit. A free crawl that hits its limit is an incomplete sample, not a full inventory of a larger site.
Identify the intended domain, subdomains, protocols and path scope. Exclude irrelevant search or parameter combinations where appropriate, and choose a crawl rate that the authorized target can handle. Do not bypass access controls just to make the crawl larger.
Run a small crawl first
Use Spider mode to discover URLs from a starting address, or List mode to check known URLs. Enter the intended canonical starting URL, then inspect sample responses, redirects and discovered links before expanding. With the relevant licensed features, save the crawl and configuration for later comparison. With the free version, record the start URL, version, settings and exported results; do not assume you can reopen a saved crawl or render JavaScript.
If only the homepage and a few script files appear, investigate link discovery and rendering before concluding the site has no other pages. If the URL count expands unexpectedly, examine query strings, calendars, faceted navigation and subdomain settings.

Read the main reports as questions to investigate
| Report area | Useful question | Possible action |
|---|---|---|
| Response Codes | Which internally linked pages fail or redirect unnecessarily? | Fix the link or the intended destination; keep legitimate removals |
| Page Titles and Meta Description | Are important pages missing accurate, distinct descriptions? | Update the actual page fields or their template source |
| H1 and content | Does the visible heading describe the page's main task? | Resolve template mistakes or misleading page copy |
| Canonicals and directives | Are indexing and canonical choices intentional and consistent? | Correct conflicts after deciding the desired URL |
| Images | Are meaningful images accessible and appropriately described? | Repair broken resources, context and useful alternative text |
| Internal links | Can readers reach important answers through relevant paths? | Add or repair links where they serve the source page |
A long title is not automatically a ranking penalty. A decorative image can correctly have empty alt text. A canonical pointing elsewhere may be intentional. Review the page's purpose before changing every row in a filtered report.
Trace a broken URL back to its source links
Select an affected URL and inspect its Inlinks to identify the pages and anchor text that point to it. Export the relevant records for editing. When there is a genuine replacement, update links to that destination and apply appropriate redirects where needed. If no replacement exists, remove or revise the obsolete reference rather than sending every 404 to the homepage.
Check the final response after each repair. A target returning 200 can still contain the wrong page, a login screen or an error message.
Understand what an orphan-page check requires
A crawl cannot discover an orphan by following links from its starting page if no path reaches that URL. Screaming Frog’s orphan-page tutorial notes that orphan pages can still link to one another. Compare the crawl with an appropriate sitemap, CMS export or authorized analytics/search inventory. Normalize redirects and alternate URL forms, and review deliberately excluded areas before deciding a page is orphaned.
“No contextual link in article bodies” is narrower than “no internal links anywhere.” Archives, navigation and pagination may still make a page reachable. Add editorial links according to relevance, using the internal-link review method, rather than indiscriminately linking every candidate.
When JavaScript rendering is necessary
The official JavaScript rendering tutorial explains the licensed rendering workflow: choose JavaScript under Configuration → Spider → Rendering; when original and rendered HTML are needed for comparison, enable their storage under Extraction before crawling. Select a URL and inspect the rendered page, HTML and discovered links. Start with representative templates because rendering consumes more resources.
If the capture is incomplete, review blocked resources and the crawler's timeout rather than treating the tool's default as Google's indexing limit. Confirm important findings using the rendering comparison workflow.
A repair record you can actually use
Teaching example: a product guide links to a retired specification. Record the source guide, its anchor, the broken destination and the intended replacement document. Check that the replacement covers the same model and revision before updating the source link. If it does not, revise the sentence or remove the obsolete reference instead of choosing a merely accessible URL.
After the edit, recrawl both the source and destination. Confirm the old link has gone, the new response is correct, and the document still supports the source sentence. A smaller error count alone would miss a link redirected to an unrelated page.
Prioritize and verify the work
For each confirmed issue, record the affected URL or template, evidence, intended behavior, proposed edit and retest result. Fix access or unintended indexing barriers on important pages before spending time shortening every URL or chasing an arbitrary health score.
After deployment, rerun the affected scope with the saved configuration and read the live pages. Successful crawling is a technical milestone; indexing, rankings, clicks and qualified business outcomes remain separate observations.