Crawl Comparison

Comparing two crawls of the same site is how you answer the question that follows every deploy, every migration, and every round of fixes: did the site get better or worse? The comparison dialog diffs a baseline crawl against a newer one and reports what changed — pages added, removed, or altered, and every audit check that moved in either direction.

Selecting sessions

Open the comparison dialog from the toolbar. It presents two dropdowns: Baseline (older) and Compare to (newer). Each lists every completed crawl session, showing the start time, site URL, and page count.

The dialog auto-selects the two most recent crawls of the same site, which is the comparison people reach for almost every time — run a crawl, make changes, run another crawl, and diff the two. You can override either selection to compare any two sessions, including sessions from different sites when that is useful. Selecting the same session for both sides is blocked with a warning.

The comparison runs automatically once both sessions are selected. There is no separate "Compare" button to press — changing either dropdown triggers a fresh diff.

Summary

The top of the dialog shows four summary tiles:

Tile	What it counts
Added	Pages present in the newer crawl that were not in the baseline.
Removed	Pages present in the baseline that are absent from the newer crawl.
Changed	Pages present in both crawls whose status code, content, or metadata differs.
Total pages	The newer crawl's total page count, with a +/- delta against the baseline.

The Total pages tile colours its delta by direction: a higher count renders green (more pages discovered is generally good news), while a lower count renders red. The other three tiles are clickable and act as filters — click one to load the corresponding page list below.

Page changes

Clicking an Added, Removed, or Changed tile expands a scrollable list of the affected URLs. Clicking the active tile again collapses it.

Each row shows the URL and, for pages whose HTTP status code changed between crawls, the old and new status codes side by side — for example, 200 → 404 or 301 → 200. Each status code is coloured by its own class (2xx, 3xx, 4xx, 5xx), so the nature of the change is visible at a glance rather than conveyed by a single colour standing in for both sides.

Below the status codes, a list of change descriptions summarises what else differed: title changes, canonical changes, content changes, and any other metadata the crawler tracks. This makes it possible to scan a deploy's impact without opening every page individually.

Issue changes

Below the page diff, the Issue changes section lists every audit check that moved between the two crawls. Each row shows the check's group and label, the base count and target count (for example, 14 → 3), and a coloured delta.

Direction determines the colour: a positive delta (more pages failing the check) renders red because it is a regression, regardless of the check's severity. A negative delta (fewer pages failing) renders green — an improvement. Checks that did not move at all between crawls are omitted, so the list shows only what actually changed.

The deltas come from the same audit engine the main punch list uses, so the counts always agree. If a check shows 14 affected pages in the results table, the comparison will show that same number as the base or target count.

Rows are sorted by the magnitude of the change, largest first, so the most significant movements — whether improvements or regressions — appear at the top. This is useful on large sites where dozens of checks may shift by one or two pages, but only a handful move by enough to warrant attention.

Typical workflows

Crawl comparison fits naturally into several recurring tasks:

Workflow	How comparison helps
Tracking SEO progress	Run a baseline crawl at the start of an engagement, then compare after each round of work. The issue deltas show which checks improved and by how much.
Verifying fixes	After addressing findings from an audit, re-crawl and compare. A check that dropped from 47 to 0 confirms the fix landed across the site, not just on the pages you tested by hand.
Catching regressions after a deploy	Crawl before the deploy, crawl after, and diff the two. New 404s, status code changes, and issue count increases surface immediately rather than weeks later in search console data.
Monitoring a site over time	Periodic crawls with comparison give an ongoing record of how a site's technical health evolves. Each comparison is a snapshot of what changed since the last inspection.