Reference

Technical details for the systems that underpin every crawl: the issue rules applied to each page, the SQLite database that stores results, and the structured data extraction and validation pipeline.

Issue rules

Every crawled page is evaluated against 17 issue rules organized into eight groups: response codes, titles, descriptions, H1 tags, canonicals, content depth, performance, and architecture. Each rule carries a severity — critical, warning, or notice — and produces a finding when a page fails it. The same rule set runs in the desktop app, CLI, cloud tier, and API, so a page flagged in one surface is flagged in all of them. See the issue rules reference for the complete list with thresholds and severities.

The crawl database

Crawl results are stored in SQLite with WAL mode enabled, either as one file per crawl or in a shared database holding multiple sessions. Tables cover sessions, pages, links, images, headless analysis, redirect chains, and more. Because it is ordinary SQLite, you can query it directly with any tool that speaks the format — the sqlite3 CLI, DB Browser, Datasette, or your own scripts — alongside what the desktop and cloud interfaces surface. See the crawl database reference for the full schema.

Structured data

Consuela extracts JSON-LD, Microdata, and RDFa from every crawled page, validates properties against Schema.org definitions, and evaluates Google rich result eligibility for 27 types. The eligibility checks track restrictions and removals — FAQ rich results were removed in June 2025 and HowTo in September 2023, so pages carrying that markup are flagged rather than reported as eligible. See the structured data reference for supported types and validation details.