Monitor Structured Data at Scale: A Python Pipeline to Catch Silent Schema Breakage Before You Lose Rich Results

Structured data breaks quietly. A developer ships a template refactor, a plugin update rewrites the page head, or a CMS migration drops a field — and your Product markup loses its offers block. Google stops showing the price and review stars, your rich result vanishes, and click-through quietly erodes. Nobody gets an alert, because nothing “errored.” The page still returns 200, the content still renders, and the JSON-LD is still there — it’s just invalid now.

Search Console’s Rich Results reports will eventually catch it, but they lag by days to weeks and only sample a fraction of your URLs. The Rich Results Test is manual and one URL at a time. Neither scales to a site with thousands of templated pages, and neither tells you the exact moment a deploy introduced the regression. This post builds a Python pipeline that extracts structured data across your whole site, validates it against the fields Google actually requires, stores a snapshot, and diffs each run so you catch silent schema breakage the same day it ships.

Why structured-data monitoring is a distinct problem

Most teams treat structured data as a one-time implementation task: add the markup, pass the Rich Results Test, move on. But schema is not static config — it’s data assembled at render time from your CMS, your theme, and often JavaScript. Every one of those layers is a place it can silently degrade. In practice I’ve seen three recurring failure modes:

Field drift. The markup is valid JSON-LD and technically parses, but a required property Google needs for the rich result — aggregateRating on a Product, datePublished on an Article, address on a LocalBusiness — disappears. Validators that only check “is this valid Schema.org?” pass it. You still lose eligibility.

Type collapse. A template change flips @type from Product to Thing, or nests an entity one level too deep so the parser no longer recognizes it. The JSON is fine; the meaning is gone.

Render-only breakage. The JSON-LD is injected by JavaScript, so it exists in the rendered DOM but not the raw HTML. A crawler that reads raw HTML sees nothing, and if the JS bundle breaks, the markup vanishes for Googlebot too. This is the same rendered-vs-source gap covered in rendering parity audits, applied specifically to structured data.

A useful monitor has to detect all three — which means it can’t just validate syntax. It has to know what each type needs and compare today’s snapshot against yesterday’s.

Extract: pull every flavor of structured data

Structured data comes in three syntaxes — JSON-LD, Microdata, and RDFa. You want a library that handles all of them so you don’t miss legacy Microdata hiding in older templates. The extruct library does exactly this and normalizes everything into Python dictionaries.

import extruct
import requests
from w3lib.html import get_base_url

UA = "Mozilla/5.0 (compatible; SchemaMonitor/1.0)"

def extract_schema(url: str) -> list[dict]:
    resp = requests.get(url, headers={"User-Agent": UA}, timeout=20)
    resp.raise_for_status()
    base = get_base_url(resp.text, resp.url)
    data = extruct.extract(
        resp.text,
        base_url=base,
        syntaxes=["json-ld", "microdata", "rdfa"],
        uniform=True,  # normalize all syntaxes to a JSON-LD-like shape
    )
    # Flatten every detected entity into one list
    items = []
    for syntax in ("json-ld", "microdata", "rdfa"):
        items.extend(data.get(syntax, []))
    return items

For pages that inject JSON-LD via JavaScript, swap the requests.get for a Playwright fetch that returns page.content() after the network settles. Feed that HTML into the same extruct.extract call. Keep both fetch paths available and record which one you used — a discrepancy between raw and rendered extraction is itself a signal worth flagging.

Validate against what Google actually requires

This is where generic validators fall short. Being valid Schema.org is necessary but not sufficient for a rich result — Google publishes its own required and recommended property lists per feature. Encode those requirements as a small rule table so the monitor checks for eligibility, not just syntactic validity.

REQUIRED_FIELDS = {
    "Product": {"name", "image", "offers"},
    "Article": {"headline", "image", "datePublished"},
    "Recipe": {"name", "image", "recipeIngredient", "recipeInstructions"},
    "FAQPage": {"mainEntity"},
    "LocalBusiness": {"name", "address"},
    "BreadcrumbList": {"itemListElement"},
}

def normalize_type(entity: dict) -> str:
    t = entity.get("@type", "")
    if isinstance(t, list):
        t = t[0] if t else ""
    return t.split("/")[-1]  # handle full schema.org URLs

def validate_entity(entity: dict) -> list[str]:
    etype = normalize_type(entity)
    required = REQUIRED_FIELDS.get(etype)
    if not required:
        return []  # type we don't track; skip
    present = set(entity.keys())
    missing = required - present
    return [f"{etype}: missing '{f}'" for f in sorted(missing)]

Extend the table for whatever rich results matter to your site. The point is that the rule set is yours and version-controlled, so when Google tightens a requirement (as it periodically does — the Product snippet rules have changed more than once), you update one dictionary and the whole fleet is re-checked on the next run.

Snapshot and diff: catching the moment of breakage

Validation alone tells you what’s broken today. The real value is knowing what changed since the last run, because that pins the regression to a specific deploy window. Store a compact fingerprint of each URL’s schema and compare runs. SQLite is more than enough.

import sqlite3, json, hashlib
from datetime import datetime, timezone

def fingerprint(items: list[dict]) -> dict:
    """Reduce a page's schema to types present + a content hash."""
    types = sorted(normalize_type(i) for i in items)
    blob = json.dumps(items, sort_keys=True, default=str)
    return {"types": types, "hash": hashlib.sha1(blob.encode()).hexdigest()}

def run_and_store(urls: list[str], db="schema.db"):
    con = sqlite3.connect(db)
    con.execute("""CREATE TABLE IF NOT EXISTS snapshots(
        url TEXT, ts TEXT, types TEXT, hash TEXT, errors TEXT)""")
    now = datetime.now(timezone.utc).isoformat()
    for url in urls:
        try:
            items = extract_schema(url)
        except Exception as e:
            record_alert(url, f"fetch failed: {e}")
            continue
        errors = [msg for it in items for msg in validate_entity(it)]
        fp = fingerprint(items)
        prev = con.execute(
            "SELECT types, hash, errors FROM snapshots WHERE url=? "
            "ORDER BY ts DESC LIMIT 1", (url,)).fetchone()
        diff_against_previous(url, fp, errors, prev)
        con.execute("INSERT INTO snapshots VALUES (?,?,?,?,?)",
            (url, now, json.dumps(fp["types"]), fp["hash"], json.dumps(errors)))
    con.commit(); con.close()

The diff_against_previous function is where you decide what deserves an alert. A hash change with identical types is usually harmless content editing. A type disappearing, or a URL going from zero errors to several, is a regression worth waking someone up for.

A severity model instead of a wall of warnings

If you alert on everything, people mute the channel and you’re back to catching breakage weeks late. Rank findings so the signal survives. I use three tiers, weighted by business impact rather than technical noise:

def severity(url, fp, errors, prev):
    if prev:
        prev_types = set(json.loads(prev[0]))
        lost = prev_types - set(fp["types"])
        if lost:
            return "CRITICAL", f"lost entity types: {sorted(lost)}"
    if errors:
        return "HIGH", f"{len(errors)} eligibility gaps: {errors[:3]}"
    if not fp["types"]:
        return "MEDIUM", "no structured data detected"
    return "OK", ""

Critical is a type that existed yesterday and is gone today — a near-certain deploy regression on a page that previously earned a rich result. High is a validation gap: the type is present but missing a required field, so eligibility is at risk. Medium is a page with no detected markup where the template suggests there should be some. Route Critical straight to Slack, batch High into a daily digest, and let Medium accumulate for your next content sprint. This mirrors the guardrail philosophy behind automated robots.txt and meta-robots monitoring: watch the few signals that quietly cost you traffic, and make the alert impossible to ignore when they fire.

Scheduling and CI: shift the check left

Run the full-fleet scan on a schedule — daily via cron or GitHub Actions is a sane default — seeding the URL list from your XML sitemap so new templates are covered automatically. But the higher-leverage move is running a scoped version in CI against a staging build before code merges. Point the extractor at a handful of representative URLs per template on your preview environment and fail the build if any Critical or High regression appears. That turns “we lost rich results last Tuesday” into “this pull request would have broken Product markup on 400 pages,” caught before it ships.

The parallel here is deliberate: this is silent-breakage monitoring applied to markup, the same way the index coverage watchdog is silent-breakage monitoring applied to indexation. In both cases the failure is invisible in the moment, expensive over weeks, and trivially catchable with a diff against yesterday’s state.

Where to draw the line

Two honest caveats. First, this monitors presence and shape, not truth — it can’t tell you your price is wrong, only that the offers field exists. Pair it with spot checks against the live Rich Results Test API for your highest-value templates. Second, don’t try to encode all of Schema.org; track the four or five types that actually drive rich results on your site and ignore the rest. A monitor that’s narrow and trusted beats an exhaustive one that everyone mutes.

Frequently asked questions

How is this different from Google’s Rich Results report in Search Console?

Search Console reports are sampled, lag by days to weeks, and don’t tie a regression to a specific deploy. This pipeline scans every URL you feed it on your schedule, validates against your own eligibility rules, and diffs against the previous run so you can pinpoint exactly when markup broke.

Do I need Playwright, or is requests enough?

If your JSON-LD is in the raw HTML, requests plus extruct is enough and far cheaper. Use Playwright only for pages that inject structured data via JavaScript. Recording which fetch path found the markup also surfaces render-only breakage, where the schema exists in the DOM but not the source.

Won’t this create alert fatigue?

Only if you alert on everything. The severity model routes a lost entity type (Critical) to real-time notification while batching validation gaps into a daily digest and letting missing-markup findings accumulate. Alert on regressions and eligibility loss, not on harmless content edits.

How many URLs can I realistically monitor?

With plain requests and modest concurrency, tens of thousands of URLs per run is straightforward. The bottleneck is politeness, not compute — throttle requests, respect crawl budget, and sample one or two representative URLs per template rather than crawling every page if your site is very large.

Can I run this in CI to prevent breakage before it ships?

Yes, and that’s the highest-leverage use. Point the extractor at representative staging URLs per template and fail the build on any Critical or High regression. That catches template refactors that would strip required fields before they reach production.