Automate Schema Markup at Scale: A Python + LLM Pipeline for JSON-LD
Schema markup is one of the highest-leverage technical SEO assets you can ship, and also one of the most tedious to maintain by hand. Every product page wants Product + Offer markup, every recipe wants Recipe, every guide benefits from FAQPage or HowTo, and every one of them has to stay valid as Google deprecates properties and tightens its rich-results requirements. On a 50-page site you can do this manually. On a 5,000-URL programmatic site, manual JSON-LD is a non-starter.
This post walks through a production pipeline that generates, validates, and deploys structured data at scale using Python and an LLM, with a hard validation gate so you never ship markup that Google will reject. It is the same architecture I use to keep schema coverage above 95% across large content sites, and it slots directly alongside the programmatic SEO pipeline with a quality gate you may already be running.
Why “generate with an LLM” is the wrong default — and the right tool
The naive approach is to prompt an LLM with “write JSON-LD for this page” and paste the result into your template. This fails in two predictable ways. First, LLMs hallucinate properties that don’t exist in the schema.org vocabulary or that Google ignores, inflating your markup with dead weight. Second, they pull values that aren’t actually present on the page — an invented aggregateRating is a structured-data spam violation that can earn a manual action, not a rich result.
The fix is to constrain the model tightly: the LLM’s only job is mapping already-extracted page data onto the correct schema type. Extraction is deterministic Python. Validation is deterministic Python. The LLM sits in the middle doing the fuzzy classification work it’s good at — deciding that this URL is a HowTo and that one is a Product — while the facts come from the DOM, not the model’s imagination.
Pipeline architecture
The pipeline has five stages, each of which can run independently in a queue so a failure in one URL never blocks the batch:
- Crawl & extract — pull each URL’s title, headings, body text, existing microdata, breadcrumbs, prices, author, and publish date into a structured record.
- Classify — decide which schema type(s) the page should carry.
- Generate — map the extracted fields onto a JSON-LD object for that type.
- Validate — check the JSON-LD against schema.org and Google’s rich-results requirements; reject anything that fails.
- Deploy & ping — write the markup into the page and trigger re-indexing.
Stage 1 — Deterministic extraction
Extract everything you can without a model. This is the data the rest of the pipeline trusts.
import requests
from bs4 import BeautifulSoup
from dataclasses import dataclass, field
@dataclass
class PageRecord:
url: str
title: str = ""
h1: str = ""
headings: list = field(default_factory=list)
author: str = ""
published: str = ""
price: str = ""
body_text: str = ""
def extract(url: str) -> PageRecord:
html = requests.get(url, timeout=20,
headers={"User-Agent": "schema-bot/1.0"}).text
soup = BeautifulSoup(html, "html.parser")
rec = PageRecord(url=url)
rec.title = (soup.title.string or "").strip()
if soup.h1: rec.h1 = soup.h1.get_text(strip=True)
rec.headings = [h.get_text(strip=True)
for h in soup.select("h2, h3")]
meta_author = soup.find("meta", attrs={"name": "author"})
if meta_author: rec.author = meta_author.get("content", "")
pub = soup.find("meta", property="article:published_time")
if pub: rec.published = pub.get("content", "")
rec.body_text = soup.get_text(" ", strip=True)[:6000]
return rec
Notice the body text is truncated. The classifier doesn’t need the whole page — it needs enough signal to pick a type, and trimming keeps your token costs flat as the site grows.
Stage 2 — Classify the schema type
This is the one step where the LLM adds real value. Give it the page record and a closed list of supported types, and force a structured response so you never have to parse prose.
import json
from openai import OpenAI
client = OpenAI()
SUPPORTED = ["Article", "HowTo", "FAQPage",
"Product", "Recipe", "Course", "Event"]
def classify(rec: PageRecord) -> str:
prompt = (
"Pick the single best schema.org type for this page "
f"from {SUPPORTED}. Return JSON: {{\"type\": \"...\"}}.\n"
f"Title: {rec.title}\nH1: {rec.h1}\n"
f"Headings: {rec.headings[:12]}"
)
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": prompt}],
response_format={"type": "json_object"},
temperature=0,
)
t = json.loads(resp.choices[0].message.content)["type"]
return t if t in SUPPORTED else "Article"
Setting temperature=0 and validating the output against SUPPORTED turns a probabilistic model into a deterministic-enough classifier. The same disciplined LLM-as-mapper pattern powers our vision-LLM alt-text pipeline — constrain the output, validate the result, never trust raw generation.
Stage 3 — Generate JSON-LD from extracted fields
Build the JSON-LD in Python using the fields you already extracted, not values the model invents. For the parts that genuinely need synthesis — like turning H2/H3 steps into a HowTo step list, or generating FAQ pairs — pass the LLM only the verbatim page text and require it to quote, not paraphrase.
def build_article(rec: PageRecord) -> dict:
node = {
"@context": "https://schema.org",
"@type": "Article",
"headline": rec.h1 or rec.title,
"url": rec.url,
}
if rec.author:
node["author"] = {"@type": "Person", "name": rec.author}
if rec.published:
node["datePublished"] = rec.published
return node
BUILDERS = {"Article": build_article, "HowTo": build_howto,
"FAQPage": build_faq, "Product": build_product}
def generate(rec: PageRecord) -> dict:
return BUILDERS.get(classify(rec), build_article)(rec)
Keeping one builder function per type means every property is something you chose to emit. There is no path for a stray priceValidUntil or fabricated rating to slip in, because the builder simply never writes fields the page doesn’t support.
Stage 4 — The validation gate
This is the stage that makes the difference between “we generate schema” and “we ship schema Google trusts.” Validate every object before it touches a live page. A lightweight gate checks three things: the JSON parses, the required properties for that type are present, and there are no empty values masquerading as data.
REQUIRED = {
"Article": ["headline"],
"HowTo": ["name", "step"],
"FAQPage": ["mainEntity"],
"Product": ["name"],
}
def validate(node: dict) -> tuple[bool, list]:
errors = []
t = node.get("@type", "")
for prop in REQUIRED.get(t, []):
v = node.get(prop)
if not v:
errors.append(f"missing required: {prop}")
if node.get("@context") != "https://schema.org":
errors.append("bad @context")
return (len(errors) == 0, errors)
For a stronger gate, feed each object through Google’s Rich Results Test programmatically or run the open-source schema validator in CI. Whatever you use, the rule is absolute: a record that fails validation does not deploy. It goes to a review queue. This mirrors the quality-gate philosophy that keeps automated content from poisoning your index — generation is cheap, but only validated output earns a slot on the live site.
Stage 5 — Deploy and trigger re-indexing
Write the validated JSON-LD into a <script type="application/ld+json"> block in the page <head> — via your CMS API, a template variable, or an edge worker that injects it at the CDN. Once the markup is live, don’t wait for the next organic crawl. Push the changed URLs straight to search engines with the same approach from our IndexNow and Google Indexing API guide so the new rich-result eligibility is picked up in hours rather than weeks.
Results and what to measure
The metric that matters is not “schema coverage” in the abstract — it’s valid, eligible markup that earns enhancements. Track three numbers in Google Search Console’s Enhancements and Rich Results reports: the count of valid items by type, the count of items with errors (your gate should keep this near zero), and the click-through-rate delta on URLs that gained a rich result versus a control set that didn’t. On the sites I run this pipeline against, the validation gate typically drops error-state items from the double digits to under one percent, because nothing invalid ever ships in the first place.
A realistic rollout on a few thousand URLs costs a few dollars in model calls — the classifier runs on a mini model, generation is mostly Python, and you only pay tokens for the genuine synthesis work. The expensive part was never compute; it was the engineering discipline of never trusting unvalidated output.
Takeaways
Treat the LLM as a constrained mapper, not an author: extraction and validation are deterministic Python, and the model only classifies and synthesizes from verbatim page data. Build one explicit JSON-LD builder per schema type so every emitted property is intentional. Put a hard validation gate between generation and deployment, and route failures to a queue instead of to production. Finally, close the loop by pinging the indexing APIs the moment markup goes live. Do this and structured data stops being a manual chore and becomes a self-maintaining layer of your SEO stack.
Want the next automation playbook in your inbox? Bookmark SEOAutomationClub and check back each week — we ship a new working pipeline you can copy. If you’re building the broader system, start with our programmatic SEO pipeline with a quality gate and layer this schema stage on top.
Frequently asked questions
Should JSON-LD or microdata be used for schema at scale?
Use JSON-LD. Google explicitly recommends it, and it decouples your structured data from your HTML markup, which makes it far easier to generate, validate, and inject programmatically. Microdata forces you to weave attributes through your template, which doesn’t scale cleanly across thousands of dynamically generated pages.
Will Google penalize AI-generated schema markup?
Google penalizes inaccurate or spammy structured data — markup that describes content not visible on the page, or fabricated ratings and reviews — regardless of how it was produced. AI-generated JSON-LD that accurately reflects on-page content and passes the Rich Results Test is completely fine. The validation gate in this pipeline exists precisely to keep your markup truthful.
How often should automated schema markup be re-validated?
Re-validate whenever the underlying page content changes, and run a full sweep monthly to catch Google’s evolving requirements and any schema.org deprecations. Wiring the validator into CI so it runs on every content deploy is the cleanest approach — it catches regressions before they reach production.
Can this run without writing Python, in a no-code tool?
Yes — the same five stages map onto an n8n or Make workflow: an HTTP node for extraction, an LLM node for classification, a function node for the builders, a validation node, and a CMS node for deployment. The Python version gives you tighter control over the validation gate, but the architecture is identical regardless of the orchestrator.
