How to Audit Your Site for Programmatic Content Risk in 30 Minutes

Most content sites have programmatic content patterns hiding in plain sight. The site we audited and discussed in the opening post of this series had been running its templated pipeline for nine months before anyone on the team treated it as a risk. By the time the demotion landed, more than 50% of the portfolio carried a fingerprint that a classifier could detect in seconds. The audit that would have surfaced the same pattern in advance takes 30 minutes and uses tools you already have.

This post is that audit, written out step by step. By the end, you will have an inventory of which URL patterns on your site are concentrated, which posts behind those URLs share a templated structure, and a decision tree for what to do with each cohort — keep, rewrite, consolidate, or unpublish. We will use the same case-study site as a worked example throughout, with all identifying information stripped.

What you need to start

You need three inputs and one piece of software. The inputs:

• A full URL list for the site you are auditing. Export it from Search Console (Pages report → Export → CSV), from your sitemap, or from a crawl. Aim for completeness, not just the URLs that get traffic.
• A sample of post bodies for the sites in question. The WordPress REST API returns this at /wp-json/wp/v2/posts?per_page=100&_fields=id,slug,title,content with pagination; static sites can ship a JSON dump from a build script.
• Search Console “Pages” report data, filtered to Crawled — currently not indexed. This is the clearest single tell that the classifier is already discounting URLs on your site.

The software is Python with pandas and beautifulsoup4. Nothing exotic.

Step 1 — Extract all slugs and bucket by pattern

The first signal we are looking for is slug-pattern density. Templated pipelines produce URLs that all share the same opening tokens because the prompt that generated the title also generated the slug.

import re, collections
import pandas as pd

def slug_bucket(url):
    slug = url.rstrip("/").split("/")[-1]
    parts = slug.split("-")
    # First two tokens capture the template archetype:
    # "best-X-for-Y" -> "best-*"; "how-to-write-X" -> "how-to-*"
    if parts[0] in {"how", "best", "what", "why"} and len(parts) >= 2:
        return f"{parts[0]}-{parts[1]}-*"
    return f"{parts[0]}-*"

df = pd.read_csv("urls.csv")
df["bucket"] = df["url"].map(slug_bucket)
counts = df["bucket"].value_counts()
print(counts.head(20))

Two thresholds tell you whether you have a problem:

• Any single bucket above 100 URLs. That is a template that has been run at scale.
• Top three buckets account for more than 30% of the site. That is a content operation that produces output along a small number of fixed axes — which is exactly the shape the classifier flags.

On the case-study site, the top three buckets — best-*-for-*, how-to-write-*, and how-to-use-* — together accounted for 41% of the URL list. That number alone was a leading indicator that should have been flagged months before the demotion.

Step 2 — Run the regex over the literal patterns

Slug bucketing tells you which archetypes exist. The next step is to run a regex over the full URL list to surface the literal patterns inside each archetype. Templated pipelines tend to fill the same slot with similar parameter words.

patterns = {
    "best_for":  re.compile(r"^best-[a-z]+-for-[a-z]+(-[a-z]+)?$"),
    "how_write": re.compile(r"^how-to-write-[a-z-]+$"),
    "how_use":   re.compile(r"^how-to-use-[a-z-]+$"),
    "what_is":   re.compile(r"^what-is-[a-z-]+$"),
    "vs":        re.compile(r"^[a-z]+-vs-[a-z]+$"),
}

for name, rx in patterns.items():
    matches = [s for s in df["slug"] if rx.match(s)]
    print(f"{name:10s}  {len(matches):4d} matches  first 5: {matches[:5]}")

You are looking for two things. First, raw counts: any single regex matching more than 80 URLs is a candidate template. Second, parameter overlap: if best-*-for-* matches 200 URLs and the second slot is almost always one of five entity types (“students,” “beginners,” “small businesses,” “freelancers,” “remote teams”), you have confirmation that the template was driven by a small list of variables rather than by editorial judgment.

Step 3 — Sample 5 posts per pattern and inspect the bodies

URL patterns are necessary evidence but not sufficient. The next step is to pull five randomly chosen posts from each high-density bucket and inspect the bodies. We are looking for structural similarity inside the cohort.

import requests, random

def sample_bodies(bucket_urls, n=5):
    sampled = random.sample(bucket_urls, min(n, len(bucket_urls)))
    bodies = []
    for url in sampled:
        slug = url.rstrip("/").split("/")[-1]
        r = requests.get(
            f"https://example.com/wp-json/wp/v2/posts",
            params={"slug": slug, "_fields": "id,slug,title,content"},
            headers={"User-Agent": "Mozilla/5.0"},
            timeout=20,
        )
        bodies.extend(r.json())
    return bodies

For each sampled post, dump the H2 outline. If three or more of the five posts in a single bucket share the same first H2 and the same number of total H2s, you have a structural template, not a coincidence.

Step 4 — Score each cohort against the templated-AI rubric

Now you score each high-density cohort. We use five criteria, each rated 0 to 2.

Criterion	0 (low risk)	1 (medium risk)	2 (high risk)
URL pattern density	< 30 URLs	30–100 URLs	> 100 URLs
Structural similarity	Outlines mostly distinct	First H2 shared in 3/5	Full outline shared in 4/5
Excerpt templating	Varied openings	Same opening verb cluster	Same opening + same closing
Brand-mention frequency	Long-tailed (most 0–1×)	Clustered (most 2–3×)	Every post 2–3× in same slot
Source diversity	Avg ≥ 2 distinct citations	Avg 1 citation	No citations across the cohort

A cohort with a total score of 7 or higher is a high-risk cluster — the classifier will see it. Treat it as a candidate for one of the dispositions in Step 6.

On the case-study site, the best-*-for-* cohort scored 9 (URL density 2, structural similarity 2, excerpt templating 2, brand-mention frequency 2, source diversity 1). The how-to-write-* cohort scored 8. The how-to-use-* cohort scored 8. Together these three cohorts represented 41% of the site and contributed disproportionately to the templated-AI signature.

Step 5 — Build the whitelist of exceptions

Some pages in a high-risk cohort are actually high-value and should not be touched. Before you start dispositioning, identify them. The whitelist criteria are concrete and intentionally narrow:

• High-traffic URLs. Any page that delivered more than 100 organic clicks in the trailing 90 days, per Search Console. These are pages where users have already voted that the content meets the intent.
• Linked URLs. Any page with at least one earned external backlink from a domain outside your own portfolio. Use any backlink tool you trust; the exact number is less important than the threshold of “more than zero earned links.”
• Conversion URLs. Any page known to drive an offline outcome — email signups, downloads, transactions — at a rate above your portfolio median, regardless of traffic.

The whitelist is a safety net. It prevents you from unpublishing the small number of templated-looking pages that, despite the signature, are doing real work.

Step 6 — Apply the decision tree to every non-whitelisted URL

For each URL in a high-risk cohort that is not on the whitelist, walk through this tree:

Is the page indexed?
├── No (Crawled — currently not indexed)
│   └── Has it had any organic clicks in 90 days?
│       ├── Yes -> REWRITE
│       └── No  -> UNPUBLISH or REDIRECT
└── Yes
    └── Are there 3+ other URLs in the cohort covering the same topic?
        ├── Yes -> CONSOLIDATE into a pillar URL
        └── No  -> REWRITE

The four dispositions:

REWRITE means the URL stays, the title stays, the slug stays, and the body is rebuilt by hand. You change the structure (different H2 outline from the template), the sourcing (add at least two named citations), and the voice (no opening template, no closing brand insertion). Rewrites take 60–90 minutes each, which is why you do not rewrite everything. You rewrite the URLs that earned the right to keep their slot.

UNPUBLISH means you set the post to private or delete it outright. We default to unpublishing rather than 410-ing for posts that were not indexed in the first place; the 410 is only worth the effort for URLs that have inbound links or any external referral history.

REDIRECT applies to URLs that are unpublishable but have inbound signal. You 301 to the closest topical match on the surviving site. Avoid redirecting everything to the homepage — that is the move that converts a quality cleanup into a different kind of signal Google reads negatively.

CONSOLIDATE is the move that pays the highest dividend on the cleanup. When you have eight to twelve templated posts in a single cluster covering subtopics of one parent topic, you merge the keepers into a single pillar URL with deep coverage, then redirect the rest. The classifier reads a single deep page as a different object than ten shallow pages, and the pillar usually outranks the templated cohort within one to two re-evaluation cycles.

Step 7 — Log the actions and set the review window

Every disposition you ship goes into the editorial decision log we described in the previous post on the 12-day algorithmic lag. The review window for any cleanup action is at least 14 days out, often 21. Do not interpret the absence of recovery in the first ten days as evidence that the cleanup failed; the next re-evaluation is what will tell you.

The worked example: what 30 minutes of this audit found

On the case-study site, running this exact audit produced the following inventory before any disposition was applied:

• 1,128 total URLs in the export.
• 462 URLs across the top three slug buckets, structurally interchangeable inside each bucket.
• 1,044 of 1,128 URLs sitting in Crawled — currently not indexed.
• 94 URLs whitelisted by traffic, links, or conversion.
• 368 URLs flagged for consolidation into eight pillar pages.
• Approximately 600 URLs flagged for unpublish or rewrite.

The audit itself took 28 minutes once the URL list was exported. The dispositions took several weeks to ship, which is the topic of the final post in this series on the 6-sprint recovery plan.

Frequently asked questions

How do I run this audit on a static-site or headless setup?

The slug bucketing and the regex pass do not depend on a CMS. Replace the WordPress REST call in Step 3 with whatever fetches a post body from your stack — a Git-hosted Markdown file, a Contentful query, a build-time JSON dump. The audit is platform-agnostic; only the body-fetch step is wired to your CMS.

What if my site has more than 10,000 URLs?

Stratify. Run the slug bucketing across the full set, then sample 30 posts per high-density bucket instead of 5. The structural-similarity signal becomes more reliable, not less, when the cohort is large. The 30-minute target stretches to 60–90 minutes for portfolios above 10,000 URLs.

Do I need to do this for every site in my portfolio?

Yes. Templated pipelines tend to run across every site an operator owns because the workflow is shared. If one of your sites has the signature, the others probably do too. Sequencing matters — start with the site that is closest to a demotion or already in one.

How often should I re-run the audit?

Quarterly, at minimum. Monthly if you are still publishing AI-driven content at any meaningful volume. The signature accumulates faster than most operators expect.

What is the single most predictive metric in the audit?

Slug-bucket density. If your top three buckets account for less than 20% of the site, the rest of the signature usually does not exist in a problematic form. If they account for more than 30%, the rest of the signature almost always does. Use slug density as your first-pass triage.

What to do next

Block out 30 minutes this week and run Steps 1 through 4 against your largest site. The output of those four steps is enough to tell you whether you have a programmatic-content risk. If you do, schedule the dispositions in Step 6 using the 6-sprint plan in the final post of this series. If you do not, file the audit and re-run it next quarter — the goal is to keep your portfolio off the classifier’s radar before a demotion makes the work urgent.