How to Automate Broken Link Detection with n8n: A Self-Hosted Dead-Link Monitor
Broken links are the quiet tax on a site’s SEO. A dead internal link wastes crawl budget and strands link equity; a dead outbound link erodes the trust signals that reviewers and readers use to judge a page. Most teams only find these links when a reader complains or a quarterly crawl surfaces a wall of 404s that has been rotting for months.
If you already run n8n for SEO work, you don’t need a quarterly crawl. You can build a self-hosted workflow that checks your links on a schedule and pings you the moment one breaks. This guide walks through that build end to end — and, just as important, tells you when n8n is the wrong tool and you should reach for a dedicated crawler instead.
The thesis: n8n is a monitor, not a crawler
Here is the honest framing before you invest an afternoon in this. n8n is excellent as a scheduled, self-hosted, alerting monitor that watches a known set of URLs and fires a notification into the stack you already use. It is not a full site crawler. It does not render JavaScript, it does not discover pages by following links three levels deep, and it will get slow and fragile if you point it at tens of thousands of URLs.
So the workflow below is built for the job n8n actually does well: take the URLs you already publish (your sitemap), extract the links inside them, check each link’s HTTP status, and alert on the broken ones — every day, without you touching it.
Who this is for (and who should skip it)
This build is for you if you run a small-to-mid content site (hundreds to a few thousand published URLs), you already have an n8n instance, and you want ongoing alerts rather than a one-off audit report. It pairs especially well with editorial sites where outbound links to sources go stale over time.
Skip it — and use a dedicated crawler like Screaming Frog or Sitebulb — if your site is very large (tens of thousands of URLs), heavily JavaScript-rendered, or if what you actually need is a full technical crawl with rendering, depth, and orphan-page detection. n8n checking 40,000 links on a $5 VPS will time out and frustrate you. Right tool, right job.
How the workflow works
The pipeline is six nodes. Each does one job, which keeps the flow debuggable when a step misbehaves.
| Node | Type | Job |
|---|---|---|
| 1. Schedule | Schedule Trigger | Runs the check daily (e.g. 06:00 in your timezone) |
| 2. Get URLs | HTTP Request | Fetches your sitemap.xml and returns the list of pages |
| 3. Extract links | HTTP Request + HTML Extract | Loads each page and pulls every <a href> |
| 4. Check status | HTTP Request | Sends a HEAD/GET to each link and records the status code |
| 5. Filter broken | Filter / IF | Keeps only 4xx and 5xx responses (and timeouts) |
| 6. Alert | Slack / Email / Sheets | Sends the list of dead links to where you’ll see it |
Build it step by step
1. Trigger on a schedule
Drop in a Schedule Trigger node and set it to run once a day. Broken-link checking is I/O-heavy and slightly rude to the servers you hit, so daily is plenty — you are not monitoring uptime, you are catching rot. If you run several SEO workflows, stagger their cron times so they don’t all fire at once; the mechanics of that (timezones, overlapping runs) are covered in our guide on scheduling n8n SEO workflows with cron.
2. Fetch your URL list from the sitemap
Use an HTTP Request node pointed at https://yoursite.com/sitemap.xml (or the sitemap index, then each child sitemap). Set the response format to handle XML, then use an XML node to convert it to JSON. You now have an array of every URL you publish — the authoritative list of pages worth checking. Using the sitemap instead of crawling means you check exactly what you intend to rank, and nothing you don’t.
3. Extract the links from each page
Loop the page URLs into another HTTP Request node to fetch the HTML, then an HTML Extract node with the CSS selector a and the attribute href, returning an array. Split the output so each link becomes its own item. Two cleanups matter here: resolve relative URLs (/about to the absolute form) and drop mailto:, tel:, and anchor-only # links, which will otherwise pollute your results.
4. Check each link’s HTTP status
Feed the deduplicated link list into an HTTP Request node configured to not throw on error responses (turn on “Continue On Fail” or the equivalent “never error” option) so a 404 returns data instead of stopping the run. Prefer a HEAD request to save bandwidth; fall back to GET for servers that reject HEAD. Set a sane timeout (10 seconds) and a real User-Agent header — many servers block the default n8n agent and would otherwise report false 403s.
5. Filter for the genuinely broken
Add a Filter node that keeps items where the status code is 400 or higher, plus items that timed out. This is where you separate signal from noise. A 301 or 302 is not broken — it redirects — though a long redirect chain is its own problem worth auditing separately. A 999 from LinkedIn or a 403 from Cloudflare is usually bot-blocking, not a dead page, so consider excluding known false-positive hosts.
6. Alert where you’ll actually see it
End with a Slack, Email, or Google Sheets node. For a living monitor, Slack is best: post a message only when the broken list is non-empty, with the source page, the dead URL, and the status code. Appending to a Sheet gives you a historical log you can trend over time. If you want this to feed a wider report, wire it into your existing Search Console weekly digest pipeline so dead links show up alongside your other SEO signals.
n8n vs a dedicated crawler vs a Python script
Choosing the wrong tool is the most common way this project fails. Here is the honest comparison.
| Factor | n8n workflow | Dedicated crawler | Python script |
|---|---|---|---|
| Best for | Scheduled monitoring + alerts | Deep one-off audits | Custom logic, full control |
| Scale ceiling | ~a few thousand links | Hundreds of thousands | Depends on your code |
| JS rendering | No | Yes | Only with headless browser |
| Ongoing alerts | Built in | Manual re-runs | You build it |
| Setup effort | Low (visual) | Low (install app) | High |
| Cost | Free (self-hosted) | Free to paid | Free |
The pattern is clear: use a crawler when you need depth and rendering, a Python script when you need bespoke logic, and n8n when you need a hands-off monitor that lives inside the automation stack you already run.
Common pitfalls (and how to avoid them)
False positives from bot-blocking. Sites like LinkedIn, Amazon, and anything behind aggressive Cloudflare rules will return 403 or 999 to an automated request even though the page is alive. Maintain a small allowlist of hosts to ignore, or treat those codes as “warn” rather than “broken.”
Rate limiting and politeness. Firing hundreds of requests in parallel can get your VPS IP throttled or temporarily banned. Add a small batch size and a short delay between batches. You are checking daily, so there is no prize for finishing in ten seconds.
Soft 404s. Some servers return a 200 status on a page that says “not found.” Status-code checking alone misses these. If they are a real problem for you, add a step that flags 200 responses whose body contains phrases like “page not found,” but keep it conservative to avoid noise.
Timeouts treated as deaths. A slow server is not necessarily a dead one. Re-check timeouts once before alerting, and only escalate a link that fails twice. This single change removes most of the flapping that makes people abandon link monitors.
Frequently asked questions
How often should the workflow run?
Daily is the sweet spot for most content sites. Links do not break on an hourly cadence, and daily keeps the load on both your instance and the target servers reasonable. Very large sites may prefer weekly to stay polite.
Does this replace Screaming Frog?
No. Screaming Frog and similar crawlers do deep, JavaScript-rendered site audits far better than n8n ever will. This workflow does one thing they don’t: it runs unattended on a schedule and alerts you. Use both, the crawler for periodic deep audits and n8n for continuous monitoring.
Will it check outbound links too?
Yes, and that is arguably its biggest value. Internal broken links are usually caught at publish time; outbound links to third-party sources rot silently over months. The workflow checks every <a href> it finds, internal and external alike.
Do I need to self-host n8n for this?
Self-hosting is recommended because broken-link checks burn a lot of workflow executions, which can be costly on n8n Cloud’s metered plans. A self-hosted instance on a small VPS runs this for effectively free.
