Automate Competitor Content Monitoring with n8n
You almost never find out the day a competitor publishes something that will outrank you. You stumble on it three weeks later — already ranking, already earning links — while you were buried in your own backlog. Competitor content monitoring closes that gap: instead of manually checking a dozen rival blogs, you let n8n watch their sitemaps and feeds and ping you the moment anything new ships.
This guide builds a self-hosted n8n workflow that tracks new URLs across any list of competitor domains, deduplicates them so you are alerted only once, and drops a tidy notification into Slack or email. It runs on a schedule, costs nothing beyond your existing n8n instance, and takes about an hour to wire up.
Who this is actually for (and who should skip it)
Let me be direct about the trade-off, because “monitor your competitors” is advice that gets oversold. This workflow earns its keep when you have a short, stable list of direct competitors — three to fifteen domains whose content calendar genuinely influences yours. A niche SaaS blog, an affiliate site in a tight vertical, or a local services company watching two or three rivals: this is the use case. Knowing a competitor just published a “best X for Y” piece within hours lets you refresh your own page, file it for a future update, or decide it is not worth chasing.
It is not for you if you are trying to watch hundreds of domains, need historical archives, or want SERP-position data. At that scale you have outgrown a DIY watcher and should pay for a tool like Ahrefs or Similarweb that already crawls the web for you. Automation is leverage, not a substitute for judgement: an alert that a rival published is only useful if you have the capacity to act on a fraction of them. If every ping just becomes noise in a channel nobody reads, you have built anxiety, not intelligence.
The three ways to detect new competitor content
Before touching n8n, decide how you will detect “new.” There are three practical methods, and the right one depends on what each competitor exposes.
1. RSS / Atom feeds
Most WordPress, Ghost, and Substack sites expose a feed at /feed/, /rss/, or /atom.xml. Feeds are the cleanest signal: structured, ordered newest-first, and carrying titles, publish dates, and often summaries. If a competitor has a working feed, always prefer it. n8n has a native RSS Read node that parses the feed into items with zero custom code.
2. XML sitemap lastmod
When there is no feed, fall back to the XML sitemap (usually /sitemap.xml, or a sitemap index linked from robots.txt). You parse the <loc> and <lastmod> entries and treat any URL you have not seen before as new. Sitemaps are noisier than feeds — they list every page, not just posts, and lastmod dates are frequently unreliable — but almost every serious site has one.
3. Homepage or blog-index HTML diffing
The last resort is fetching the competitor’s blog index and diffing the list of article links against the previous run. This is the most brittle method: it breaks on redesigns and needs a CSS selector per competitor. Use it only for sites that stubbornly lack both a feed and a usable sitemap.
| Method | Reliability | Latency | Setup effort | Best for |
|---|---|---|---|---|
| RSS / Atom feed | High | Minutes to hours | Low (native node) | Blogs on WordPress, Ghost, Substack |
| XML sitemap lastmod | Medium | Hours | Medium (parse XML, filter) | Sites with no feed |
| HTML index diffing | Low | Depends on schedule | High (per-site selector) | Sites lacking both feed and sitemap |
Building the workflow in n8n
Here is the shape of the workflow. Each step maps to one or two nodes.
Trigger and competitor list
Start with a Schedule Trigger set to run every few hours — for content monitoring, every 4 to 6 hours is plenty; nobody needs sub-hourly alerts about a blog post. Follow it with a Set or Code node holding your competitor list as an array of objects, each with a domain, a method (feed or sitemap), and the feed/sitemap URL. Keeping the list in one node makes it trivial to add or drop a competitor later. If you are still deciding which platform to build on, our n8n vs Zapier comparison walks through the cost trade-offs at scale.
Fetch and parse
Split the list with Loop Over Items (Split In Batches) so each competitor is processed independently and one broken feed does not kill the whole run. For feed competitors, use the RSS Read node; for sitemap competitors, use an HTTP Request node to fetch the XML followed by an XML node to convert it to JSON. Normalise both branches to the same output: an array of items with url, title, and published fields.
Deduplicate against state
This is the part most tutorials skip, and it is what decides whether your workflow is useful or spammy. You must remember which URLs you have already alerted on. Two clean options: store seen URLs in n8n static workflow data (or the Data Store), or write them to a Google Sheet or a small database table. On each run, compare freshly fetched URLs against the stored set, keep only the genuinely new ones, then append those to the store. Skip this and you will re-alert every URL on every run — and everyone will mute the channel by Tuesday.
Alert
Pipe the new-URL items into whatever you actually check: a Slack node posting to a #competitor-watch channel, a Send Email node, or a Discord webhook. Format the message with the competitor name, the article title, the URL, and the detected date. Batch multiple new URLs from one run into a single message so a busy publishing day does not flood you.
The edge cases that will actually bite you
A demo workflow works on the happy path. A workflow you trust for months survives these:
- Sitemap index files. Large sites serve a sitemap that only links to other sitemaps. Detect the <sitemapindex> root and fetch the child sitemaps (usually the post sitemap) before parsing URLs.
- Unreliable lastmod. Some CMSs stamp every URL with today’s date on every crawl. Do not trust lastmod as your “new” signal — trust the URL itself against your seen-set.
- Feed truncation. Feeds usually expose only the latest 10 to 20 items. If a competitor publishes 25 posts between runs, you will miss some. Run often enough that this never happens, or back the feed up with the sitemap.
- Rate limiting and blocks. Set a realistic User-Agent, space requests out with the loop, and never hammer a competitor’s server — you are monitoring, not attacking. If a site returns 403, respect it.
- Trailing-slash and UTM noise. Normalise URLs (strip query strings, unify trailing slashes) before comparing, or the same article looks “new” twice.
Turning alerts into an actual advantage
The workflow is the easy 80%. The 20% that matters is what you do with each ping. A competitor publishing a new comparison post is a prompt to check whether your equivalent page needs a refresh; a burst of posts on one theme signals where they are betting their content budget. Feed the alerts into a lightweight weekly triage rather than reacting to each one. If you already run scheduled SEO jobs, this slots neatly beside them — the same discipline that keeps scheduled n8n workflows from overlapping applies here. And if you want change-detection for your own pages instead of competitors’, the same building blocks power an on-page SEO change watchdog.
Frequently asked questions
Do I need to self-host n8n for this?
No. n8n Cloud runs this workflow fine. Self-hosting mainly helps if you are watching many domains or want to avoid execution limits; for a handful of competitors on a 4-hour schedule, either works.
Is reading a competitor’s sitemap or RSS feed allowed?
Reading public RSS feeds and XML sitemaps is standard, low-impact behaviour — that is exactly what those files exist for. Keep request volume tiny, identify yourself with a User-Agent, and respect robots.txt and any 403s. Avoid aggressive HTML scraping of sites that clearly do not want it.
How often should the workflow run?
Every 4 to 6 hours covers almost every content team. Sub-hourly runs add server load and alert fatigue without meaningful benefit, since you rarely need to react to a competitor’s post within minutes.
Can I track keyword rankings this way too?
Not with this workflow — it detects new URLs, not positions. Rank tracking needs a SERP data source, so pair this with a dedicated tracker if you need positions.
What if a competitor has no RSS feed and no sitemap?
Fall back to diffing their blog-index HTML with an HTTP Request node and a CSS selector. It is more fragile and needs per-site tuning, so reserve it for competitors that matter enough to maintain.
