AI is sending traffic to pages you don't have

404 isn't a new problem. The user is

A 404 simply means the resource isn't there. The status code exists to answer requests for content that doesn't exist, not to handle typos.

The volume never came from people mistyping URLs either. It came from us. New design, new URL structure, moved content, and suddenly everything old pointed at nothing. Google started weighting this early, and by around 2012 hunting 404s was standard practice in any serious web analytics setup. The most expensive cases were always the same two: a URL sitting in the search results that led nowhere, and an ad you were paying for that landed on an error page.

We've become far better at redirects since then. Not necessarily at the right redirects. A great many sites solve 404s by sending everything to the homepage. That removes the error code and preserves nothing of the relevance.

That held up as long as the traffic came from a crawler following links. Now it comes from something else.

Hallucination is no longer the main explanation

Worth stating plainly, because half the industry still repeats it.

Early models constructed URLs through pure pattern matching. They knew what a website tends to look like, and guessed. It became a large enough problem that providers built dedicated layers around link handling. Modern assistants pull URLs from actual searches and retrieved pages far more often than they compute them from memory. Guessed URLs still occur, but the share has fallen sharply and continues to fall.

What hasn't fallen is everything else.

Four causes that still apply

1. The model remembers correctly, but out of date. This is the important one, and the one nobody discusses. A model has no reliable internal timestamping. It doesn't always know what it learned in 2023 versus 2025. If you launched a new site in 2025, the old structure from 2023 lives on in memory and can get used as though it were current. That isn't hallucination. That's technical debt suddenly receiving traffic again.

2. The model cites something you removed. A campaign page, an old article, a product page you unpublished. It lives on in the model's memory and in third-party citations long after it disappeared from your site.

3. The link breaks in transit. Trailing parentheses, truncated URLs, markdown formatting that doesn't survive being copied out of the chat window. Trivial, but measurable.

4. The model still guesses occasionally. Less often than before, but it happens. Particularly on sites with little retrievable content, where the model has little else to work with.

This is a selection, not a complete list. 404s also arise from broken internal links, faulty canonical URLs, third parties linking incorrectly, expired campaign URLs and a long list of other causes. The point here isn't to cover them all, but to flag the ones that are new.

What this looks like in one site's logs

We ran the numbers on our own site. The table below is a snapshot of the bot log for citationlab.no taken on 10 August 2026, ordered by request count. Every row adds up: 200s plus 404s equals total requests, so these are complete counts per bot, not a sample.

Half of all bot traffic hit a page that doesn't exist. But the distribution is the interesting part. Search engines as a group ran at 16.9% — Bingbot, the largest single source of requests, at 6.6%. The AI crawlers as a group ran at 89.1%.

One row carries that number. CCBot, Common Crawl's crawler, missed on 1,522 of 1,524 requests. Common Crawl is one of the corpora that feeds model training data, and it works through a URL list that is years old — cause number one above, observed directly rather than argued. Take CCBot out and the remaining AI crawlers sit at 39.8%, still more than twice the search engines. The honest reading is that one crawler dominates the headline figure, and that the pattern survives its removal.

The row that matters most commercially is ChatGPT-User: 87 of 183 requests, 47.5%. That isn't a crawler indexing in the background. It's the agent fetching a page because a real person is sitting in the chat waiting for the answer. Nearly half of them got an error page.

This is one site at one point in time, not an industry benchmark. The number to act on is your own.

The volume argument no longer holds

A year ago it was reasonable to dismiss this. AI referrals were marginal for the vast majority of sites, and 404 hits were a fraction of that again.

That changed quickly. We see sites where AI-driven traffic has multiplied several times over in six months, and where it now ranks among the most important organic sources. Not at Google's volume, but in significance.

And even if the volume were still low, the argument would hold. Because the real argument is value per visit.

A user arriving from ChatGPT or Perplexity has already been through the research and evaluation phase, with the model as their adviser. They asked follow-up questions. They received a recommendation. They aren't browsing. They're acting.

That isn't a click from position four in the search results. It's a referral — which is also why ranking and being cited are two different games.

Meeting that referral with "Sorry, this page doesn't exist" is among the most expensive per-visit losses anywhere in your funnel. And unlike a lost organic click, you don't get a second chance. The user doesn't return to the chat to ask for an alternative. They go to the competitor the model mentioned in the same sentence.

The 404 log as content data, and why it has an expiry date

Here's an angle that has been genuinely useful, and that's worth using right now.

When a model repeatedly guessed at a URL you didn't have, it told you something valuable: what it expected a company like yours to offer. If several models independently point toward /pricing, you have a transparency problem. If they point toward a calculator you never built, you have a content gap validated by the very system you're trying to get cited in.

But be honest about the shelf life. The basis for this method disappears as models stop guessing URLs. Use it while it lasts. Don't build a methodology on it.

What replaces it is fan-out. When a user asks one question, the model breaks it into a series of sub-questions it searches on internally before answering. That's where the actual demand sits, and it's far richer data than a handful of guessed URLs. Fan-out tells you which questions you need to answer. The 404 log only told you which URLs someone thought you had.

How to do this in practice

First, check whether you measure the 404 page at all. 404 pages can be measured perfectly well. The problem is that almost nobody does it. Some sites redirect every 404 hit to the homepage, which looks tidy and simultaneously destroys the only data that would show you this is happening. Others have a 404 page served outside the normal template system, so it carries no analytics tagging. Either way the problem doesn't exist in your reports. It only exists in reality. Server logs or Cloudflare analytics catch what GA4 misses.

Identify the AI traffic. Filter on referrals from chatgpt.com, perplexity.ai, gemini.google.com and copilot.microsoft.com. Some assistants also append UTM parameters automatically — ChatGPT uses utm_source=chatgpt.com, which makes tracking easier. Be aware that a significant share of AI referrals arrive with no referrer at all, because the user copied the link manually. Server logs are the truth here, not your analytics tool.

Expect the mix to be lopsided. Across twelve weeks on our own site (14 May to 8 August 2026), GA4 recorded 594 sessions from AI sources. 559 of them — 94% — came from chatgpt.com. Perplexity accounted for six. Claude.ai for one. Planning your measurement around four platforms weighted equally will misread what is effectively one platform plus a long tail.

Expect false positives too. The fourth "AI source" in that same report was not AI traffic at all: a Vercel preview deployment whose hostname happened to contain the string "gemini," swept up by a filter matching on substring. Twenty-eight sessions, misfiled. A hostname filter is a starting point, not an answer — read the source list before you trust the total.

Run a redirect audit against historical URL structures. This is the biggest and fastest win available, and it follows directly from cause number one. Pull a sitemap or crawl from before your last migration and check what actually responds today. Redirect to the nearest equivalent page, not to the homepage.

Turn the 404 page into a navigation point. Search field. Main categories. One clear next step. Not an apology and a link to the homepage. Keep the 404 status code. Don't convert it into a soft 404 returning 200, which creates indexing problems while solving nothing.

Act on patterns, not individual hits. Does a URL keep recurring? If the page existed before, set a 301. If it never existed, consider actually building it. Single hits are noise.

Don't build a page for every guess. That way you create doorway pages and index bloat, trading one problem for a larger one.

Where to start

Pull the last 90 days of 404 hits from your server logs, sort by frequency, and start with the URLs that used to exist — those are redirects you owe yourself, and they follow from the cause that matters most. If you want to see which pages the models think you have before you dig through the logs, run a free AI visibility check, or track it continuously with AI Monitor.