Ten of My Apps Were Serving 500s to Crawlers — You Cannot See This in a Browser

While checking on my web apps I typed a path that doesn't exist, out of habit. /zh/. That app supports Korean, English and Japanese only.

I expected a 404. I got a 500.

On your site, does a path that doesn't exist return 404 or 500? I had never once checked.

So I hit all 43 hosts

I needed to know whether it was one app or many, so I pulled the host list and swept every one.

500: zodiac
500: ddi
500: mindage
500: bloodtype
500: biorhythm
500: lovelang
500: lifeweeks
500: animalface

Plus two more outside that list. Ten in total.

The worst was a Japanese-only app. It supports ja and zh, which means /ko/ and /en/ were the 500s — the two paths a Korean visitor would try first on that site. I only ever open it at /ja/, so I was never going to see it.

Why 500 and not 404

In the Next.js App Router, app/[locale]/layout.tsx looks like this:

export function generateStaticParams() {
  return LOCALES.map((locale) => ({ locale }));
}
 
export default async function LocaleLayout({ params }) {
  const { locale } = await params;
  if (!isLocale(locale)) notFound();   // <- never gets this far
  ...
}

You call notFound(), so you expect a 404. But before that runs, the router tries to render the unlisted param dynamically. It isn't a static path from build time, so it attempts a runtime render, and when that throws you get a 500.

One line fixes it:

// Unsupported locales (/zh etc.) get a clean 404 at the route layer, not a 500.
export const dynamicParams = false;

Now any param generateStaticParams didn't emit is never rendered at all — it's just a 404.

The deflating part: 33 of my apps already had this line. I swept this exact problem once before, and the apps missed in that sweep stayed missed. Same root as the dead /ja links I traced earlier — except that time I was looking at 404s, and this time at 500s.

A 500 is worse than a 404

In a browser both are "the page didn't load," so they feel identical. To a crawler they are nothing alike.

A 404 is a definitive "this URL does not exist." The crawler drops it and moves on. A 500 is "the server is broken." The crawler treats the site as unstable, burns crawl budget retrying, and can throttle its crawl rate outright.

So a page that doesn't exist eats into the indexing of pages that do. When distribution is already your bottleneck, that's pure loss.

Here's where I got it wrong — writing it down as it happened

I added the line to eleven apps, got all of them building, and ran the deploy script. Eight printed OK.

Then I re-checked live. Still 500.

if [ "${1:-}" = "prod" ] || [ "${1:-}" = "--prod" ]; then
  vercel --prod --yes

I ran deploy.sh with no argument. No argument means a preview deployment. Production never changed — and those eight runs consumed the free plan's 100-deploys-per-day limit, so nothing else ships today.

Writing it down because it's the lesson. "OK" means the deploy succeeded, not that anything changed. The other half of today's work was the same story in a different costume: exit 0 is not a result. Had I not re-hit the live URLs, I would have reported "fixed ten apps."

Three things to check

1. Have you actually hit a nonexistent path? One line: curl -o /dev/null -w "%{http_code}" https://your.site/nonexistent-locale/. If it's a 500, you're leaking crawl budget right now.
2. When you "swept" a fix, was it exhaustive or a sample? I believed I had swept this and ten apps were still broken. Enumerating a list and hitting each one is not the same as opening a few.
3. Does your deploy script default to production? If the default is preview and you forget the flag, the script succeeds and your users keep seeing the old build.

The honest part

I don't think this brings in traffic. The real bottleneck for these apps isn't locale routing — it's that few people arrive at all, and one 500 doesn't move that.

But this is the stop-the-bleeding kind of work. It doesn't grow the index; it stops something that was eating it. And the cost was one line per app, so there was no reason not to.

What actually stings is how I found it. These apps are monitored daily, deployed regularly, and I've run design audits across all of them — and every one of those checks only looked at paths I visit. Nobody was looking at the paths crawlers walk.

Do one thing right now: curl a subpath of your own site that doesn't exist. A 404 means you're fine. A 500 means you just found today's work.