My report only kept the top 10 rows, and I concluded there was no demand

Search Console emailed me to check my traffic. I opened it and saw query data landing for the first time. So I asked the obvious question — am I feeding this into my bots' self-improvement loops?

The answer was "no," and the review concluded "and I shouldn't." That was a 30-minute job. The problem showed up in the 30 minutes after. The data I based that conclusion on had been truncated by a limit I wrote myself.

Before we start, one question. Do you know how many rows the report you make decisions from is allowed to hold? I didn't. And I wrote the tool.

First conclusion: "there is no long tail"

I opened the search report. Over 28 days: 3,255 impressions, 30 clicks. Eight queries had clicks, and six of those were brand-name searches for one specific app. Non-brand clicks: two.

The rest? Impressions, but ranked 50–99. Page five and beyond. That isn't "keywords with demand" — that's Google trying your freshly indexed pages against random queries once each.

So I concluded: feeding this into bot self-learning would train on noise. I had reasons — no sample (30 clicks), and beyond that, shorts and text bots can't move web search rankings, so the feedback loop never closes at all. A loop that doesn't close isn't learning, it's superstition. I'd already cut one loop that was feeding on tiny samples, so that call came fast.

That part was right. The problem was one sentence I added: "the goal is finding long-tail queries, but there is no long tail in the data."

The sentence that should have stopped me

In the same paragraph I also wrote: "only the top 10 queries per site are stored." Two sentences side by side, and I didn't see it for a while.

TOP_N = 10                # top queries/pages kept

I was ruling that "no long tail exists" using data engineered not to contain a long tail. Top 10 per site. On any site with brand searches, the brand fills all ten rows. The long tail begins at row 11 by definition — and row 11 onward was being thrown away.

One line:

TOP_N = 100               # brand queries filled the top 10, hiding the tail

What 100 revealed

Query rows: 112 → 467. And this appeared:

772 impr  picross logic nonogram        pos 37.5
112       pcos diet                     pos 64.2
 82       money anxiety disorder        pos 93.6
 74       hrv breathing                 pos 39.3
 24       nonogram picross solving tips pos  9.7  <- page one, zero clicks

Look at the top row. 772 impressions on a single query — sitting just below the ten rows I kept, in the exact dataset I'd declared demand-free. And the bottom row: position 9.7, meaning page one of Google, with zero clicks. That's not a ranking problem, it's a title-and-snippet problem — the one place where a cheap fix genuinely exists.

It wasn't missing. I wasn't keeping it.

Same day, same error again

Annoyed, I pointed the same question at another metric: what is this number allowed to contain?

The day before, I had wired up the first measurement of whether my bots produce app visits at all and recorded this: "one app accounts for 73% of all sessions. Stop spreading across 34 apps and double down on that one."

So I counted how many apps actually had the analytics tag.

HAS   6
MISS 26

Six out of 32. The 73% wasn't 73% of the portfolio — it was 73% of the six apps I could see. I published that same number in the channel-comparison post that went up the same day, so consider the app-share part of that post corrected here (and the channel split itself was measured against the same partial denominator, so it needs a re-read once instrumentation is complete). The other 26 didn't have zero sessions; they were uncountable. One of those 26 had the most search clicks of any property that month (18) and simply did not exist in the traffic report.

Here's the embarrassing part. Those six weren't an oversight — they were my own earlier decision. Months ago I judged: "only tag the five apps that receive UTM links; tagging all 38 is over-engineering." That was correct then. Meanwhile the bots started linking to far more apps, and the decision outlived the condition that justified it, with nobody watching.

So — what would you do? Trust the dashboard and double down on the winner, or count the instrumentation coverage that produced the number first? I was the former for a full day.

What I changed

• Search collection limit TOP_N 10 → 100. But the screen still shows 10 rows, sorted by clicks (the full set lives in the JSON ledger). Raising a limit and displaying everything are different problems.
• Analytics tags rolled out to 26 apps — code injection, env var, production deploy, 26/26 verified live. App-to-app comparison is finally fair.
• Added campaign and content dimensions to the traffic report, so a single short can be traced to which app and which language it sent visits to.
• And I wrote the gate into the doc: this value does not enter any automatic weighting until a cell has 30+ sessions. Current reading is 1–2 per cell. Fixing a limit doesn't manufacture a sample.

Three-line self-check

1. Where in the code is the row limit of the report you're deciding from? If it takes more than a minute to find, don't use that number as evidence.
2. If you're reading "A accounts for X%," how many things are in the denominator? Unmeasured is not zero — it's unknown.
3. If you once narrowed scope with "this much is enough for now," does the condition that justified it still hold?

The honest part

You'd expect the conclusion to have flipped by now. It didn't.

The long tail exists. But it ranks 37–93, so clicks are still zero, and total clicks over 28 days are 30 — mostly brand navigation. Search data still doesn't get wired into bot self-learning. What changed is the quality of the reasoning. Before it was "because there's no long tail." Now it's "because the long tail is real but sits entirely past page four." The first was my tool's illusion; the second is a fact.

One thing did come out of it: the target got sharper. A 772-impression query at position 37, and a page-one query with zero clicks. Both lived outside the top ten rows, so until today I didn't know they existed.

Catching the same error twice in one day means it isn't a mistake, it's a habit. A dashboard presents its numbers with the face of "this is everything." What it actually shows is "everything I was allowed to keep." I once misread a vendor report's restatement structure for two months — but that was someone else's tool. This was mine, and mine stays invisible longer, because there's no reason to suspect it.

Do one thing right now. Open the report you check most often, then grep its source for limit, LIMIT, or [:10]. I'd like to know what you found.