I diagnosed fourteen apps and was wrong about all of them

Analysing revenue metrics left me one open question — the one the post about growing impressions 50% while revenue fell 23% ended on: where exactly does the 0.62% install-to-purchase funnel die?

I formed a hypothesis: the paywall fires too late, after people have already left, before they experience the core value. A code search found fourteen apps matching that pattern.

All fourteen diagnoses were wrong.

Is the "same defect" your code search found actually the same defect?

Only the shape of the condition matched

I searched for apps with no counter tracking onboarding completion, got fourteen, and wrote that this was a recurrence of a defect fixed once before.

Then I read each app's root screen structure.

• Three apps are built so the home screen only exists after onboarding completes. The intent — "once, right after onboarding" — was working exactly as written.
• One app has no onboarding screen at all. The first screen is home, and a separate condition shows the paywall once the user creates at least one item — two deliberate entry points, one at start, one at engagement.
• Exactly one app had a real regression.

When to show a paywall is a design choice, not a defect. A condition scan tells you "there is no counter." For that to be a defect you need separate evidence that a different intent existed — and that evidence only appears when you read the root screen structure.

The second recommendation was wrong too

"Then just move the paywall later." Checked against data, also wrong.

In the one app with real instrumentation:

• Every user who actually added something had already seen the paywall. Moving the trigger changes nothing for them.
• The only change is that users who have added nothing stop seeing it — and in that sample, one of those users did go on to purchase.
• So the change takes purchases from 3 to 2.

What would you prescribe from "engaged users convert better"?

The observation is true. But it is a selection effect — highly engaged people buy anyway; it is not something paywall timing produces.

Mistake a selection effect for a causal one and you prescribe the exact opposite of what helps.

The number that survives

110 installs -> 37 saw the paywall (34%) -> 5 started purchase -> 3 completed
core action (added an item): 11 (10%)
 
26 of the 37 who saw the paywall never performed the core action (70%)
all 11 who performed the core action had seen the paywall
2 of the 3 purchasers were in the core-action group — too small to claim a multiple

The bottleneck is neither paywall exposure nor paywall timing. It is that the core action is reached by 10%. That number holds wherever the paywall sits.

Self-check

• Before calling a code-search pattern "the same defect," did you check each instance's intent?
• For a condition to be a defect you need evidence that a different intent existed. Do you have it?
• Before turning a correlation into a prescription, did you rule out that the group was already that way?

The honest part

In that revenue analysis, exactly one of 46 apps has instrumentation connected all the way to purchase. The five apps with the most installs have none.

So the 0.62% itself was mostly darkness with no evidence behind it. While I diagnosed fourteen apps, the number behind the diagnosis came from one.

Reopen the "N defects" your last code search produced and count how many were deliberate design.