AI-Assisted Dev3 min read

I wrote up the trap, then fell into it a month later

I knew the shared auth trigger writes a profile row into every app's schema. Then I used exactly that table as the denominator for a retention cohort.

#reality-check#methodology#analytics#gotchas
Concept diagram: a shared auth trigger fans profile rows into every app schema, making per-app denominators fleet-wide
A diagram summarising the post.

About a month ago I wrote a post about this: several apps share one authentication database, and the signup trigger creates a profile row in every app's schema. Count per-app users from that table and you are always counting fleet-wide signups.

My own note went further: "per-app users and retention are counted only as distinct actors in the activity table."

Yesterday, building a retention cohort, I used that table as the denominator.

Which trap that you documented did you fall into again last month?

The number looked reasonable

signups                957
walked at least once   127
activation             13.3%

87% signed up and never walked. I read that as "onboarding is broken," sliced weekly cohorts, computed D1/D7/D30, and had a table ready to report.

The problem was that nothing looked odd. 13% is a perfectly plausible activation rate and 87% drop-off is an ordinary story. When a wrong value is plausible, you skip the sanity check.

One line arrived from the side

"Don't forget the member DB is shared."

Those 957 were not users of this app. Anyone who signed up in any other app also gets a row in this app's profile table. The denominator was fleet-wide signups.

My "13.3% activation" was a number with no meaning.

The correct denominator was already written down

The same note contained the proxy. This app fills country_code from the device region when it actually runs, and the trigger never touches that column. So accounts with it populated are accounts that really launched the app.

Recounted:

With fan-out Actually ran the app
Accounts 957 428
Walked 1+ 127 127
Activation 13.3% 29.7%

Activation more than doubled. And the real bottleneck turned out to be somewhere else entirely — not between opening the app and walking, but at the login gate in front of it.

A fork in the road

When this happens there are two responses. "Be more careful next time," or "admit that care was never going to hold this line and change the structure."

Which do you pick?

Honestly, I nearly stopped at the first. But if I walked into a trap I had already documented, attention is a defence that has already failed once.

There are structural options: a view that counts per-app users so queries never touch the raw table, or a comment on the table so the next reader sees it immediately. I have done neither yet. This post is the record of not having done it.

Self-check

  • Have you traced the denominator of your key metrics down to the table definition?
  • Are you producing "per-product" numbers from a table several products share?
  • Do you have anything that surfaces a documented trap at query time? If not, that document is only good for explaining things afterwards.

The honest part

This was a mistake tooling could have prevented. The value was plausible, so I skipped verification, and without one line from someone else it would have gone out as a conclusion.

Every app had the same users is the post where I first found this trap. Its author used the same table as a denominator a month later.

Open one card on your dashboard with "users" in the title and read the FROM clause of the query behind it. I opened mine because somebody pointed at it.

Related