Every reconnect rolled the billing clock backwards

I sell time. Users buy call minutes; the server deducts what they use. In that shape, the scariest failure is cost going out without billing going in — because the leaking side is the quiet one.

While re-measuring unit cost, I put the provider's usage numbers next to my own database. One session stood out.

• Started 13:32:33, ended 13:56:25 — 23.9 minutes of wall time
• Granted: 10 minutes
• Billed: 4.3 minutes
• Last heartbeat 13:56:21 — alive to the end
• Status: ended normally

A ten-minute grant, twenty-four minutes used, four minutes charged. Where would you look first?

The rewind was one line

The settlement function looked like this:

const elapsed = Math.floor((now - startedAt) / 1000);
const ceiling = Math.min(Math.max(elapsed, 0), granted);
return Math.max(Math.min(Math.floor(reported), ceiling), 0);

Take what the client reported, cap it by elapsed time and by the grant. It looks reasonable. Its own comment said it was "a function that caps a cooperative client's report," and it did exactly that.

What was missing was a floor.

The server has a self-recovery path: if the app drops and comes back, it resumes the live session. But the app restarts its own counter at zero on reconnect. That lower number was written straight through, and time already billed stopped existing.

Token mints — reconnects — tracked the loss precisely.

Summed: 48.1 minutes. Actual billed time over the same period was 49.9 minutes — we were failing to charge for roughly half the time we sold.

The second hole was already in a comment

Looking at the same data I found another one. Sessions whose app vanished without sending an end signal get swept by the server periodically — and that sweep changed status without touching the ledger.

The comment read: "A reaper that settles per-user properly is a pre-launch task."

An honest note. But the task stayed open, and it leaked the whole time: 13 sessions, 975 seconds.

How it was fixed

Instead of patching line by line, I moved settlement into a single database function. It locks the session row and does the ledger increment, the wallet deduction, and the settled flag in one transaction.

That bought something unexpected. Because settlement now runs behind a row lock, by the time the function returns, whoever won the race has already committed. Previously there was defensive code that carried a locally computed value around "in case we read before the commit" — and removing that defence removed a place where new bugs kept appearing.

Three things to check

If you bill by usage:

1. Is your settled value monotonic? Capping a client report with min() alone lets one reconnect walk it backwards.
2. Does your cleanup path settle? A sweeper that only changes status is a permanent leak.
3. Can a failed settlement be found again? If a state combination matches no query, that money is gone for good.

The honest part

Tests did not catch this. There were 39 pure-function tests, all green. What caught it was putting the provider's usage next to my own database — two numbers counting the same event differently.

Fixing it took twelve rounds of code review, and each round found something new. A large share of those were introduced by my own fix from the round before. That deserves its own post.

Go find the place in your billing code that trusts a client-reported number, and ask whether that number can ever be smaller than the one before it.