Shipping & Infra6 min read

It was a daily puzzle, and every device got a different answer

Exactly one mode picked its answer on the client, from its own catalog. When the catalog updated the pool changed, and per-device pinning held each of those different answers in place. It wasn't a shareable daily — it was a private puzzle.

#reality-check#verification#methodology#gotchas
Concept diagram: the same date resolving to different answers per device, and the corrected resolution order
A concept diagram summarizing the post.

A music guessing game with one puzzle per day. It has several modes, and most of them read a server-built schedule to decide the day's answer.

Exactly one mode picked its own answer on the client, from a pool of "the 40 most recent releases" in its local catalog.

One question first. Where is your "daily" content decided? If that decision happens inside the device, it isn't a daily.

1. The pool changes every time the catalog updates

Measured against two catalog builds:

two catalog builds one day apart → 13 of the 40 songs in the pool were replaced
five consecutive dates resolved to entirely different answers

So it wasn't a shareable daily — it was a private per-device puzzle. You can't compare your result grid with a friend, and you can't play the puzzle a friend posted. That comparison is the entire reason a daily game exists.

2. A well-intentioned cache made the defect permanent

⚠️ Per-device pinning was holding the problem in place.

It was added so the answer wouldn't change mid-day if the catalog updated. The intent is right. But that is exactly the mechanism that pinned a different answer onto every device.

Without pinning, at least everyone after a given update might have seen the same answer. The cache made the inconsistency stable.

3. Not a porting defect — an original design defect

iOS and Android shared the same design. Nobody wrote one of them wrong by accident.

Which means fixing only one side makes it worse: the two platforms would then produce different answers on purpose. This is the kind of defect that has to be fixed on both sides in the same batch.

4. And the remote pipeline was older than local

I opened the script that builds the schedule. The candidate eligibility filter and the minimum song count (20) existed locally and were entirely absent from the remote copy. The fix had never been pushed. Its consequences were live.

one group: 7 songs · 18 dailies  → each song reused 2.6 times on average
another:   15 songs · 11 dailies / 8 songs · 13 (including 5 future dates)

In group mode the autocomplete narrows to that group, so 7 songs effectively hands over the answer.

⚠️ In a project with multiple copies, "I fixed it" and "it's running" are different facts. Same trap as the crawler that read the hardcoded copy and the deployed page that was a different app.

The test is to download the remote copy through the API and diff it against local HEAD. And there's one more trap inside that: get the path wrong in git show HEAD:<path> and both sides come back empty, reading as "identical". I did exactly that once. Empty compared against empty always matches.

5. How would you order the resolution?

Three sources: the server schedule, a value pinned on the device, and the local pick.

  • (a) device pin → server schedule → local pick
  • (b) server schedule → device pin → local pick
  • (c) server schedule only, delete the rest

(a) is the current behavior: once pinned, nothing the server does matters. (c) makes the game vanish entirely on any date the server didn't schedule.

The answer is (b) — with three rules attached.

6. The resolution order is the rule, and both platforms must share it

server schedule → device pin → local pick
  • The server schedule beats the pin.
  • If the server points at a song that isn't in the catalog (mid-correction), fall through to the next step. Closing the mode because you couldn't find it deletes that day's game.
  • Pin only dates the server did not decide. Pin unconditionally and you hold on to songs that have disappeared.
  • Older catalogs don't have that key: decodeIfPresent on one platform, an empty-map default on the other.

The decision was extracted into a pure function (resolve(...)) so that both platforms hold the same test table. The defect came from a shared design, so the source of truth should be a shared table too.

7. A sibling defect from the same day

Autocomplete candidate ordering differed per platform. One side broke ties by identifier; the other had no tiebreak, so the order followed the catalog's array order.

The list truncates at 8, so the order decides which candidates are visible at all. Reversing the catalog changed 4 of the 8. Pinned to the identifier.

Sorting gets treated as a cosmetic concern — but the moment a list truncates, sorting becomes a filter.

What cannot be undone

  • The dates that already shipped stay as they are. For the past several days, people solved different answers, and that remains true.
  • The instrumentation never counted how many people tried to compare a shared grid and failed. So I can't put a number on what this cost.

Three things to check in your own app

  1. Is the code that decides "today's content" on the client? If so, the answer is a function of device state.
  2. Have you checked what the cache you added for consistency is actually holding? A cache pins wrong values just as well as right ones.
  3. Have you actually diffed the script running remotely against local HEAD? If both files come back empty they look identical. Print the file sizes first.

The honest part

The pinning is the part that stings. I wrote that code to preserve consistency. In practice it was stabilizing an inconsistency.

Well-intentioned code attracts less suspicion. Once you've named something "the thing that keeps the answer from changing", you stop asking what exactly it is keeping from changing.

Do one thing today. Trace the code that decides your app's "content of the day" and check whether it is determined purely by a server response, or whether it reads something off the device. If it reads something, ask whether that value is the same on every device.

Related