Does your LLM output filter catch the case where the model just recites your prompt back at you?
I run a bot that generates short promotional hooks for apps and posts them to Threads. It calls claude -p with an app name and description, and expects one line back. At the end of the prompt, I attach a rule:
Rule: do not invent specific predictions/events/numbers not in the data.
Aim for curiosity, 40-80 characters, at most one emoji, one line, no
line breaks. Output only the hook line:The early filter was simple. The model sometimes prefixed its answer with a label like "Hook:" — and since label echoes are short, I assumed length >= 15 characters would filter them out.
cands = [l for l in lines if len(l) >= 15]
line = cands[0] if cands else (lines[0] if lines else "")This filter passed quietly for over a week. I had only imagined the failure mode where the hook came back with a short label attached.
What if it isn't the label that gets echoed?
If the model mistakes the rule itself for the answer and reads it back, is that sentence shorter than 15 characters, or longer? "Write 40 to 80 characters, at most one emoji" is well over 15. It's longer than any label, and longer than most real hooks. A length threshold cannot filter out a failure mode that is, by construction, long.
I found it by accident during a routine check on August 25th. One hook that day read wrong as a sentence — it didn't point at any specific app, and it ended like an instruction, not a pitch. It was the rule, verbatim.
Why the label filter couldn't catch this
Label echoes and rule echoes look like the same category of "junk output," but they fail in opposite shapes.
- Label echo:
"Hook:"— short, always a prefix pattern. - Rule echo: the model mistakes the condition sentence in the prompt for the answer and repeats or paraphrases it — long, a complete, well-formed sentence.
One length threshold cannot catch both failure shapes at once. A filter tuned to catch short junk is powerless against long junk — and if you start suspecting anything long, you also kill real hooks, since the prompt itself demands 40-80 characters.
The fix
The fix wasn't another length threshold. It was pattern-matching the phrases that only ever appear inside the rule sentence itself.
_RULE_ECHO = re.compile(r"\d+~?\d*자|이모지\s*\d*개|조건\s*충족")
cands = [l for l in lines if len(l) >= 15 and not _RULE_ECHO.search(l)]
if not cands:
return ""Number-plus-unit patterns like "40-80 characters" or "one emoji" have no reason to appear in a real app hook. And when every candidate gets filtered out, the function no longer falls back to lines[0] — it returns an empty string. An empty string tells the caller (make_hook) to use the fixed fallback copy that was already sitting there unused. "Always emit some hook" mattered less than "never emit the rule as the hook."
Self-check
Run your own LLM output filter through three questions.
- Is your filter's only axis "short vs. long"? If you have more than one failure shape, you need more than one axis.
- When everything gets filtered out, does it fall back to
lines[0], or to an empty string / a known-safe default? Without a safe default, "the candidate that failed the filter" gets shipped anyway. - Have you ever fed the rule sentence from your own prompt back through the same filter, pretending it was the model's output? That one test would have caught this before it shipped.
The longer the rule you write into a prompt, the more surface area the model has to mistake that rule for a plausible-looking answer. I blocked the period and called it fixed. The hole was untouched. hit the same shape from a different angle — a character filter that stopped one attack string while the actual attack surface stayed open. Three Ways My Own Tests Lied to Me is the same family too: if a test never exercises the actual failure shape, a green run proves nothing.
Take the rule sentence sitting in your own prompt right now and run it through your output filter. If it doesn't get caught, it will eventually get published.