Paper-only experiments. Numbers here are generalized; the exact figures come from each bot's own report output, concept diagrams here; any real chart would come from the bot's own report output, never fabricated.
The easiest person to fool with a trading strategy is yourself, after the fact. Pre-registration is how I stopped.
The idea
Before an experiment collects data, I write down the decision rule: the sample size that counts, the metric, and the threshold that means kill or keep. Once that's committed, the data can't be reinterpreted to fit what I wish had happened. A losing run I "decide" to keep because it "should" work is exactly the failure the pre-registered gate prevents.
What a gate looks like
Concretely, for a paper bot I pre-register things like:
- A minimum sample size before the main gate even evaluates (below it, the verdict is INSUFFICIENT, not "looks promising").
- A kill threshold on cumulative PnL, decided in advance.
- For an early-close variant, a separate gate: a minimum count and a confidence-interval lower bound above zero before I'll adopt it.
The point of writing the numbers first is that when a bad day arrives, I'm reading the gate, not negotiating with it.
Decide the rule when nothing is invested in the answer.
The honest part
Every discretionary "let me give it a few more days" is a small edit to the experiment after seeing the result — and those edits are where overfitting to your own hopes lives. Pre-registration doesn't make a bad strategy good; it makes you notice it's bad on schedule, instead of explaining the losses away one day at a time. The discipline is boring and it's the whole point: decide the rule when you have nothing invested in the answer, then obey the rule when you do.