The cause we found by elimination was wrong
What we saw
A simulation collapsed after a harmless-looking change. We ruled out the other explanations and settled on one.
How we found it
Running a control group, with the failure criteria written down before running.
Cause
The cause we'd settled on was only part of it. The real one, an ordering bug, wasn't on our list of candidates at all.
Fix
Fixed the ordering, and kept the experiment with its control as a regression test.
The rule now
A cause reached by ruling out others is a hypothesis. Prove it with a control and criteria written in advance.