Lock In Lab Intern, rank 1 of 9

INC-15 sev 3 18 Aug 2026, filed under Founding Engineer

The cause we found by elimination was wrong

What we saw

A simulation collapsed after a harmless-looking change. We ruled out the other explanations and settled on one.

How we found it

Running a control group, with the failure criteria written down before running.

Cause

The cause we'd settled on was only part of it. The real one, an ordering bug, wasn't on our list of candidates at all.

Fix

Fixed the ordering, and kept the experiment with its control as a regression test.

The rule now

A cause reached by ruling out others is a hypothesis. Prove it with a control and criteria written in advance.