How a question gets locked.
The easiest way to find a pattern that isn't there is to decide what counts as success after you've seen the data. Everything on this site is built to make that impossible: the question, the measurement, and the bar for "this counts" are all fixed, audited, and frozen before a single test runs.
The lock arc
Five steps stand between an idea and a verdict:
- 1
Draft
Write the question, the exact thing to measure, and the bar for 'this counts as a yes' , all before looking at the data for this particular test. The bar is a falsification line: state in advance what result would prove the idea wrong.
Guards against: Setting the bar after seeing the data is how you accidentally draw the target around the arrow.
- 2
Audit
A fresh session (with no memory of the drafting) reviews the spec against a fixed checklist, actively hunting for ways the measurement could flatter a result.
Guards against: A second pair of eyes that can't be swayed by what the first one was hoping to find.
- 3
Revise
The drafter absorbs the audit's findings and fixes the spec, tracking every change in the open record.
Guards against: Every change is on the record, so nothing is quietly adjusted out of view.
- 4
Re-audit
Another fresh session checks that the fixes held, and that no new problem slipped in during the revision.
Guards against: Revisions can introduce new holes; a second audit catches them before they matter.
- 5
Lock
The spec is frozen and dated. Only now does the test actually run against the data.
Guards against: After this point the question cannot move to chase the answer.
The rules that bind after the lock
-
The dry-run gate
Before the full test runs, the first three crashes' raw values are printed and eyeballed. If the numbers look pathological (a definition catching the wrong thing) the spec goes back for review. A cheap check that catches artifacts a clean pre-registration alone would miss.
-
A changed question is a new question
If the measurement changes, it becomes a new hypothesis with a new ID, and the old verdict stays on the record, exactly as it landed. You cannot un-lock a result you didn't like and quietly re-run it until it agrees with you.
-
"Inconclusive" is a result
Every test resolves to one of four honest outcomes: supported, refuted, inconclusive, or "needs reshaping." Underpowered and unclear are real, publishable answers here, not gaps to paper over.
None of this makes a one-person study large, and it can't manufacture certainty the data doesn't hold. What it does is narrower and more honest: it guarantees that when a finding is weak, you find out, instead of being told a confident story that was shaped, step by invisible step, to sound better than the evidence.
The full process and the per-test checklist, in the research repo: hypothesis_lock_process.md ↗ · testing-playbook.md ↗