Experiment design for data scientists who set up tests that actually answer questions.
Experiment design sessions cover hypothesis, success metrics, sample size, and potential confounders — all of which belong on a whiteboard where the whole team can debate them. BoardSnap reads the design and turns it into a structured experiment spec before the session ends.
Why data scientists love this workflow
A poorly designed experiment is worse than no experiment — it produces data that looks credible but answers the wrong question. The whiteboard experiment design session is where you prevent that. The team debates the primary metric, the randomization unit, the minimum detectable effect, and the analysis plan.
BoardSnap reads the experiment design — hypothesis, metrics, sample size math, randomization approach, and guardrail metrics — and produces a structured experiment spec. The design is documented and reviewable before the experiment launches.
The exact flow
- Write the hypothesis on the board
State it precisely: 'If we change X, then metric Y will increase by Z because of mechanism M.' Precision here prevents ambiguity later.
- Define primary and guardrail metrics
List the primary success metric and the guardrail metrics that must not be harmed. Debate which matters most.
- Calculate sample size and duration
Write the MDE, power, and significance level. Show the sample size calculation. Mark the minimum runtime.
- Map potential confounders
List factors that could confound the result. Note how the randomization strategy addresses them.
- Snap the experiment design
Open BoardSnap and capture the full design — hypothesis, metrics, sample size, and confounder map.
What you'll get out of it
- The experiment spec is documented before the first line of tracking code
- Hypothesis precision is locked in — no post-hoc redefinition of success
- Guardrail metrics are named and tracked from the start
- Sample size calculations and their assumptions are preserved
- The design is reviewable and shareable with stakeholders before launch
Frequently asked
Can BoardSnap read statistical notation like power, alpha, and MDE?
Yes. Statistical notation is captured as written — 'α = 0.05,' '80% power,' '2% MDE.' The structured output preserves these parameters with their values.
How does documenting the design before launch prevent p-hacking?
A pre-registered design document is your commitment to the analysis plan. If the hypothesis and primary metric are documented before the experiment runs, there's no room to selectively report the metric that happened to move.
Can I use the AI chat to refine the experiment design?
Yes. With BoardSnap Pro, you can ask questions like 'given these parameters, what sample size do we need' or 'what confounders did we identify for this experiment.'
What format should the experiment design whiteboard use?
Any structured format works. Common sections: Hypothesis, Primary Metric, Guardrail Metrics, Randomization Unit, Sample Size, Duration, Analysis Plan, Known Confounders. BoardSnap reads each section from the board.
Data Scientists: try this on your next experiment design.
Three taps. Action items in your hand before the room clears.