Experiment Analysis
When to use
Use this skill for experiment analysis when you need to interpret an experiment against its registered goal. The expected result is an experiment readout with a decision.
Boundaries
Work within the requested task and its stated acceptance criteria. Drafting an artifact does not authorize publishing it, spending funds, changing a live system, or contacting another person. Identify any such action separately before taking it.
Inputs
Inspect assignment, sample, primary metric, duration, and data exclusions. Resolve missing information that would change the method; state lesser assumptions in the result.
Method
- Diagnose. Declare entity grain, time window, metric formula, denominator, missing values, and the trusted comparison total.
- Decide. Estimate effect size and uncertainty before recommending action.
- Produce. Build an experiment readout with a decision from the inspected material; keep assumptions distinguishable from observed facts.
Decision rules
- Segment results only where sample size and data quality support it. Test whether the insight survives an alternative cut.
- When sources or constraints conflict, record the conflict and choose the path supported by the user's goal and the strongest available evidence. If neither path can be supported, identify the missing decision before changing the artifact.
Domain rules
- Declare metric grain, denominator, date window, and missing-data handling.
- Reproduce key calculations and reconcile totals to a trusted source.
Verification
Check randomization, guardrails, and segment consistency. Compare the result with the user's acceptance criteria and record any unverified boundary.
Deliver reproducible calculations, a chart or table with units, a decision, and data quality caveats.
Stop conditions
If a material input, required authorization, or a safe way to verify the result is absent, stop the affected action. Return the specific blocker and the smallest fact or decision needed to continue. Do not report an unrun check as passed.
Output
Provide an experiment readout with a decision. Include the decisive evidence and actual verification result. Name any artifact location and unresolved issue that affects its use.
