TestSanity

TestSanity is a web app that audits A/B test plans and results for common statistical and instrumentation failures before teams act on them. You connect data sources (e.g., Segment/GA4/Amplitude export + your warehouse) and upload or define an experiment: hypothesis, primary metric, guardrails, targeting, ramp plan, and stopping rules. The app runs automated checks: sample ratio mismatch, peeking risk, multiple comparisons, metric volatility, bot/internal traffic leakage, event schema drift, and mismatched exposure logging. It then generates a plain-English “Go/No-Go” report with confidence grading and concrete fixes (e.g., “your exposure event fires after conversion; results invalid”). This is not a full experimentation platform; it’s a lightweight QA layer that sits on top of whatever you already use. It’s a combination traditional + AI app: deterministic tests plus an LLM to explain issues and propose remediations.