SRMWatch

SRMWatch is a web app that continuously monitors your running A/B tests for sample-ratio mismatch (SRM), bot traffic spikes, broken randomization, and instrumentation drift—issues that quietly invalidate experiments. It connects to common analytics and experimentation sources (e.g., GA4/BigQuery, Segment, Optimizely/LaunchDarkly exports) and runs automated checks on assignment balance, event volume anomalies, and metric definition changes. When it detects likely contamination, it alerts you with a plain-English diagnosis, confidence level, and a short “what to do next” checklist (pause test, re-randomize, exclude traffic, fix event). It also keeps an audit trail so you can prove test integrity to stakeholders. This is not another experimentation platform; it’s a guardrail layer that helps teams trust the results they already produce.