SpecSentry

SpecSentry is a web app (with optional CLI) that continuously monitors deep learning models in production for silent failures: data drift, label shift, performance decay, and schema changes. You connect it to your prediction logs and (optionally) ground-truth outcomes; it builds baselines, runs statistical drift tests, and uses lightweight deep-learning embeddings to detect distribution changes in unstructured inputs (text/images). When risk crosses thresholds, it generates actionable incident reports: what changed, which segments are affected, and what to do next (retrain, rollback, add data, adjust thresholds). It also tracks model versions, feature pipelines, and alert history to help teams pass audits. This is an AI app + traditional app combo: AI for embedding-based monitoring and root-cause hints; traditional for observability, alerting, and governance. It’s not glamorous, but it’s the kind of tooling teams pay for once they’ve been burned.