SchemaSentry

SchemaSentry is a web app (with optional CLI) that monitors schema and data-quality drift across big-data tables and streams in Snowflake, BigQuery, Databricks/Delta, and Kafka. Instead of generic “DQ scores,” it focuses on the failures that actually wake teams up: breaking schema changes, silent type coercions, null explosions, cardinality spikes, and upstream contract violations. It learns baselines per table/partition, then alerts only when changes are likely to break downstream jobs or dashboards. It generates a human-readable incident report: what changed, when it started, impacted models/queries, and the smallest rollback/patch suggestion (e.g., add nullable column, cast fix, backfill window). The MVP targets fast setup: connect sources, pick critical tables, and get drift alerts within an hour. It’s not a full observability suite; it’s a focused “early warning system” for big-data drift.

← Back to idea list