What dependable data quality requires
My approach connects structural validation with semantic checks, reconciliation across independent sources, anomaly detection, lineage, observability, and evidence that can explain why a result should be trusted.
Structural and semantic corruption
Structural failures are visible when fields, types, or required records are missing. Semantic corruption is harder: the record is technically valid, yet its meaning has drifted, its source is stale, or it conflicts with another valid system.
Engineering practices
- Validate meaning as well as format.
- Reconcile independent evidence instead of assuming a single source is authoritative.
- Make data-quality decisions observable and explainable.
- Design recovery paths, not only alerts.
- Measure the quality of the evidence used during incident response.
Related work: public research, engineering case studies, and technical ideas.