The problem
Traditional validation is good at catching missing fields, invalid formats, and broken schemas. Enterprise decisions fail in a different way: systems disagree about quantity, timing, identity, or state while every record remains structurally valid.
Why it was hard
There was rarely a universal source of truth. One system was authoritative for the transaction, another for current operational state, and another for the business adjustment that had not yet propagated. Choosing one source blindly could make the result less accurate.
How we approached it
We layered technical controls with business-state reconciliation, temporal checks, audit history, and restart-aware recovery. The goal was not merely to label data “good” or “bad,” but to show where disagreement began and which decision it could affect.
- Track technical receipt separately from business completeness.
- Compare balances and relationships across domains.
- Preserve error history rather than overwriting it.
- Make run dates and time windows first-class controls.
- Re-validate before writing after partial recovery.
Tradeoffs
Strict controls protect decisions but can delay useful data. Loose controls improve availability but allow uncertainty to propagate. The practical answer is not one threshold. It is risk-aware handling: block when consequences are high, quarantine when evidence is incomplete, and annotate when uncertainty is acceptable.
What stayed with me
Reconciliation is not a cleanup task after integration. It is part of the architecture of any system that turns multiple sources into a business decision.