The problem
AI discussions often begin with model selection and evaluation. Enterprise reliability begins earlier: source data, semantic context, lineage, ownership, validation, and how the output is consumed.
Why it was hard
Generative systems can return answers that are fluent and structurally valid while introducing facts that never existed in the source. Once those answers enter a downstream pipeline, ordinary schema checks may accept them.
How we approached it
I frame AI as one component inside a decision system. Controls therefore need to cover retrieval quality, source attribution, output constraints, semantic validation, human review boundaries, and downstream propagation.
- Separate model confidence from evidence confidence.
- Require provenance for consequential claims.
- Validate semantics after generation, not only input format.
- Limit blast radius before learning from production feedback.
- Keep deterministic fallbacks for high-risk paths.
Tradeoffs
More control can reduce speed and flexibility. Less control increases the chance that a plausible error becomes operational truth. The right design depends on reversibility: an answer used for exploration can tolerate more uncertainty than an answer that writes into a system of record.
What stayed with me
AI does not create trust. It amplifies the quality—or weakness—of the data system around it.