Hallucination Controls: Runtime Gating, Abstention, and Evidence

Auspexi • Updated:

High‑reliability AI is less about “eliminating” hallucinations and more about controlling their impact at decision time. Our platform implements practical guardrails that prefer a correct abstention over a confident error, with auditable evidence for every change.

TL;DR: Optimize a 3‑state objective (correct, abstain, wrong), gate responses on information sufficiency, verify outputs, and ship decisions with signed evidence and SLOs.

The 3‑State Objective

Objective: maximize expected utility over {correct, abstain, wrong}; abstain carries smaller cost than wrong.
Operating points: choose coverage–precision tradeoffs by risk class and latency budget.

Figure: Abstention + gating yields a tunable operating point with higher precision at the same coverage.

Information‑Sufficiency Gating

Support checks: lightweight measures estimate evidence sufficiency before answering.
Fallbacks: if support is thin, route to retrieval/tools, ask for clarification, or abstain.
Verification: schema/units/range, citation consistency, and simple cross‑checks; fail‑closed on unverifiable outputs.

Shadow Evaluation and SLOs

Every change runs shadow tests under explicit SLOs (utility, stability, privacy, latency).
Breaches trigger rollback; all outcomes are logged in signed evidence bundles.

Grounded Inputs Without Raw Data

Synthetic outputs are calibrated against anchor bundles (aggregates) or ZKP‑protected seeds. No raw row transfer; provenance includes anchor_hash.

Falsifiable Test (Production‑Practical)

Fix latency.
Clamp coverage with rejection sampling.
Measure wrong‑answer and re‑ask rates on a fresh‑news holdout, pre/post gating.

FAQ

Will controls over‑abstain? Tuned per domain; abstain rates monitored via shadow evals and adjusted for useful coverage.
What about domain shift? Stability/drift monitors surface shift; OPs and retrieval sources adapt under change control.
Is this auditable? Yes—decisions, thresholds, and results ship in signed evidence bundles.

For deeper details, see our Whitepaper: Hallucination Controls and FAQ.