Deterministic Inference: Batch‑Invariant Mode for Reliable AI

New: turn on Deterministic mode in the Stability Demo to force batch‑invariant behavior. Identical prompts will produce identical outputs under fixed precision and microbatching. Evidence logs a determinism profile.

Why

Reproducibility simplifies debugging, strengthens audits, and stabilizes evaluator scores. Nondeterminism often comes from batch‑dependent kernels and floating‑point subtleties; we expose a mode that minimizes these effects.

How

• Microbatch = 1 on critical paths
• Float32 matmul; fixed KV cache precision
• Prefer deterministic attention/kernels where available
• Evidence: determinism_profile.json with flags, kernels, unique_outputs (N runs), slowdown_factor

Trade‑off

Slightly slower throughput. We route high‑risk and acceptance flows through deterministic mode; fast paths can remain default.

See also: Whitepaper · FAQ · Stability Demo