Degrade visibly.
The worst failure mode is a confident wrong answer; the second worst is a system that fails silently.
Why it matters
There are three failure modes for an AI system: silent (it does the wrong thing without telling you), confident‑wrong (it tells you the wrong thing with conviction), and visibly degraded (it tells you exactly what it couldn’t do). The first two are how AI loses trust. The third is how AI keeps trust.
The cost of not doing it
Your AI is asked a complex question that requires synthesising two facts. It only finds one of them. Rather than say “I found A but not B, here’s A only,” it confidently answers based on A alone. The user assumes both A and B were considered. They make a decision based on incomplete information.
How BRAG does it
BRAG has four explicit outcome states, each with its own degradation handling: Normal (full answer, all evidence covered), Partial (answer is incomplete; the disclosed gap explains what’s missing), Clarification Request (the question is ambiguous; the system asks for more), and Hard Halt (the question is out of scope; the system refuses). Every answer is annotated with its degradation level. Every disclosed gap, contradiction, and refutation is surfaced in the response itself. The reader sees not just the answer, but the quality of the answer.
Apply it in your own system
Outcome states don’t need to be the same as BRAG’s. They just need to be explicit.
- Define your outcome states explicitly. (3–5 is usually enough.)
- Surface them in the UI, not just in the logs.
- Train your users to read them. A small “I’m only ~70% confident in this” tag is more useful than a confident‑looking long answer.
- Track the distribution of outcomes over time. A system that’s always “Normal” is probably hiding partial failures.
This is one of six principles abstracted from BRAG — a retrieval‑augmented generation system with two LLM skeptics gating every confident answer. Tested on ten years of Netflix’s public financial reporting.