The Cost of Reliable AI

For each case the agent emits a raw confidence and a verdict; a calibration map turns the raw confidence into c ≈ P(correct). With an escalation threshold τ, the agent answers autonomously when c ≥ τ and escalates to a human otherwise. Writing φ(τ) for coverage (autonomy rate), α(τ)=E[c | c≥τ] for selective accuracy, and a_H for human accuracy on escalated cases:

System reliability   R(τ) = φ(τ)·α(τ) + (1 − φ(τ))·a_H
Cost-optimal gate    τ*   = min { τ : R(τ) ≥ Y }
Cost per case        c_unit = k_m + k_h · escalation(τ*)

Model cost k_m is paid on every case; human cost k_h only on escalated cases. Because the system is more reliable when borderline cases go to the (more accurate) human, raising τ raises reliability, so we pick the smallest threshold that still clears the target Y, which is also the cheapest. Two boundaries frame the curve:

• Floor. If base accuracy already clears Y, no escalation is needed and c_unit = k_m.
• Ceiling. You can never exceed R = a_H, to promise 98% reliability, the human review process must itself clear 98%.

What the curve shows

Each arm's model cost is fixed, while its measured review fraction changes with the reliability target. Raising the human-review cost therefore leaves the reliability points unchanged and reprices the oversight required at each point. This makes the framework's thesis directly measurable: as agent quality improves, the required human-oversight cost declines.

Where calibration and the escalate class enter

Two error types drive the cost. Under-escalation, the model confidently answers a case it gets wrong, or one a human expert says it should have flagged, is a reliability failure that forces τ* higher and raises cost everywhere. Over-escalation, deferring cases it would have gotten right, is wasted human cost. Good calibration collapses both: genuinely insufficient-evidence cases (the Escalate / Uncertain class) naturally receive low confidence and route to a human through the same gate. Calibration error therefore acts as a safety margin, you must set τ above its nominal value to stay robust, and that margin has a price.