Refusal rate
Also called Refusal Rate
How often a model declines to answer — the dial you tune between too permissive and too preachy.
Think of it like
A smoke alarm’s sensitivity: too low and it misses fires, too high and it screams every time you make toast.
Example
A model that refuses to help with a chemistry homework question because it superficially resembles a dangerous request is showing an over-high refusal rate.
How it actually works
Refusal rate has to be split by intent: high on genuinely harmful asks is good, high on benign asks (over-refusal) is a usability tax that trains people to work around the model. The two failure modes trade off, so teams track both harmful-compliance and benign-refusal. Getting the balance right is one of the subtlest parts of safety tuning.
For product teams
Over-refusal is a real product cost — annoyed users route to a less careful competitor.
For engineers
Report refusal separately on harmful vs. benign eval splits; optimizing overall rate alone hides the over-refusal failure mode.
Related
- Jailbreak resistance — A key measurement of jailbreak resistance.
- Safety evaluation — Its over-cautious cousin is a known failure of safety evaluation.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome