Assertion
A logical check that a model's output satisfies some requirement—like 'this summary is under 100 words'.
Think of it like
Like a quality control checklist: pass or fail for each criterion.
Example
Assertion: 'Output contains exactly 3 bullet points.' Model output: '• Point 1\n• Point 2\n• Point 3.' Passes.
How it actually works
Assertions are useful for structural requirements (format, length) but don't measure quality (is the summary good?). You need multiple assertions to cover different aspects. They're fast to compute and good for regression testing.
For product teams
Use assertions for structural requirements; combine with human eval for quality.
For engineers
Implement assertion libraries for common checks (length, format, presence).
Related
- Unit Test for Prompts — Used in.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome