Encoder-Only
A transformer that only reads and understands text — no generation half at all.
Think of it like
Like a book reviewer who deeply understands a manuscript but was never hired to write one.
Example
BERT is the poster child: brilliant at classification and search, incapable of writing a paragraph.
How it actually works
It keeps only the encoder stack, reading the whole input at once (bidirectionally) to build rich representations, and is trained by predicting masked-out words. That makes it strong and cheap for understanding tasks but unable to generate freely. As generative chat models took over, encoder-only fell out of fashion, though it still powers plenty of search and embedding work.
For product teams
A lean, fast choice when you only need to understand or classify, not generate.
For engineers
A transformer using only the encoder stack, trained bidirectionally for representation tasks.
Related
- BERT — The canonical example.
- GPT Architecture — The generation-focused opposite.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome