Bidirectional Model
A model that reads each word using context from both its left and its right at the same time.
Think of it like
Like understanding a pun only after you’ve read the whole sentence, not word by word as it arrives.
Example
To fill in “the ___ barked,” a bidirectional model uses “barked” on the right to guess “dog.”
How it actually works
Because it sees the full sentence at once, it can use future words to interpret earlier ones, which is great for understanding but incompatible with left-to-right generation. That’s why bidirectional models like BERT dominate comprehension tasks while generative chatbots use one-directional (causal) attention. The direction of attention is the whole difference.
For product teams
Superb for understanding tasks; structurally unable to generate text on its own.
For engineers
A model with unmasked attention, conditioning each position on the full sequence.
Related
- Causal Language Model — The one-directional counterpart.
- Masked Language Model — The task it’s trained with.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome