Causal Language Model
A model that can only look leftward — predicting each word from the words before it, never after.
Think of it like
Like writing a story with a card covering everything you haven’t written yet — you decide the next word from what’s already on the page.
Example
Every generative chatbot is a causal LM: it produces text one token at a time, each based only on what came before.
How it actually works
“Causal” means the attention is masked so a token can attend to earlier positions but not future ones, which is exactly what lets the model generate text left to right. It’s the standard setup for GPT-style models. The constraint is also its power: by never peeking ahead, it can be trained on plain text and then generate coherently.
For product teams
The default shape of any model that writes, chats, or completes text.
For engineers
A model with causal (left-to-right) attention masking, trained on next-token prediction.
Related
- Causal Attention — The masking that enforces it.
- Bidirectional Model — The both-directions opposite.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome