Language model
Also called LM
A system that learns the patterns of language well enough to predict and produce text.
Think of it like
Like someone with such a feel for a language that they can finish almost any sentence you start.
Example
Give it “The capital of France is” and a language model completes “Paris” because that continuation is overwhelmingly likely.
How it actually works
At heart a language model assigns probabilities to sequences of tokens, which lets it both score and generate text. Modern ones are large neural networks trained on huge corpora; “LLM” is just a language model scaled up. The framing matters: it predicts likely text, and likely is not the same as true.
For product teams
The base primitive under every chat product; understanding it as a predictor explains most of its quirks.
For engineers
A model of P(token sequence) trained to predict tokens; scaled up, it becomes an LLM.
Related
- LLM — The scaled-up version.
- Next-token prediction — The prediction task at its core.
- Small language model — A compact version.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome