Transformer
The architecture behind nearly every modern language model — it reads a whole sequence at once using attention.
Think of it like
A reader who can glance at every word on the page simultaneously instead of one at a time.
Example
GPT, Claude, and BERT are all transformers under the hood, stacking attention layers to process text.
How it actually works
Introduced in 2017, the transformer ditched the step-by-step recurrence of earlier models for attention, which lets every position look at every other position in parallel. That parallelism is why transformers scale so well on modern hardware. Stack enough of these layers, train on enough text, and you get today’s large language models.
For product teams
The foundational design that made large language models practical to train at scale — the “T” in GPT.
For engineers
An attention-based architecture that processes sequences in parallel, stacking self-attention and feed-forward layers.
Related
- Attention — Its core mechanism.
- Recurrent Neural Network — What it replaced.
- Encoder-Decoder — Built from encoder and decoder blocks.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome