Decoder. plain-English AI glossary

Dense Transformer

● Core

The classic transformer where every token passes through every parameter — no experts, no skipping.

Think of it like

Like a restaurant where every dish goes through the entire kitchen staff, not a specialized station.

Example

Most well-known models before the mixture-of-experts wave were dense transformers.

How it actually works

In a dense model, all weights are active for every token, so compute per token scales directly with model size. It’s simpler to train and reason about than sparse mixtures, with no routing to get wrong, but it can’t grow its knowledge without paying full price on every token. It’s the baseline against which sparse designs are measured.

For product teams

The straightforward, predictable model shape — capability and cost rise together.

For engineers

A transformer with all parameters active per token; contrast with sparse MoE.

Sparse Mixture — The sparse alternative.
Transformer — The architecture it’s a form of.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome