Decoder. plain-English AI glossary

CLS Token

▼ Fading

Also called Classification Token

A special slot added to the front of the input whose final vector is trained to summarize the whole thing.

Think of it like

Like a designated note-taker who sits in on the whole meeting so you can just read their one summary afterward.

Example

BERT prepends a [CLS] token and uses its output vector as the sentence representation for classification.

How it actually works

You prepend a dummy token that carries no word meaning; because attention lets it look at everything, it learns during training to gather a useful summary of the whole sequence into its final vector. It’s the classic trick in encoder models like BERT. Newer decoder-only models often rely on the last token or pooling instead, so it’s fading from the spotlight.

For product teams

A tidy way older models produced one “whole-input” vector; less common now.

For engineers

A prepended token whose final hidden state serves as an aggregate sequence representation.

BERT — The model that made it famous.
Pooling — The broader operation it competes with.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome