Vocabulary size
Also called Vocabulary Size
How many distinct tokens the model knows — a trade-off between shorter sequences and a bigger, heavier vocabulary.
Think of it like
The number of keys on a keyboard: more keys means fewer keystrokes per word, but a bulkier keyboard.
Example
Bumping the vocabulary from 32k to 128k tokens lets the model encode the same text in fewer tokens, especially in other languages.
How it actually works
A larger vocabulary encodes text in fewer tokens — cheaper and faster context-wise — but inflates the embedding and output layers, adding parameters and memory. Too small and everything fragments into many tokens; too large and rare tokens are undertrained and the model gets heavier. Modern models trend toward larger vocabularies partly to serve many languages more fairly.
For product teams
A dial affecting speed, cost, and how well non-English text is handled.
For engineers
The token count in the vocabulary; larger shortens sequences but enlarges embedding/output layers.
Related
- Tokenizer training — The process that sets it.
- Vocabulary — The set of tokens it counts.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome