One-hot encoding
Representing a category as a long row of zeros with a single 1 marking which one it is.
Think of it like
Like a row of light switches where exactly one is flipped on to name the choice.
Example
With a 50,000-word vocabulary, the word “cat” becomes a vector of 49,999 zeros and one 1 at the “cat” slot.
How it actually works
It’s the simplest way to hand a discrete symbol to math, but it’s sparse and says nothing about similarity — “cat” and “dog” are as far apart as “cat” and “bulldozer.” That limitation is exactly why embeddings exist: they compress one-hot vectors into dense ones where distance carries meaning.
For product teams
Textbook groundwork; it explains why dense embeddings were such a leap.
For engineers
A sparse binary vector of vocab length with one active index; the input that an embedding layer maps to a dense vector.
Related
- Embeddings — The dense upgrade.
- Vocabulary — The dimension that makes it sparse.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome