Word embedding
A word turned into a list of numbers so that similar words land near each other in space.
Think of it like
Like giving every word a home address in a city where synonyms are neighbors and opposites live across town.
Example
A classic model places “king” and “queen” close together, and the gap from “man” to “woman” mirrors the gap from “king” to “queen.”
How it actually works
Early methods like word2vec and GloVe gave each word one fixed vector learned from co-occurrence. That was a breakthrough, but the flaw is context-blindness: “bank” gets a single vector whether it’s a river or a vault. Transformers replaced them with context-dependent representations, which is why static word embeddings are fading.
For product teams
Foundational history — mostly superseded by contextual models, but the intuition still explains embeddings.
For engineers
Static, context-independent vectors per token learned from co-occurrence statistics; a precursor to contextual representations.
Related
- Embeddings — The general idea of vector meaning.
- Sentence embedding — Whole-sentence version.
- Representation — What replaced fixed vectors.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome