Mean Pooling
The simplest way to summarize a sequence: just average all the token vectors together.
Think of it like
Like taking the average grade of a class to represent how the class did overall.
Example
Many sentence-embedding models mean-pool a sentence’s word vectors to get one vector for search.
How it actually works
You add up every token’s vector and divide by the count, giving equal weight to each. It’s cheap, stable, and surprisingly effective for embeddings, though it can wash out a crucial word by drowning it in the average. When every token matters roughly equally, mean pooling is hard to beat for its simplicity.
For product teams
A dependable default for turning text into one searchable vector.
For engineers
Elementwise averaging of token vectors, often masked to ignore padding.
Related
- Pooling — The general operation it’s a case of.
- Sentence embedding — What it’s commonly used to build.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome