Bag of words
Represent text as an unordered collection of word counts, ignoring grammar and word order — simple, fast, and surprisingly effective.
Think of it like
Like describing a book by listing every word that appears and how often, forgetting sentences and structure.
Example
Classify emails as spam or not: extract every word, count occurrences, feed counts to a classifier. "Free money winner you" looks spammy by frequency alone.
How it actually works
Bag of words discards all word order: "dog bites man" and "man bites dog" are identical. This is a huge limitation, but the tradeoff is simplicity and speed. You vectorize text as a histogram of word frequencies or binary presence. With simple methods like Naive Bayes or logistic regression, it works decently for classification tasks. Neural networks and embeddings replaced it because they capture word order and semantics, but BoW remains useful as a baseline and for very large-scale classification where speed matters.
For product teams
A durable baseline that's fast and interpretable; useful when neural models overkill.
For engineers
A text vectorization method where documents become vectors of word counts, losing order and syntax.
Related
- TF-IDF — An improvement by weighting terms.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome