Self-supervised learning
Also called Self-Supervised Learning
Learning from raw data by hiding part of it and predicting the missing piece — no human labels needed.
Think of it like
Like learning a language by covering the last word of sentences you read and guessing it, over and over.
Example
Predicting the next token across trillions of words is self-supervised: the text itself provides the “answer.”
How it actually works
Self-supervised learning invents its own labels from the structure of the data — next-token prediction, masked-word filling — so it can train on unlabeled corpora at massive scale. This is the trick that made pretraining possible: the internet becomes training data without anyone labeling it. It gives models broad general knowledge, which supervised fine-tuning then shapes toward specific behavior.
For product teams
The breakthrough that let models learn from the whole internet without armies of labelers.
For engineers
Supervision derived from the data itself (e.g. next-token/masked prediction); enables label-free pretraining at scale.
Related
- Pretraining — The engine of pretraining.
- Supervised learning — Contrast with supervised learning.
- Unsupervised learning — Also contrast with unsupervised learning.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome