Data curation
Also called Data Curation
The careful work of choosing, cleaning, and organizing training data so a model learns from good material.
Think of it like
A librarian deciding which books belong on the shelf and which get pulled — the collection is only as good as the choices.
Example
Before a pretraining run, a team dedupes documents, strips boilerplate, drops low-quality pages, and balances the sources.
How it actually works
Curation spans deduplication, quality filtering, removing toxic or private content, balancing domains, and deciding what to include at all. It is unglamorous but decisive — many of the biggest quality jumps between model generations came from better data, not bigger models. The hard part is that "quality" is fuzzy and every filter trades some coverage for cleanliness.
For product teams
Often the highest-leverage, least-visible work behind a model that just feels better.
For engineers
The pipeline of selection, cleaning, dedup, and balancing that determines pretraining data quality.
Related
- Data filtering — The filtering step within it.
- Data mixture — The proportions it sets.
- Pretraining corpus — The raw pool it refines.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome