Mixture model
A model that combines several specialized sub-models, blending their outputs into one answer.
Think of it like
Like a panel of specialists where each weighs in and their votes are averaged into a diagnosis.
Example
A system routes a coding question to its code-savvy component and a poem request to another, mixing their strengths.
How it actually works
A mixture model composes multiple components, often weighting them by how relevant each is to the input. It’s the statistical idea behind mixture-of-experts architectures, where a router activates only the most relevant sub-networks per token — capturing diverse patterns without one monolithic model doing everything.
For product teams
The concept behind “big model, but only part of it runs per query,” which keeps costs sane at scale.
For engineers
A model combining weighted sub-models; the basis for mixture-of-experts routing over specialized components.
Related
- Ensemble — A group of models voting.
- Dense model — Its dense counterpart.
- Sparse model — The sparse version it enables.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome