Mamba
A state space model that adds input-dependent control, rivaling transformers with linear-time scaling.
Think of it like
A thermostat smart enough to change how hard it reacts based on what’s happening right now.
Example
Mamba handles very long sequences at lower cost than attention while staying competitive on language tasks.
How it actually works
Mamba’s key move is making the state space model’s parameters depend on the input — “selective” state spaces — so the model can choose what to remember or ignore per token, something earlier SSMs couldn’t. Combined with a hardware-aware implementation, it achieves linear scaling in sequence length while matching transformers on many tasks. It’s a leading example of the post-attention exploration.
For product teams
A live bet that not every strong model has to be a transformer — especially for long sequences.
For engineers
A selective SSM with input-dependent dynamics and a hardware-aware scan, giving linear-time sequence modeling.
Related
- State Space Model — A selective state space model.
- Transformer — Competes with the transformer.
- RWKV — Another recurrent-style challenger is RWKV.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome