State Space Model
Also called SSM
A sequence model that carries a compact running state, mixing RNN-like memory with parallel training.
Think of it like
A thermostat that tracks a single evolving reading over time instead of re-reading the whole day’s log.
Example
For very long sequences, an SSM keeps a fixed-size state so cost grows linearly, not quadratically like attention.
How it actually works
State space models process sequences through a linear dynamical system: a hidden state evolves step to step, summarizing history in fixed size. Cleverly, they can be computed either recurrently (cheap at inference) or as a convolution (parallel at training), sidestepping the RNN’s training bottleneck. Their linear scaling with length makes them attractive for very long contexts where attention gets expensive.
For product teams
A promising challenger to attention for very long inputs, with cheaper scaling.
For engineers
A linear-recurrence sequence model with a fixed-size state, computable as convolution for parallel training.
Related
- Mamba — A prominent example is Mamba.
- Transformer — An alternative to the transformer.
- Recurrent Neural Network — Descends from recurrent ideas.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome