Recurrent Neural Network
Also called RNN
A network that reads a sequence one step at a time, carrying a memory of what it has seen so far.
Think of it like
Reading a sentence word by word while keeping a running gist of it in your head.
Example
Before transformers, RNNs handled translation and speech by processing text left to right, updating a hidden state.
How it actually works
An RNN maintains a hidden state updated at each timestep, letting information flow forward through a sequence. That sequential dependency is also its weakness: it can’t parallelize across time, and long-range signals fade or explode through many steps. Gated variants like LSTM and GRU helped, but attention-based transformers ultimately outscaled them and pushed RNNs to the margins.
For product teams
The prior era’s approach to sequence modeling, now mostly of historical and niche interest.
For engineers
A network with a recurrent hidden state updated per timestep, limited by sequential compute and vanishing gradients.
Related
- LSTM — Improved by the LSTM.
- GRU — And the GRU.
- Transformer — Largely replaced by the transformer.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome