Hidden state
The model’s running scratchpad — the numbers it carries in its head between input and answer.
Think of it like
Your train of thought mid-sentence: not the words on the page yet, just the shape of what you’re about to say.
Example
When a chatbot “remembers” the start of your paragraph while finishing it, that memory lives in the hidden state, not the visible text.
How it actually works
At each layer, every token gets a vector of numbers — its hidden state — that encodes meaning-so-far. These aren’t words; they’re high-dimensional representations the next layer reads and rewrites. The final one gets turned back into a prediction over the vocabulary.
For product teams
It’s the internal “understanding” you can’t see and can’t directly edit — only shape through inputs.
For engineers
The intermediate activation vector at a given layer and position, dimension d_model, passed forward through the stack.
Related
- Latent space — Lives in a high-dimensional latent space.
- Representation — A kind of representation.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome