Decoder. plain-English AI glossary

Inference

● Core

Using a trained model to actually answer — the running phase, as opposed to the learning phase.

Think of it like

The difference between studying for the exam and sitting it: inference is test day, no more learning.

Example

Every prompt you send to a chatbot is inference; the months of training happened long before you showed up.

How it actually works

Inference is a forward pass on new input with frozen weights — no gradients, no updates. It’s what you pay for per request and what latency and throughput optimizations target. Training is a one-time (huge) cost; inference is the ongoing one.

For product teams

This is your recurring cost line — it scales with usage, unlike training’s one-time hit.

For engineers

Forward-pass evaluation with fixed parameters; the serving/decoding phase, optimized for latency and throughput.

Forward pass — One is a forward pass.
Training — The opposite phase is training.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome