Decoder. plain-English AI glossary

Serverless Inference

▲ Rising

Running inference without managing servers—you pay per request and the infrastructure scales automatically.

Think of it like

Calling a taxi instead of owning a car; you only pay when you use it.

Example

AWS Lambda powered by provisioned throughput to serve a model on-demand without maintaining EC2 instances.

How it actually works

Request handler spins up, inference completes, handler shuts down. Cold starts introduce latency. Pricing is request-based, not reservation-based. Best for bursty, low-volume workloads or strict SLAs that do not require always-on availability.

For product teams

Eliminates infrastructure management overhead and scales to zero cost when idle, but adds latency for cold starts.

For engineers

Model loaded into ephemeral function runtime; inference executes within request window, then container is deallocated.

Inference — Trades off vs.
Containerization — Complements.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome