Decoder. plain-English AI glossary

Time to first token

● Core

How long between sending a request and receiving the first output token.

Think of it like

The delay between asking a question and hearing the first word of the answer.

Example

You send a request to a language model. After 500ms, you get the first token. TTFT = 500ms.

How it actually works

TTFT is critical for user experience. Low TTFT feels responsive; high TTFT feels frozen. It's dominated by prompt encoding time (especially for long contexts), not by generation speed. This is why systems batch prompts separately from generation — you pay encoding cost once per unique prompt, generation cost per output token.

For product teams

TTFT is king for interactive systems. Users notice even 100ms delays.

For engineers

Profile with realistic workloads. Optimize prompt encoding (KV cache reuse, sparse attention). Batch aggressively.

Key latency metric.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome