Decoder. plain-English AI glossary

Tracing

▲ Rising

Following a single request through all services—inference server calls embedding service which queries vector DB—to find bottlenecks.

Think of it like

Painting a drop of food coloring into a river and tracking where it goes.

Example

A request spans: model loading (50ms) then prompt build (10ms) then inference (200ms) then token streaming (150ms) = 410ms end-to-end.

How it actually works

Logs let you ask "what happened?" Traces let you ask "how did this request get here and why was it slow?" Requires instrumentation in code (OpenTelemetry) and a collector (Jaeger, Datadog). Sampling is critical—tracing every request is expensive; sample high-latency requests more often.

For product teams

Pinpoints exact bottlenecks in production without guessing.

For engineers

Instrumentation in code creates spans; each span tags service, duration, metadata; collector gathers spans into traces; UI shows waterfall chart.

Complements.
Paired with.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome