Decoder. plain-English AI glossary

FP16

● Core

Storing numbers as 16-bit floating-point values, the middle ground between float32 and int8.

Think of it like

A reasonable compromise — smaller than float32 but safer than integer quantization.

Example

Weight: 0.847385 (float32, 4 bytes) → 0.8474 (float16, 2 bytes).

How it actually works

FP16 is 2x compression with minimal accuracy loss for most tasks. Modern GPUs (Nvidia A100, H100) are optimized for FP16. Training in FP16 is standard; serving in FP16 is even faster. The downside: FP16 has smaller dynamic range, so some models with very large or very small values suffer.

For product teams

Best balance of compression and safety for most models.

For engineers

Train and serve in FP16 by default. Use FP32 only if you see numerical issues.

Sweet spot for precision.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome