Decoder. plain-English AI glossary

VRAM

● Core

The memory on a GPU, where model weights, activations, and intermediate results live during computation.

Think of it like

The scratch pad a calculator uses to store partial results.

Example

A 7B model in BF16 needs 14GB VRAM. Batch size 32 and intermediate computations might need another 20GB. Total: 34GB VRAM.

How it actually works

VRAM is the bottleneck for most large model serving. You can't fit the model if VRAM < model size + activation memory. This is why quantization and model parallelism exist. Estimating VRAM usage correctly is critical — underestimate and you get OOM errors; overestimate and you buy GPUs you don't need.

For product teams

VRAM cost is your serving cost. Every bit of memory reduction matters.

For engineers

Estimate: params × bytes per param + (batch × seq_len × hidden × factor) for activations. Test before deploying.

Scarce resource.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome