Decoder. plain-English AI glossary

GGUF

▲ Rising

A file format for storing quantized models that's lightweight and efficient for CPU and small-device inference.

Think of it like

A portable, self-contained suitcase for neural networks.

Example

Run a 7B-parameter model on your laptop by loading a GGUF file into llama.cpp.

How it actually works

GGUF stores quantized weights in a streamlined binary format with metadata. It's designed to be fast to load and require minimal memory. Llama.cpp and Ollama popularized it by making quantized open-source models accessible on regular hardware.

For product teams

GGUF is how millions of people run LLMs on laptops and phones — it democratizes access.

For engineers

GGUF is the standard for CPU-friendly inference; llama.cpp, Ollama, and many others support it.

llama.cpp — Popular inference engine that uses GGUF.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome