AWQ
A newer quantization method that quantizes less aggressively on activations that matter most, beating GPTQ on speed.
Think of it like
Like GPTQ's smarter cousin — spends effort protecting the parts that do the most work.
Example
AWQ-quantized models run faster than GPTQ at the same quantization bit-width because the runtime is more efficient.
How it actually works
AWQ observes that some activations contribute more to the output than others (high variance). It protects those by scaling them up before quantization, then scales back down. It's also designed to be faster to run (not just faster to quantize). AWQ trades off against GPTQ: sometimes GPTQ is more accurate, sometimes AWQ is faster.
For product teams
AWQ is gaining adoption for inference optimization — speeds up inference compared to GPTQ.
For engineers
AWQ runtimes are optimized for speed; check if your inference engine supports it.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome