Matrix multiplication
The bulk math behind a model — grids of numbers multiplied together, billions of times a second.
Think of it like
A spreadsheet where every cell is a weighted sum of a row and a column, computed all at once.
Example
The reason GPUs run AI is that they’re built to do enormous matrix multiplications in parallel.
How it actually works
A layer’s core operation is multiplying an input matrix by a weight matrix. Almost all of a model’s compute is this one operation, which is why hardware is designed around it. Understand matmul and you understand where the FLOPs go.
For product teams
It’s the workload GPUs are optimized for — the physical reason AI compute costs what it does.
For engineers
The dominant kernel: dense GEMM operations that consume most training and inference FLOPs.
Related
- Dot product — Its building block is the dot product.
- Weights — Applies the weights.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome