Tensor Cores
Specialized hardware units in GPUs that perform matrix multiplications ultra-fast, the core operation of neural networks.
Think of it like
A carpenter with a power drill instead of a hand drill—same job, orders of magnitude faster.
Example
H100 Tensor Cores deliver 1.5 PetaFLOPS for matrix ops; CPU cannot compete.
How it actually works
Tensor Cores multiply two matrices in a single cycle. Requires data to be in the right format (Tensor Float 32, BFloat16). Not all operations use Tensor Cores—memory bandwidth becomes bottleneck for attention. Modern GPUs have multiple types (TF32, BF16, FP8) for different precisions.
For product teams
Determines inference and training speed; choose hardware with the right Tensor Cores for your model precision.
For engineers
Use libraries (cuBLAS, CUTLASS) that map operations to Tensor Cores; avoid data format conversions that stall pipelines.
Related
- GPU — Found in.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome