CUDA
Nvidia's parallel computing platform that lets you write programs that run on GPUs.
Think of it like
A programming language specifically designed to talk to your GPU.
Example
You write a CUDA kernel that processes 1000 samples in parallel. Each GPU core handles one sample simultaneously.
How it actually works
CUDA is how researchers and engineers squeeze extra performance from GPUs. Writing optimized CUDA is hard (you have to manage thread blocks, memory hierarchies, and synchronization), but it can be 10x faster than unoptimized code. Most ML frameworks (PyTorch, JAX) use CUDA under the hood.
For product teams
Framework choice often determines CUDA optimization. More mature frameworks = better CUDA kernels.
For engineers
Usually you don't write CUDA directly. Use frameworks that have well-optimized kernels. Profile with nvprof.
Related
- Low-level GPU programming.
- What ML frameworks use.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome