Optical Flow
Estimating the direction and speed of motion for each pixel - where objects are moving between consecutive frames.
Think of it like
Watching a stream: the water flows in certain directions. You trace how each droplet moves.
Example
Two consecutive video frames: optical flow shows where each part of a moving car or person went.
How it actually works
Classic method: Lucas-Kanade (local patches, assumes constant motion), Horn-Schunck (global smoothness). Modern: deep learning - FlowNet, RAFT train end-to-end on synthetic (FlyingChairs) or real data (Sintel, KITTI). Challenges: occlusion, lighting changes, large displacements, reflective surfaces. Used in video compression, action recognition, video interpolation, and autonomous driving.
For product teams
Enables video understanding, compression, and autonomous driving.
For engineers
Benchmark: Sintel (artistic), KITTI (real-world). Metric: end-point error (EPE), average angular error. Models: RAFT, FlowNet2, PWCNet.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome