Vision Transformer
A transformer architecture applied to images, treating image patches as tokens - enabling pure attention-based vision.
Think of it like
Like BERT for text, but for images: split image into patches, embed them, apply self-attention.
Example
Split a 224x224 image into 16x16 patches (196 patches), embed each, run transformer. Classification: use [CLS] tokens final hidden state.
How it actually works
Key insight: transformers dont inherently understand locality; they learn it. But they excel at global context and long-range dependencies. ViT requires many images to train (ImageNet alone is insufficient; needs 14M images or pre-training tricks). Patch size trade-off: large patches = fewer tokens (faster), small patches = more tokens (richer). Hybrid (CNN backbone -> patches to transformer) is also common. Recent: efficient variants (DeIT, DeiT-III), multi-scale (Swin Transformer).
For product teams
Versatile backbone for vision tasks; scales to large models (foundation models).
For engineers
Input: image -> patches -> embed -> transformer blocks -> classification head. Metric: ImageNet top-1 accuracy. Models: ViT-B/16, ViT-L/16, Swin.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome