ViT
Shorthand for Vision Transformer - a transformer model for image understanding.
Think of it like
BERT is ViTs sibling: BERT processes text, ViT processes images.
Example
ViT-Base trained on ImageNet; can be fine-tuned on medical imaging, COCO, or used as a backbone for other tasks.
How it actually works
Canonical form: ViT splits images into 16x16 patches (patch size P), embeds them (D dimensions), then runs 12-24 transformer blocks. Position encoding is learned (2D). Classification uses a learnable [CLS] token (like BERT). Requires large-scale pre-training; fine-tuning on small datasets underperforms CNNs. But pre-trained on large data (ImageNet-21K, JFT-300M), ViT outperforms CNNs and scales better to larger models.
For product teams
Strong baseline for multimodal models (CLIP combines ViT image encoder with text transformer).
For engineers
Architecture: N patches of D dim, transformer. Training: supervised on ImageNet-21K or self-supervised (MAE, DINO). Metric: ImageNet top-1 accuracy.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome