Patch Embedding
Dividing an image into small patches and embedding each patch into a vector - the first step of Vision Transformer.
Think of it like
Breaking a large poster into tiles; each tile is then scanned and digitized.
Example
A 224x224 image with 16x16 patch size = 14x14 = 196 patches. Each patch is a 3x256 vector (16x16x3 = 768 dims flattened, then linearly projected to D).
How it actually works
Linear projection (conv 1x1) or learned embeddings map each patch to embedding dimension. Positional encoding added (learned, absolute, or relative). Trade-off: large patches = fewer tokens (efficient), small patches = more tokens (finer granularity). Patches of 14x14 or 16x16 are common. No CNN inductive bias; transformer learns locality from data.
For product teams
Enables transformer-based vision, scalable to large models.
For engineers
Implementation: reshape, linear projection, add positional embeddings. Typical: 768-2048 D, 196-576 patches.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome