Image Tokenization
Converting an image into discrete tokens - like text tokenization - enabling language models to process images as sequences.
Think of it like
Text is tokenized ("hello" -> [hello, world]); images can be tokenized (image -> [token_1, token_2, ...]).
Example
Codebook-based (VQ-VAE): image -> latent codes, each code maps to a vector in a learned codebook. BPE-based: image -> visual tokens via hierarchical clustering.
How it actually works
Vector quantization (VQ-VAE, VQ-GAN) learns a discrete codebook; images are encoded, quantized to nearest codebook entries, then decoded. Enables autoregressive generation (predict tokens sequentially). Alternative: DALL-E uses BPE-like tokenization on learned visual features. Advantage: unified text-image processing (same transformer). Disadvantage: lossy compression, training overhead.
For product teams
Enables unified text-image models (autoregressive image generation).
For engineers
VQ-VAE: encoder -> codebook lookup -> decoder. Codebook size: 2^k entries (e.g., 8192). Metric: reconstruction quality (LPIPS).
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome