LoRA
Also called Low-Rank Adaptation
A cheap way to fine-tune by training tiny add-on matrices instead of the model’s billions of weights.
Think of it like
Like adding sticky notes to a reference book instead of reprinting the whole thing — small, removable, and you can keep several sets.
Example
You adapt a 7B model to legal writing by training a few million LoRA parameters on one GPU, leaving the base weights untouched.
How it actually works
LoRA freezes the original weights and learns small low-rank matrices that adjust each layer’s behavior, cutting trainable parameters by orders of magnitude. This slashes memory and cost, and the adapters are swappable — one base model, many specialties. It usually matches full fine-tuning on narrow tasks, though it can lag on the most demanding adaptations.
For product teams
Why custom models got affordable — train a tiny adapter, keep dozens of “personalities” on one base.
For engineers
Injects trainable low-rank matrices into frozen layers; drastically fewer trainable params, mergeable at inference.
Related
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome