Skip Connection
The general name for wiring that lets a signal jump over one or more layers instead of passing through them.
Think of it like
An express lane that skips a few local stops so passengers can bypass congestion.
Example
U-Nets and transformers alike use skip connections so early features reach later layers untouched.
How it actually works
Skip connection is the umbrella term for any path that routes information around intermediate layers; the residual connection — adding input to output — is its most common form. These shortcuts preserve detail that deep stacks would otherwise blur and keep gradients from vanishing. They show up across many architectures, not just transformers.
For product teams
A foundational design idea that keeps deep models both trainable and detail-preserving.
For engineers
Any bypass path routing activations around layers, most often an additive residual.
Related
- Residual Connection — The residual add is one kind.
- Deep learning — A staple of deep learning.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome