Decoder. plain-English AI glossary

Activation function

● Core

The little on/off-ish dial inside a neuron that lets a network bend, not just draw straight lines.

Think of it like

A dimmer switch, not a light switch — it decides how much of a signal passes through, and “a little” is what makes curves possible.

Example

Swap every ReLU in a network for nothing at all and your fancy deep model collapses into plain linear regression.

How it actually works

After a neuron adds up its weighted inputs, the activation function squashes or gates that number — ReLU zeroes out negatives, sigmoid squishes to 0–1. Without this nonlinearity, stacking layers gains you nothing: a pile of linear steps is still just one linear step.

For product teams

It’s plumbing you’ll never tune, but it’s the reason a model can learn anything shaped like the real world.

For engineers

A nonlinearity (ReLU, GELU, sigmoid) applied elementwise after the affine transform, giving the network capacity to approximate nonlinear functions.

Layer — Stacked into layers to build depth.
Weights — Applied to the weighted sum from weights.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome