Decoder. plain-English AI glossary

Horizontal scaling

● Core

Adding more machines to spread the load instead of making one machine more powerful.

Think of it like

Opening more checkout lanes instead than upgrading a single lane's speed.

Example

When load grows from 100 to 1000 requests/sec, you add 10x more inference servers instead of upgrading one.

How it actually works

Horizontal scaling works well for stateless services (inference). You just add replicas and load-balance. It doesn't work for databases (replication is complex). Horizontal scaling has overhead — load balancing, coordination — so it's not free, but it's more cost-effective at scale than vertical scaling.

For product teams

Horizontal scaling is how cloud services grow without hitting hard limits.

For engineers

Design services to be stateless so horizontal scaling is simple.

Vertical scaling — Making one machine more powerful.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome