Decoder. plain-English AI glossary

Cluster

● Core

A group of computers connected by a network, working together to run a single large job.

Think of it like

A team of workers instead of one person — they can tackle bigger jobs by dividing work.

Example

A training cluster for a 70B model: 256 GPUs across 64 servers, connected with fast networking.

How it actually works

Clusters let you train or serve models too large for a single machine. The networking between machines becomes a bottleneck — slow networking kills scaling efficiency. This is why distributed training frameworks optimize for communication. Large clusters are expensive (especially the networking) and hard to debug.

For product teams

Cluster cost scales with model size. Know your break-even point.

For engineers

Use distributed training frameworks (PyTorch DDP, DeepSpeed). Profile communication overhead. Invest in fast networking.

How to scale beyond one machine.

Read anything AI without the jargon

Look up any term in plain English, or save terms as you read with the free Chrome extension.

Open Decoder Add to Chrome