Training cost
The upfront cost to train a model — includes compute, data, and development time.
Think of it like
The cost to build a factory, vs the cost to run it.
Example
Training a 7B model on 1T tokens costs ~$100K in compute; inference on that model costs pennies per user.
How it actually works
Training cost is a one-time or periodic expense. Once a model is trained, you can serve it infinitely. Training is bursty (you use compute for weeks, then nothing for months). Infrastructure choices here are different than inference.
For product teams
Training cost is investment; inference cost is recurring expense.
For engineers
Invest in training efficiency; distributed training, better hardware, and optimized algorithms all matter.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome