API
A web interface that lets users send requests to your model and get responses back.
Think of it like
A restaurant's delivery app — you place an order, they fulfill it.
Example
OpenAI's API: you POST {"model":"gpt-4","messages":[...]} and get back a response.
How it actually works
APIs hide complexity. Users don't care how you're serving (GPU, TPU, quantized, distilled). Your job: take requests, run inference, return results. Challenges: rate limiting, billing, reliability, latency SLOs, and preventing abuse (token limits, content filters, cost caps).
For product teams
APIs are how you monetize models. Design for scalability and robustness.
For engineers
Use async frameworks (FastAPI, Gunicorn). Implement request queuing, caching, monitoring. Plan for scaling.
Related
- How models reach users.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome