batch API
An API endpoint where you upload a file of requests, pay a discount for processing in batch (not real-time), and get results back when they're ready.
Think of it like
Bulk mail—drop off 1000 letters at the post office, get a discount, pick them up tomorrow.
Example
You have 10,000 customer reviews to classify. Upload a JSON lines file to the batch API at 50% cost, queue it, retrieve results in an hour.
How it actually works
Batch APIs trade latency for cost. You don't get results immediately, but you pay 50% less per token. Good for non-urgent work (nightly processing, backfill, evaluation runs). Implementation: upload JSONL → get batch ID → poll status → download results.
For product teams
Cuts costs for large-scale offline inference; enables affordable batch analytics.
For engineers
Async processing; bulk JSONL uploads; usually 50% discount.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome