realtime API
An API that ingests audio streams, transcribes, runs LLM reasoning, and outputs audio responses—low latency, no round trips.
Think of it like
A phone call instead of email—talk and get an immediate response, with reasoning happening behind the scenes.
Example
User speaks: 'What's the weather in NYC?' Realtime API transcribes (speech-to-text), asks the LLM (which checks a weather tool), and speaks the response back—whole loop <500ms.
How it actually works
Realtime APIs use WebSocket or similar for true streaming. They interleave transcription, LLM inference, and TTS. Requires high reliability and low-latency infrastructure. Relatively new; OpenAI, Anthropic, and others are rolling these out. Game-changer for voice agents.
For product teams
Enables voice-first experiences; reduces friction for hands-free interaction.
For engineers
WebSocket/streaming architecture; complex latency management; requires efficient inference.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome