Speaker Diarization
Identifying and separating who spoke when in an audio recording - marking "Speaker A: [0:00-1:23], Speaker B: [1:24-3:00]."
Think of it like
Reading a script with multiple characters and labeling which character speaks each line.
Example
A meeting recording: the system marks "Alice: 0:00-1:30, Bob: 1:31-2:15, Alice: 2:16-3:00."
How it actually works
Combines voice activity detection (is anyone speaking?), voice embedding extraction (speaker signature per segment), and clustering (group similar speakers). Assumes a speaker has consistent acoustic characteristics. Challenges: similar voices, overlapping speech, channel differences, long recordings (speaker drift). Typically: extract speaker embeddings (x-vector, d-vector), run clustering (spectral, agglomerative). Recent: end-to-end differentiable models (PyAnnote, ECAPA-TDNN).
For product teams
Essential for meeting transcripts, podcast chapters, media indexing, and accessibility.
For engineers
Datasets: AMI, ICSI. Metrics: diarization error rate (DER), composed of miss, false alarm, and speaker confusion. Models: x-vector, ECAPA-TDNN.
Read anything AI without the jargon
Look up any term in plain English, or save terms as you read with the free Chrome extension.
Open DecoderAdd to Chrome