Voice AI Space Paris - Quail Voice Focus Model for Speaker Isolation - Theo Fuhrmann @ ai-coustics

Overview of Quail Voice Focus 2.0

Quail Voice Focus 2.0 is a speaker isolation model developed by ai-coustics to prevent background speech and noise from interrupting voice agents. The model acts as a filter to isolate the primary speaker while removing far-field distractions such as background conversations, televisions, and radios.

Model Training Approach

The model's training is guided by two main principles:

Speech-to-text optimization: The training prioritizes transcription accuracy over pristine human audio perception.
Real-world simulation: To teach the model to identify the primary speaker in real time, training data includes "room simulations." This process virtually places a primary speaker close to a microphone while adding secondary speakers, media devices, and noise at various distances with realistic reverberation.

Evaluation and the Dawn Chorus Dataset

Standard open-source evaluation datasets often feature clean audio that does not accurately represent noisy, real-world environments. To address this, ai-coustics created Dawn Chorus, an open-source, semi-synthetic evaluation dataset. The dataset was generated by playing clean speech through an artificial mouth (a speaker mimicking human speech) alongside background speakers and noise. The audio was recorded on phones and computers and compressed through messaging apps to replicate a realistic audio lifecycle.

Performance and Customization

Evaluating the model against commercial speech-to-text providers revealed a significant reduction in word error rates, primarily driven by a decrease in insertions (transcriptions of background noise). However, highly aggressive filtering can lead to increased deletions. To manage this trade-off, the model exposes an adjustable enhancement level parameter, allowing users to customize the level of filtering to suit their specific speech-to-text provider and audio stack.

Integration

Quail Voice Focus 2.0 is integrated with real-time AI pipeline platforms, including LiveKit and Pipecat, and can also be implemented directly into custom stacks using the ai-coustics SDK.