Voice AI Space Paris - How conversation context makes/breaks Voice AI - Laure Piekielko @ pyannoteAI

pyannote.ai is a speaker intelligence platform built on top of the open-source pyannote.audio models, which have been developed over 12 years by co-founder Hervé Bredin. The platform is designed to address the limitations of traditional voice AI models, which are often trained on clean, sequential audio and struggle with real-world conditions like background noise, overlapping speech, and multiple speakers.

Key Capabilities

Rather than replacing Speech-to-Text (STT) models or Large Language Models (LLMs), pyannote.ai adds a layer of speaker intelligence to the voice AI stack. It processes conversational audio to determine:

• Who spoke when and for how long (speaker diarization).
• The presence of interruptions, silences, and overlapping speech.
• Exclusive diarization, where the model identifies and prioritizes the most important speakers.

Platform Demonstration

The technology was demonstrated using the pyannote playground, showcasing its performance on different audio types:

• Clean Audio: In a conversation between two speakers, the model successfully identified the speakers, mapped their speech segments, and provided a time-aligned transcription using the Parakeet model.
• Noisy Audio: In a noisy restaurant environment where speakers were far from the microphone, the model successfully filtered out the noise to distinguish the speakers and identify exactly when they spoke.

Questions and Answers

• Real-Time Diarization: The models currently operate in batch mode, but a streaming diarization feature is actively being developed to provide low-latency, real-time results in the near future.
• Speaker Scaling: The accuracy of the diarization remains consistent regardless of the number of speakers (e.g., two versus five or more). It can even identify speakers in noisy environments that are barely audible to the human ear.
• Multilingual Conversations: Diarization is entirely language-independent because it detects the physical presence of human speech rather than the words spoken. To transcribe multiple languages, users must pair pyannote with a multilingual STT model like Parakeet.
• Voice Prints: If a single speaker switches between different languages, pyannote can still identify them as the same person by utilizing biometric "voice prints" to track their unique vocal signature.