dots.tts

dots.tts is an open-source 2-billion parameter text-to-speech model that introduces a fully continuous, end-to-end autoregressive pipeline for high-fidelity speech synthesis, voice cloning, and instruction-controlled voice editing.

1. For the Non-Technical Reader

Imagine painting a portrait with smooth watercolours instead of placing blocky digital pixels. Most AI voice systems chop human speech into rigid, discrete audio 'chunk' blocks before assembling them. dots.tts removes these digital blocks entirely, maintaining speech as a smooth, continuous sound wave from start to finish. The result is ultra-realistic, expressive voice cloning and natural phrasing without robotic glitches or artificial breaks.

2. For the Technical Reader

• Architecture: End-to-end autoregressive (AR) backbone pairing a semantic encoder, an LLM, and an autoregressive flow-matching acoustic head built over a 48 kHz AudioVAE, completely avoiding discrete tokenization.
• Benchmarks: Achieves top average performance on Seed-TTS-Eval with Word Error Rates (WER) of 0.94% (zh), 1.30% (en), and 6.60% (zh-hard), paired with similarity (SIM) scores of 81.0, 77.1, and 79.5. Highest average speaker similarity (83.9) on the 24-language MiniMax multilingual benchmark.
• Latency & Efficiency: Integrated with SGLang Omni for continuous batching and streaming PCM. Delivers first-chunk latencies down to 68 ms for text-only generation and 204 ms for voice cloning, with a Real-Time Factor (RTF) p50 of 0.13–0.15. Peak throughput reaches 19.86 audios/s at concurrency 16 on a single NVIDIA H100 GPU.
• License & Availability: Open-source under the Apache-2.0 license, featuring fine-tuning code, distilled models (MeanFlow/sCM), and a specialized speech editing variant (dots.tts.edit).

3. Why It Matters

By eliminating discrete codec tokens in favor of continuous flow-matching representations, dots.tts proves that open-source models can match or outperform proprietary closed systems in audio quality, expressiveness, and multi-language fidelity. Its low latency and permissive Apache-2.0 license lower the cost barrier for self-hosted, enterprise-grade conversational AI and double-streaming TTS applications.

4. The "Voice AI Space Lab" Idea

The Real-Time Dynamic Audio Drama Studio: Build an interactive audiobook engine where developers can prompt character voices and perform instant micro-edits in real-time. Using dots.tts.edit, dynamic game masters or creators can adjust a character's tone ("whisper this line anxiously") or swap out mispronounced named entities on the fly without re-generating the entire audio sequence.

Explore the project on the dots.tts GitHub Repository.