Baseten launched the Voice Agent Stack on November 16, infrastructure that runs a voice agent's orchestrator, speech-to-text, LLM and text-to-speech together on dedicated GPUs in the same cloud region. It also launched Baseten Realtime Voice, a reference agent built on the stack that Baseten says costs about $0.01 per audio minute. Both are in private preview, waitlist only.
TL;DR
- Two products: the Voice Agent Stack (infrastructure) and Baseten Realtime Voice (a ready-made agent API on top of it).
- In Baseten's test setup, Realtime Voice is a cascade of open-weight models: Qwen3-ASR for speech-to-text, Gemma-4-31B as the LLM, Qwen3-TTS for speech, plus GLM-5.3 Fast as an optional reasoning model.
- Baseten reports 93% on Full-Duplex-Bench and 73.5% on τ²-Bench Voice, on its own setup.
- Cost claim: about $0.01 per audio minute at 32 fully used sessions.
- Availability: private preview, waitlist only. No public price list.
What is the Baseten Voice Agent Stack?
Baseten is an AI inference provider. The Voice Agent Stack packages what a team needs to run a voice agent on its dedicated infrastructure: runtime optimization for the models, networking between them, per-component autoscaling, cross-cloud failover and post-training support.
The core design choice is co-location. The orchestrator and every model (speech-to-text, LLM, text-to-speech) run in the same cloud region on an in-cluster network. Baseten argues that a remote orchestrator adds hundreds of milliseconds of network round trips per turn, and that running it next to the models removes them.
Teams can swap any model in the pipeline, post-train their own (Baseten's example: tuning speech-to-text to the jargon of a vertical such as healthcare or financial services), or keep some components on third-party APIs while they migrate. For a refresher on how the pieces fit together, see our explainer How Voice Agents Work: STT, LLM, TTS, WebRTC.
How does Baseten Realtime Voice work?
Realtime Voice is the reference agent. It uses what Baseten describes as a custom, Pipecat-based orchestrator. It adds a "talker and thinker" pattern Baseten calls delegation: the fast, co-located model keeps the conversation going while a stronger model (GLM-5.3 Fast on Baseten Model APIs) handles reasoning, web search and tool calls in the background.
It supports client-side and server-side tool calling and MCP servers. The API matches the core OpenAI Realtime protocol, which Baseten says it validated with unnamed design partners. It also offers keyword boosting, configurable acknowledgments and voice cloning, in audio-in/audio-out and text-in/audio-out modes.
What do the benchmarks say?
All figures below come from Baseten's post and its own test setup: Qwen3-ASR and Qwen3-TTS on one RTX-6000 each, Gemma-4-31B on one B200.
Metric |
Baseten's reported result |
Notes |
|---|---|---|
Full-Duplex-Bench (Artificial Analysis split) |
93% weighted (93.01% overall) |
Tests turn-taking, pauses, interruptions, backchanneling |
τ²-Bench Voice, English split |
73.5% pass^1 (93.9% on telecom) |
Self-run on Sierra's harness, not measured by Artificial Analysis |
Median time to first audio |
0.94 s at 1 call, 1.14 s at 56 calls |
Same setup |
Median turn latency |
About 1 s, flat from 1 to 56 sessions |
Context up to 16K tokens |
Cost |
About $0.01 per audio minute |
$18.39/hr list price ÷ 32 fully used sessions |
Source: Baseten, November 16. Baseten itself notes the τ²-Bench score is not directly comparable to Artificial Analysis results.
Baseten says Realtime Voice costs less per minute ($0.012) than every hosted speech-to-speech API it evaluated at 32 or more concurrent streams. That comparison excludes the per-token cost of the GLM-5.3 Fast delegate model. Baseten adds that the comparison is conservative for token-billed providers, because it leaves out the cached tokens they re-bill on every turn. Baseten separately estimates $0.008 per call-minute at 56 sessions with delegation included, based on Baseten's utilization assumptions.
Baseten also says its optimized Whisper, Qwen3-ASR and Qwen3-TTS models lead third-party benchmarks, including Coval's, on final transcript latency and time to first audio.
What's missing
- Access: waitlist only. No date for general availability.
- Price: no public price list. The per-minute figures assume fully used GPUs.
- Independent numbers: the τ²-Bench result is self-reported, and the latency figures come from Baseten's own measurements.
What it means for teams building voice agents
Baseten pitches the stack to teams that want to own their models, post-train them, and pay for GPUs instead of per token. It says the OpenAI Realtime compatibility lets existing agents move over with minimal changes. Its per-minute cost figures assume fully used GPUs at 32 or more concurrent sessions. Turn handling is a big part of the claim; our explainer Turn-Taking in Voice AI, Explained covers why it matters.
Sources
- Baseten, "Introducing the Baseten Voice Agent Stack: own and optimize voice agents end-to-end", November 16: https://www.baseten.co/blog/introducing-the-baseten-voice-agent-stack/
- Baseten docs, Realtime Voice overview: https://docs.baseten.co/inference/audio/realtime-voice/overview
- Baseten, Realtime Voice private preview waitlist: https://www.baseten.co/talk-to-us/realtime-voice-private-preview/



