Boson AI has announced Higgs Realtime Duplex, a full-duplex model that listens, watches, talks and renders a face of its own at the same time. The company published it on November 5 as the next generation of its Higgs Realtime speech-to-speech model. The API is not open yet: there is a waitlist.

  • What: a 45B mixture-of-experts (MoE) "front model" that takes in audio, video and text continuously and outputs voice and a real-time face.

  • Timing: all streams are cut into 160 ms chunks, in and out.

  • Training: about 2 trillion tokens of text, audio and video, on top of a text pre-trained model.

  • Background work: it can think, call tools and hand tasks to a larger backend model without pausing the conversation.

  • Availability: API "coming soon" with a waitlist. No price, no open weights announced.

What is Higgs Realtime Duplex?

It is a single model that handles a live conversation over voice and video. It watches the user's camera or shared screen, speaks, and shows a generated face synced to its voice. Boson says it decides when to speak on its own: it backchannels with "mm-hm", can answer before the user finishes, stops when interrupted, and stays quiet when a remark is meant for someone else.

The launch page shows the model in a team meeting, presenting an architecture and taking questions, plus three shorter demos: a spoken-English tutor that watches the learner's mouth, a drive-thru where the model ignores the order the driver gives the cashier, and a hands-free engineering session where it asks Claude Code to build a tool in the background.

Boson calls it its first full-duplex "interactive omni" model. The existing Higgs Realtime API, which our Week 31 roundup covered, is already documented as full-duplex for voice. The new part is continuous video in, a face out, and reasoning that runs alongside the conversation.

How does Higgs Realtime Duplex work?

Every 160 ms, the model reads the chunk that just arrived (user voice, camera frames, tool results, backend replies) and writes the next chunk out (its voice, its face, its background work). Silence is processed as chunks too. For a primer on why this differs from turn-based pipelines, see our explainers on how speech-to-speech models work and turn-taking in voice AI.

Three mechanisms raise answer quality without stopping the conversation:

  • Thinking: the model starts reasoning while the user is still talking, and keeps that thread going across chunks while it listens and speaks.

  • Tool calls: search, knowledge bases, booking, CRM lookups or custom functions, called asynchronously. The model can start answering from partial results. (Our explainer on how voice agents use tools mid-call covers the general pattern.)

  • Delegation: harder tasks go to a larger backend model with the shared context. Boson says this works best when the backend is a frontier model.

Boson positions this against Thinking Machines Lab, whose interaction models also pair a real-time model with an asynchronous background model. Boson's description is that the Thinking Machines front model leaves thinking and tool calls to the background model, while Higgs Realtime Duplex thinks and calls tools itself and saves the backend for turns that need it. Thinking Machines' own post says its interaction model can search, browse the web or generate UI while it speaks and listens.

Token budget. Boson describes a tight budget to keep long sessions cheap: 6.25 audio tokens per second in, 12.5 out, one camera frame every six chunks, and two image latents per chunk for the face. That comes to about 2.6K tokens per minute for audio and video, or roughly 2.6K to 5K per minute with thinking and tool calls. The model usually keeps the last five minutes of streams (about 13K to 25K tokens) plus older history kept as text.

Voice and face. Default voices and faces are not built into the model. A short reference audio clip sets the voice and a single reference image sets the face, at deployment time, with no fine-tuning. A causal flow-matching decoder renders speech; a causal diffusion transformer (DiT) renders the face from the reference image and the two most recent latents.

Training. Boson continued training a text pre-trained model on about 2 trillion tokens of text, audio and visual data, on its own NVIDIA H100 cluster in a data center it built. Post-training uses multi-teacher on-policy distillation: specialist teachers trained with reinforcement learning (including GRPO) on conversation, interaction, delegation and task completion, then distilled into one model. The team says it tried larger front models, which scored higher but cost more to run.

How does it compare?

Boson compared the model with Thinking Machines Lab's TML-Interaction-Small and with OpenAI's and Google's realtime and live models. A selection of the reported scores:

Benchmark

Higgs Realtime Duplex (45B MoE)

TML-Interaction-Small (276B MoE)

Best GPT config listed

Best Gemini config listed

Big Bench Audio (accuracy)

98.9

96.5

97 (RT-2 High)

98 (3.8 Live Ext. Thinking High)

Full-Duplex-Bench v1.5 (avg)

75.5

77.8

80.1 (GPT-Live-1)

54.3 (3.1 Flash Live Minimal)

τ-Voice (task completion, pass@1 %)

69.8

not run

74.5 (GPT-Live-1, Astra medium)

68.6 (3.8 Live Ext. Thinking High)

IFEval (accuracy)

94.1

89.7

95.2 (RT-2 xHigh)

90.0 (3.1 Flash Live High)

All figures from Boson AI's launch post, November 5. Boson says other models' scores come from published reports and leaderboards (Thinking Machines, Artificial Analysis, Scale AudioMC, OpenAI) and that it reproduced more than half of them. The Higgs Realtime Duplex scores are Boson's own runs.

The results are mixed rather than one-sided. Higgs Realtime Duplex posts the top Big Bench Audio score in the table and beats TML-Interaction-Small on four of the six benchmarks where both were run in Boson's full table, with roughly one-sixth of its total parameters. TML-Interaction-Small leads on Full-Duplex-Bench v1.5 and ProactiveVideoQA. GPT-Live-1 leads on Full-Duplex-Bench v1.5 and τ-Voice. Boson also notes that GPT and Gemini usually reach their best scores with thinking set high. Voice and avatar cloning were not benchmarked because the other models do not offer them.

How it sits among other real-time systems:

Higgs Realtime Duplex

TML-Interaction-Small

Dessa (Block)

Inputs

Audio, video, text

Audio, video, text

Audio

Outputs

Voice + rendered face

Voice, plus generated UI

Voice

Chunk size

160 ms

200 ms

Not disclosed

Size

45B MoE

276B MoE, 12B active

Not disclosed

Status

API waitlist

Research preview

Research preview, waitlist

Sources: Boson AI, Thinking Machines Lab, our Dessa article.

Open-weight full-duplex options for comparison include Kyutai's Moshi and NVIDIA PersonaPlex, both 7B, voice only, with weights on Hugging Face. Teams that need a talking face today usually pair a voice model with an avatar provider such as Tavus or Anam. Boson's own current option works the same way: Higgs Realtime talks while its separate Higgs Avatar model lip-syncs the speech in a LiveKit room, per its docs. Higgs Realtime Duplex folds both into one model.

What does it cost and where can you use it?

Higgs Realtime Duplex has no public price yet. Boson's stated design target is that an hour with the model costs at most one-tenth of an hour of a person's time.

The current model, Higgs Realtime, is available now. On Boson's pricing page it is metered on tokens ($0.75 per 1M input tokens, $4.50 per 1M output tokens), which Boson estimates at $0.0023 a minute for audio input and $0.014 a minute for audio output. Its API is compatible with OpenAI's Realtime protocol and has a Pipecat integration, per the docs. A first-party LiveKit integration is listed as coming soon. New accounts get $10 in trial credit.

What it means for teams building voice agents

  • Turn-taking moves into the model. Boson says the model itself decides when to backchannel, yield to an interruption or ignore a side conversation, rather than relying on fixed end-of-turn rules.

  • Latency and reasoning are split differently. The front model answers in real time and works on tool calls in the background, instead of each turn waiting for a large reasoning model.

  • Video becomes an input, a face an output. Use cases that need screen sharing or an on-screen agent (tutoring, support, drive-thru, kiosks) can run on one model instead of a voice stack plus an avatar layer.

What's missing

  • No price, latency figure or rate limits for Higgs Realtime Duplex itself.

  • No word on supported languages. Boson's homepage lists 100+ for the current Higgs Realtime.

  • No open weights, paper or model card. The benchmark numbers have not been reproduced outside Boson.

  • No timeline for the API beyond "coming soon".

Sources