NeuTTS-2E: on-device emotional TTS, built for the sentences that don't help it

Ask an emotional text-to-speech model to read "I'm just so upset and unhappy right now" angrily and it will probably nail it. Ask it to read "the meeting is at four" angrily and, Neuphonic argues, most models start guessing.

That gap is the whole reason NeuTTS-2E exists. Neuphonic's new on-device model, out now in early alpha, is built around one specific question: can a 125M-parameter model follow an emotion instruction when the sentence itself gives it nothing to work with?

Why emotional TTS gets graded on easy questions

Neuphonic's case is that emotional speech models tend to look better than they are, because of how they're evaluated and how they're trained.

The demo sentences are almost always semantically aligned. Angry delivery, angry words. Sad delivery, sad words. When the vocabulary already carries the signal, the model can lean on the text and produce something convincing without ever having learned emotion as a separate, controllable thing.

Three problems stack up underneath that.

Data is scarce. Clean recordings with clearly differentiated emotional performances are far rarer than ordinary speech data, and what exists varies wildly in recording quality, acting intensity, and what each dataset even means by "sad."

Labels are subjective. One annotator's "sad" is another's "tired" or "disappointed." Fear and surprise blur. Anger and disgust blur. You can expand a dataset with an automatic emotion classifier, but a weak classifier just injects noisy supervision, and Neuphonic's argument is that many of those classifiers have the same semantic dependency as the TTS models they're used to train, which would compound the problem rather than fix it.

Semantic bias is baked in. If nearly every training example has words and delivery agreeing, then a neutral or conflicting sentence is out of domain at inference time. The model hasn't learned emotion. It's learned correlation.

So the useful test isn't "does angry sound angry." It's whether you can request sadness for cheerful words, or fear for reassuring ones, and get it.

What's actually in the box

NeuTTS-2E takes three inputs: text, one of four fixed speakers (emily, paul, sophie, steven), and one of seven modes (angry, disgusted, fearful, happy, sad, surprised, neutral). No phonemizer, no reference audio, no system dependencies.

from neutts import NeuTTS2E
import soundfile as sf

tts = NeuTTS2E()
wav = tts.infer(
    "I can't believe it's finally here!",
    speaker="emily",
    emotion="happy",
)
sf.write("test.wav", wav, 24000)

The specs, from the model card and repo:

  • Active backbone parameters: roughly 125M, on a qwen3 architecture

  • Total including tied embeddings and head: roughly 236M

  • Context window: 2048 tokens, around 30 seconds of audio including the prompt

  • Codec: NeuCodec, 50 tokens per second, single codebook, 0.8 kbps, 16 kHz in and 24 kHz out

  • Formats: safetensors plus Q8 and Q4 GGUF, with an ONNX decoder (including int8) for the codec

  • English only, four speakers, released under the NeuTTS Open License 1.0

Sticking with a standard qwen3 backbone is the quietly practical decision here. It means the GGUF builds drop straight into llama.cpp and llama-cpp-python, and streaming works through tooling you'd already have running.

Generation is sampled, not deterministic. Every call prints the seed it used, so when a take lands you can pin it and reproduce it exactly across PyTorch and GGUF, batch or streaming. Small thing, but it's the difference between a demo and a build pipeline.

Four speakers instead of cloning, on purpose

Every other model in the NeuTTS family does instant voice cloning from a few seconds of reference audio. This one doesn't, and Neuphonic is upfront that arbitrary cloning is technically possible but weak in the current release.

That reads like a missing feature until you think about what a 125M-parameter model is being asked to juggle. Cloning means holding an unfamiliar speaker's identity stable while also maintaining intelligibility, naturalness, prosody, and the requested emotion. Those objectives compete for capacity. Fixing the speaker set frees the model to spend that capacity on emotional consistency instead.

The four voices ship pre-encoded in the repo, which also means they work as ordinary cloning references for the other NeuTTS models if you want them elsewhere.

What you'll still need to test yourself

This is an alpha, and the interesting claims are the ones you should verify against your own use case before committing.

Semantic independence. The core pitch. Feed it deliberately neutral text and deliberately conflicting text, not the demo sentences. This is the thing to measure.

Intensity control. There isn't one. Seven categories, no continuous parameter, so "angry" covers everything from mild irritation to shouting. Because generation is sampled you can reroll for a different take and pin the seed when one lands, but that's a slot machine, not a dial. Mid-generation emotion switching is also not in this release.

Speaker consistency across modes. Does emily still sound like emily when she's furious? Worth checking directly rather than assuming.

Throughput on your hardware. Neuphonic publishes CPU and GPU benchmarks in the repo, but for NeuTTS-Air and NeuTTS-Nano, not for 2E. The parameter counts are within a few million of each other, so Nano's numbers are a plausible proxy. They aren't a measurement. Also note the published figures cover the speech language model only, not the codec, so full pipeline latency will be higher.

Watermarking. Perth perceptual watermarking is on by default, which is the right call. One gotcha worth knowing: if you install inside the repo with uv sync, the Perth dependencies don't get pulled and watermarking disables, with nothing but a missing-dependency warning in the logs to tell you. Easy to scroll past. Install via pip if you need it active, and verify it is.

Who this is for

Cloud TTS still wins when you want the best available quality, a large voice library, or the least integration work. That's not what this is competing on.

NeuTTS-2E makes sense when you're generating a lot of speech at runtime and per-request pricing starts to hurt, when the thing has to work offline, when audio and text can't leave the device, or when you need marginal costs you can actually predict. Embedded agents, games, robotics, toys, offline assistants. Characters that need to sound like they mean it.

It's an alpha aimed at a narrow question rather than a general-purpose replacement for larger expressive systems, and it's better to treat it that way. But the question it's asking is the right one, and it ships in a form (small, quantised, seed-pinnable) that makes it unusually easy to test that question yourself.

Try it in the Hugging Face demo, or pip install neutts and start with sentences that give the model nothing.

Published in collaboration with Neuphonic.com