tiny-audio

Tiny Audio is an efficient speech-to-text system that demonstrates how high-accuracy voice transcription can be built on a training budget of just $25. By connecting a frozen speech encoder to a pretrained Qwen LLM using a small trainable projector (~80M parameters), it delivers competitive accuracy alongside word timestamps, speaker diarization, and a high-throughput HTTP server.

For the Non-Technical Reader

Think of traditional voice AI models as custom-built sports cars: expensive to construct, costly to maintain, and requiring massive infrastructure to build. Tiny Audio is like plugging a smart universal adapter between two high-performance off-the-shelf components—an existing speech engine and a language model—teaching them to work together instantly.

For business users and creators, this means fast, formatted transcripts with multi-speaker labeling (diarization) at a fraction of the usual operational and cloud hosting cost.

For the Technical Reader

• Architecture: Connects a frozen speech encoder to a Qwen LLM via a small, trained projector (~80M trainable parameters).
• Accuracy: Achieves a 1.8% Word Error Rate (WER) on LibriSpeech test-clean and 7.4% across 12 benchmarks (11,822 pooled samples).
• Performance & Throughput: Built-in batched HTTP server reaches ~460x real-time throughput at 128 concurrent requests on a single RTX 4090. Transcribes 45 minutes of audio in ~10 seconds (or ~21 seconds with speaker labels).
• Features & Diarization: Integrates a zero-shot port of NeMo's Nemotron-3-Diarization, provides word-level timestamps, and supports token-by-token streaming.
• Hardware Requirements: Requires ~6 GB VRAM (bf16 precision) on CUDA or Apple Silicon (MPS), running locally or via RunPod.

Why It Matters

This project demonstrates a major shift toward ultra-lean AI fine-tuning. Achieving competitive, production-ready speech recognition for $25 in compute lowers the barrier to entry for developers building privacy-first, self-hosted voice tools. It directly challenges expensive proprietary ASR APIs by providing extreme throughput on modest hardware.

The Voice AI Space Lab Idea

The Privacy-First Meeting & Podcast Co-Pilot: Deploy Tiny Audio on an Apple Silicon Mac or low-cost cloud GPU. Set up a folder watcher that automatically ingests audio recordings, separates speaker turns, generates timestamped transcripts, and passes structured output directly to a local LLM to draft executive summaries—all without sensitive voice data ever leaving your network.

Explore the project: GitHub Repository | Live HuggingFace Demo | HuggingFace Model Card