Voice AI / Space
    Submit
    Back

    jarvis

    View on GitHub
    Git Repo
    pianistprogrammer

    Runs a local voice agent for Apple silicon that combines speech models with browser automation and workspace tools.

    About jarvis

    Jarvis is an open-source, voice-first local AI agent built specifically for Apple Silicon Macs. It enables hands-free voice interactions with local models that can autonomously operate a persistent web browser, execute file operations, and perform workspace tasks without sending audio or data to third-party servers.

    For the Non-Technical Reader

    Imagine having a personal assistant similar to Iron Man's J.A.R.V.I.S. residing entirely inside your MacBook. Instead of typing or manually clicking through websites, you simply speak out loud. You can ask Jarvis to open Hacker News, summarize articles, or perform calculations, and watch as it operates a real web browser on your screen in real time. Because everything runs locally, your voice and data stay completely private and secure.

    For the Technical Reader

    Jarvis achieves low-latency voice interaction by deploying an Apple-native modular cascade tailored for Apple Silicon unified memory:

    • Turn Detection & Interruption: In-browser Silero VAD enables low-latency voice activity detection and immediate user barge-in handling.

    • Speech-to-Text (STT): Fast local speech recognition powered by MLX Parakeet over an OpenAI-compatible endpoint.

    • LLM Inference: Qwen 9B (Q4KM quantization) served via llama.cpp with Metal acceleration and a 64K context window.

    • Text-to-Speech (TTS): Qwen3-TTS 1.7B (6-bit) via MLX-Audio, implementing sentence-level generation and audio playback overlap to reduce perceived time-to-first-audio (Kokoro fallback available).

    • Tool Architecture: Native persistent Playwright Chromium browser automation (DOM click/type, snapshots, live feed), guarded local shell execution, and workspace-scoped file capabilities.

    Why It Matters

    Jarvis is a clear demonstration of high-throughput, agentic local AI running on standard consumer hardware. By substituting heavy CUDA-based audio and LLM pipelines with native Apple Silicon (MLX and Metal) alternatives, it proves that real-time, privacy-preserving voice-agent interaction with complex tool use can run on devices with as little as 8 GB to 12 GB of unified memory, eliminating reliance on costly proprietary voice and LLM APIs.

    Check out the implementation and run it locally via the Jarvis GitHub Repository.