expo-gpt-live

An open-source mobile application framework built with Expo SDK 57 that demonstrates how to implement low-latency, full-duplex voice interactions using OpenAI's GPT-Live 1 API combined with interactive visual personas and real-time tool calling.

For the Non-Technical Reader

Imagine taking a phone call with an AI assistant that feels completely natural: you can interrupt it mid-sentence, ask for live web searches or weather updates, and keep the conversation going smoothly even when you lock your iPhone screen or switch to another app. The app also features interactive, animated visual avatars that react as you converse, turning raw AI voice streams into engaging digital companions.

For the Technical Reader

Architecturally, this project bridges React Native with real-time streaming voice interfaces using a robust native stack and backend proxy:

• Audio & Media Transports: Uses native WebRTC bindings within Expo SDK 57 for full-duplex, low-latency audio transmission with support for barge-in (interruption handling).
• UI & Persona Rendering: Integrates Rive for fluid, vector-based interactive persona animation variants.
• Architecture & Security: Operates via a Node.js middleware server that issues temporary access tokens and routes traffic, keeping the OpenAI API key safely off the mobile client.
• OS-Level Capabilities: Configured for persistent background audio sessions on iOS and includes dynamic tool-calling handlers for web search and weather data.
• Licensing: MIT License (Persona assets retain upstream licensing and attribution).

Why It Matters

As voice interfaces move from push-to-talk patterns to streaming multimodal interaction, developers need clear reference implementations for cross-platform mobile apps. This project addresses critical real-world hurdles in real-time voice AI: managing background audio lifecycles on mobile operating systems, securing API keys during WebSocket/WebRTC connections, and seamlessly blending real-time function calling with interactive visual rendering.

The "Voice AI Space Lab" Idea

Hands-Free Travel & Exploration Companion: Use this template to create a real-time hands-free audio tour guide. While hiking or walking through a new city with your phone in your pocket, the app can run in the background, providing context-aware audio observations, dynamically looking up real-time historical facts or weather conditions, and updating an animated mascot on screen whenever you glance at your device.

Explore the repository on GitHub.