Work
Maurice VOIP, first generation: the original phone bridge
This is the first phone bridge Blacksky built for Maurice. Its audio pipeline became the Dewey phone state that the current Maurice and every Dewey deployment run on.
Project Type
Production Voice AI System — Live
Overview
Maurice VOIP is a voice AI bridge that connects the Maurice AI chatbot to real phone calls via Telnyx. Callers dial a real phone number and have a natural voice conversation with Maurice, who retains his full personality, RAG knowledge base, and lead-tracking capabilities — all optimized for voice interaction.
The Problem
Web chatbots miss an entire audience: people who prefer to call. Traditional IVR systems are robotic and frustrating. Maurice VOIP bridges the gap — a real AI assistant that answers the phone with personality, knowledge, and memory.
How It Works
Audio Pipeline
- Caller speaks → Telnyx streams audio via WebSocket in G.711 mu-law format
- Custom mu-law decoder converts to PCM16 audio
- Whisper speech-to-text transcribes the caller's voice
- Text sent to cloud Maurice via SSE streaming endpoint
- LLM streams tokens back, accumulated into complete sentences at punctuation boundaries
- Each sentence immediately sent to Edge TTS for speech synthesis
- TTS audio converted to mu-law and played through the AudioMixer
- Overlapping pipeline: while sentence N plays, sentence N+1 is already generating TTS
Always-On AudioMixer
The mixer runs on a continuous 20ms tick, mixing two audio layers:
- Ambient loop: Hip-hop instrumental playing at low volume (0.04), always running
- Overlay: TTS speech, greeting sounds, thinking sounds played on top
Volume states with smooth ramping:
- Idle: Ambient at 0.04 — barely perceptible background presence
- Thinking: Ambient ramps up to 0.25 — fills the gap while LLM processes
- Speaking: Ambient ducks to 0.06 — stays present but doesn't compete with voice
Phone Identity
The caller's phone number IS their identity — like a cookie that never clears. Same number, same user profile, same conversation history. Whether they call once or fifty times, Maurice knows who they are and remembers previous conversations.
Latency Optimization
- Pre-cached greeting audio plays instantly on answer (no LLM wait)
- Pre-cached "let me think about that" thinking sound fills processing gap
- Quick response pattern matching intercepts common phrases (greetings, recording notices) without hitting the LLM
- Sentence-level TTS streaming starts playback before full response is generated
- Whisper runs in thread executor to prevent blocking the mixer
Technical Details
- VOIP: Telnyx TeXML with WebSocket media streaming
- STT: faster-whisper (base model, int8 quantization for CPU)
- TTS: Edge TTS (en-GB-RyanNeural voice) → ffmpeg → mu-law
- Audio: Custom ITU G.711 mu-law encoder/decoder using numpy vectorized operations
- Mixer: AsyncIO-based 20ms tick mixer with ambient loop and overlay queue
- LLM: Cloud Maurice via SSE streaming (Together AI + Llama 3.1 70B)
- Deployment: Railway (Docker with pre-downloaded Whisper model)
- Framework: Python, FastAPI, uvicorn
What Makes It Novel
- Always-on ambient audio that volume-ducks during speech — not silence between responses
- Sentence-level overlapping TTS pipeline — starts playing first sentence while generating second
- Phone number as persistent identity — no login, no cookies, just your number
- Full RAG knowledge retrieval in voice context — not a scripted phone tree
- Custom mu-law encoder following ITU G.711 spec with numpy vectorization
Status
Live and deployed on Railway. Connected to Telnyx phone number. Handling real calls.