Deploy Voice Agent to Fly.io

Deploy ChatRAG's voice agent as an always-on worker process on Fly.io for ~$3-4/month.

Worker Process Architecture

The Voice Agent is a worker process, not a web server. It connects outbound to LiveKit Cloud and processes voice interactions — no HTTP ports required.

Architecture Overview

┌─────────────────┐       ┌─────────────────┐       ┌─────────────────┐
│   Browser/App   │◄─────►│  LiveKit Cloud  │◄─────►│   Voice Agent   │
│   (Frontend)    │       │   (Signaling)   │       │   (Fly.io)      │
└─────────────────┘       └─────────────────┘       └────────┬────────┘
                                                             │
                                                             ▼
                                                    ┌─────────────────┐
                                                    │  ChatRAG API    │
                                                    │  (Vercel/etc)   │
                                                    └─────────────────┘

The frontend connects to LiveKit Cloud, and the Voice Agent also connects to LiveKit Cloud. They "meet" in a shared room — no direct connection between frontend and Fly.io.

Prerequisites

Service	Purpose	Sign Up
Fly.io	Hosting	fly.io
LiveKit Cloud	Real-time audio	cloud.livekit.io
AssemblyAI	Speech-to-text	assemblyai.com
Resemble.ai	Text-to-speech	resemble.ai
Groq (recommended)	LLM responses	console.groq.com

Install Fly CLI

# macOS
brew install flyctl

# macOS/Linux (alternative)
curl -L https://fly.io/install.sh | sh

Step 1: Create Configuration Files

Your ChatRAG codebase should already have a voice-agent/ directory with the source code. You need to create two files for Fly.io deployment.

1.1 Create `voice-agent/Dockerfile`

FROM node:20-slim

WORKDIR /app

# CRITICAL: Install CA certificates for outbound HTTPS connections
# Without this, LiveKit SDK cannot reach LiveKit Cloud's signaling server
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/*

# Install dependencies
COPY package*.json ./
RUN npm install

# Copy source and build
COPY . .
RUN npm run build

# Remove devDependencies to save space
RUN npm prune --production

# Start the agent
CMD ["npm", "start"]

The ca-certificates package is essential. The node:20-slim base image does not include CA certificates, causing "failed to retrieve region info" errors.

1.2 Create `voice-agent/fly.toml`

# fly.toml app configuration file for ChatRAG Voice Agent
# See https://fly.io/docs/reference/configuration/

app = 'your-unique-app-name'  # ⚠️ CHANGE THIS: Must be globally unique
primary_region = 'iad'        # ⚠️ CHANGE THIS: Choose region close to users

[build]
  dockerfile = 'Dockerfile'

# VM resources - most economical "always-on" option
# 512MB RAM is recommended for Node.js apps
[[vm]]
  size = 'shared-cpu-1x'
  memory = '512mb'
  cpu_kind = 'shared'
  cpus = 1

# Worker process - always restart on exit
[[restart]]
  policy = 'always'
  processes = ['app']

# Environment variables (non-secret values only)
[env]
  NODE_ENV = 'production'
  ASSEMBLYAI_LANGUAGE = 'en'

⚠️ You MUST change app = 'your-unique-app-name' to a globally unique name.

Available Fly.io Regions

Choose a primary_region close to your users:

Code	Location
`iad`	Virginia, US
`sjc`	California, US
`lhr`	London, UK
`fra`	Frankfurt, DE
`nrt`	Tokyo, JP
`syd`	Sydney, AU

Step 2: Deploy to Fly.io

2.1 Navigate and Authenticate

cd voice-agent
fly auth login

This opens a browser window for authentication.

2.2 Create the Application

fly apps create --name your-unique-app-name

Replace with your unique app name from fly.toml.

Step 3: Set Environment Secrets

This is the most critical step. Replace all placeholder values with your actual API keys:

fly secrets set \
  LIVEKIT_URL=wss://your-project.livekit.cloud \
  LIVEKIT_API_KEY=your_livekit_api_key \
  LIVEKIT_API_SECRET=your_livekit_api_secret \
  ASSEMBLYAI_API_KEY=your_assemblyai_key \
  RESEMBLE_API_KEY=your_resemble_key \
  RESEMBLE_VOICE_ID=your_voice_id \
  RESEMBLE_STREAM_URL=https://f.cluster.resemble.ai/stream \
  CHATRAG_INTERNAL_API_URL=https://your-app.vercel.app/api/internal/voice-rag \
  CHATRAG_INTERNAL_API_KEY=your_internal_api_key \
  GROQ_API_KEY=your_groq_key \
  VOICE_AGENT_LLM_PROVIDER=groq \
  VOICE_AGENT_MODEL=llama-3.3-70b-versatile

Important: The CHATRAG_INTERNAL_API_URL must point to your production ChatRAG URL (e.g., https://your-app.vercel.app/api/internal/voice-rag), not localhost.

Step 4: Deploy the Application

fly deploy

This will:

Build the Docker image using Depot (Fly's build service)
Push the image to Fly's container registry
Create and start a machine in your chosen region

Step 5: Verify Deployment

Check Status

fly status

Expected: State = started

View Logs

fly logs

Expected healthy startup logs:

Voice Agent starting...
[Agent] Silero VAD loaded successfully
[VoicePipeline] Using Groq API with model: llama-3.3-70b-versatile
[AssemblyAI] Connected. Stream ready.
[LiveKit] Connected to room: chatrag-voice
[Agent] Track published and ready for audio capture

Environment Secrets Reference

Variable	Required	Description
`LIVEKIT_URL`	✓	WebSocket URL (wss://...)
`LIVEKIT_API_KEY`	✓	API key from LiveKit Cloud
`LIVEKIT_API_SECRET`	✓	API secret from LiveKit Cloud
`ASSEMBLYAI_API_KEY`	✓	For real-time STT
`RESEMBLE_API_KEY`	✓	For streaming TTS
`RESEMBLE_VOICE_ID`	✓	Voice ID from Resemble AI
`RESEMBLE_STREAM_URL`	✓	https://f.cluster.resemble.ai/stream
`CHATRAG_INTERNAL_API_URL`	✓	Your ChatRAG RAG API endpoint
`CHATRAG_INTERNAL_API_KEY`	✓	Internal API key for RAG access
`VOICE_AGENT_LLM_PROVIDER`	✓	openai, groq, or openrouter
`VOICE_AGENT_MODEL`		LLM model (default: gpt-4o)
`GROQ_API_KEY`	*	Required if using Groq (recommended)
`OPENAI_API_KEY`	*	Required if using OpenAI

Cost & Billing

Component	Configuration	Monthly Cost
Compute	`shared-cpu-1x` @ 512MB	~$3.19
Bandwidth	Minimal (audio via LiveKit)	~$0.50
Total		~$3-4/month

Since we set a fixed machine size and count, your bill is predictable. No auto-scaling unless explicitly configured.

Common Commands

# View logs
fly logs

# Check status
fly status

# List secrets (values hidden)
fly secrets list

# Update a secret
fly secrets set GROQ_API_KEY=new_value

# Restart machine
fly machine restart

# SSH into container
fly ssh console

Troubleshooting

"failed to retrieve region info"

Cause: Docker container lacks CA certificates for HTTPS.

Solution: Ensure your Dockerfile includes the ca-certificates package as shown above, then redeploy.

"MISSING OR PLACEHOLDER CONFIGURATION"

Cause: Environment secrets were not set or were set incorrectly.

Solution: Run fly secrets list to verify, then re-run fly secrets set with correct values.

Agent keeps restarting

Check for missing secrets ("MISSING CONFIGURATION" warning)
Consider increasing memory to 1024mb in fly.toml
Check logs for JavaScript errors

LiveKit connection timeouts

Verify your LiveKit Cloud project is active (not paused)
Confirm LIVEKIT_URL uses wss:// not https://
Check that API key/secret match the LiveKit project

Frontend Configuration

The frontend does NOT connect directly to the Fly.io agent. Both connect to LiveKit Cloud and "meet" in a shared room.

Ensure your ChatRAG frontend has these environment variables set:

LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=your_api_key
LIVEKIT_API_SECRET=your_api_secret

No configuration changes are needed on the frontend regarding the voice agent's URL — it works automatically via LiveKit.

← Previous: Voice Integration Next: WhatsApp Integration →