Consuming Voice Agents in Client-Side Web Applications (WebRTC)

Integration Architecture

To establish an ultra-low latency, real-time voice call between a user's browser and your Langoedge voice agent, the connection is established over WebRTC using LiveKit.

Because connecting to a real-time room requires a secure participant token, the integration involves both your backend and frontend application:

Step 1: Create a Secure Backend Session Proxy

To request a LiveKit connection token, you must make a POST request to Langoedge's session API. Since this requires sensitive authorization credentials, you should never request this directly from the browser. Instead, set up a backend route in your application to act as a proxy.

Here is an example Next.js API Route Handler (app/api/voice-agent/session/route.ts):

import { NextResponse } from "next/server";

export async function POST() {
  try {
    const agentId = process.env.LANGOEDGE_VOICE_AGENT_ID;
    const secret = process.env.LANGOEDGE_SECRET;
    const userId = process.env.LANGOEDGE_USER_ID;
    const baseUrl = process.env.LANGOEDGE_API_BASE_URL || "https://api.langoedge.com";

    if (!agentId || !secret || !userId) {
      return NextResponse.json(
        { error: "Voice agent configuration is incomplete on the server." },
        { status: 500 }
      );
    }

    const payload = {
      participant_id: `user_${Math.random().toString(36).substring(2, 11)}`,
      first_name: "John",
      last_name: "Doe",
      metadata: {
        source: "web_app"
      }
    };

    const response = await fetch(`${baseUrl}/voice-graphs/${agentId}/sessions/connect`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "x-langoedge-secret": secret,
        "x-langoedge-user-id": userId,
      },
      body: JSON.stringify(payload),
    });

    if (!response.ok) {
      const errorText = await response.text();
      console.error("Langoedge voice session error:", errorText);
      return NextResponse.json(
        { error: "Failed to create session with voice agent backend" },
        { status: response.status }
      );
    }

    const data = await response.json();
    // Returns { serverUrl: string, roomName: string, participantToken: string }
    return NextResponse.json(data);
  } catch (error) {
    console.error("Proxy error:", error);
    return NextResponse.json(
      { error: "Internal server error during proxying" },
      { status: 500 }
    );
  }
}

Required Environment Variables

API Response

The session endpoint returns a JSON object with the following fields:

Step 2: Install LiveKit Client-Side SDKs

In your client application, install the LiveKit React components and WebRTC client SDK:

npm install @livekit/components-react @livekit/components-styles livekit-client framer-motion lucide-react

Make sure to import the LiveKit CSS styles once in your layout or main file:

import "@livekit/components-styles";

[!NOTE]
framer-motion is required if you use the advanced visualizer component below for smooth SVG animations and state transitions.

Step 3: Build the Voice Call Controller Component

The controller component manages the active session state, triggers the backend proxy to retrieve the room token, and mounts the LiveKit room when a call starts.

"use client";

import React, { useState } from "react";
import { LiveKitRoom, RoomAudioRenderer } from "@livekit/components-react";
import { PhoneCall } from "lucide-react";
import { VoiceAgentVisualizer } from "./VoiceAgentVisualizer";

export function CallFlowMockup() {
  const [isCallActive, setIsCallActive] = useState(false);
  const [isLoading, setIsLoading] = useState(false);
  const [connectionDetails, setConnectionDetails] = useState<{
    serverUrl: string;
    participantToken: string;
  } | null>(null);
  const [callError, setCallError] = useState<string | null>(null);

  const startVoiceCall = async () => {
    try {
      setIsLoading(true);
      setCallError(null);
      const response = await fetch("/api/voice-agent/session", {
        method: "POST",
        headers: { "Content-Type": "application/json" }
      });
      if (!response.ok) {
        throw new Error("Failed to start voice agent session");
      }
      const data = await response.json();
      setConnectionDetails({
        serverUrl: data.serverUrl,
        participantToken: data.participantToken
      });
      setIsCallActive(true);
    } catch (err) {
      console.error(err);
      setCallError("Could not connect. Please try again.");
    } finally {
      setIsLoading(false);
    }
  };

  return (
    <div className="w-full max-w-md flex flex-col items-center justify-center">
      {isCallActive && connectionDetails ? (
        <LiveKitRoom
          video={false}
          audio={true}
          token={connectionDetails.participantToken}
          serverUrl={connectionDetails.serverUrl}
          onDisconnected={() => {
            setIsCallActive(false);
            setConnectionDetails(null);
          }}
          className="w-full"
        >
          {/* RoomAudioRenderer mounts the browser audio element to output the remote agent's speech */}
          <RoomAudioRenderer />

          {/* VoiceAgentVisualizer listens to audio levels and displays agent status */}
          <VoiceAgentVisualizer />

          <button
            onClick={() => {
              setIsCallActive(false);
              setConnectionDetails(null);
            }}
            className="mt-4 px-6 py-2.5 bg-rose-600 hover:bg-rose-500 text-white rounded-full flex items-center gap-2 font-medium mx-auto"
          >
            <PhoneCall className="w-4 h-4 rotate-[135deg]" />
            End Call
          </button>
        </LiveKitRoom>
      ) : (
        <button
          onClick={startVoiceCall}
          disabled={isLoading}
          className="relative w-24 h-24 bg-gradient-to-tr from-emerald-500 to-teal-500 rounded-full flex items-center justify-center text-white hover:scale-105 active:scale-95 transition-transform shadow-lg shadow-emerald-500/20"
        >
          {isLoading ? (
            <div className="w-6 h-6 border-2 border-white/30 border-t-white rounded-full animate-spin" />
          ) : (
            <PhoneCall className="w-8 h-8 fill-current" />
          )}
        </button>
      )}

      {callError && (
        <p className="mt-4 text-sm text-rose-400">{callError}</p>
      )}
    </div>
  );
}

Step 4: Create the Real-Time Voice Agent Visualizer

The visualizer uses @livekit/components-react hooks to track the agent's conversational state (listening, thinking, speaking, etc.) and extract real-time multi-band audio volume levels. These volume levels drive a radial SVG equalizer with 180 frequency bands, providing rich visual feedback.

"use client";

import { useMemo } from 'react';
import { motion } from 'framer-motion';
import {
  useMultibandTrackVolume,
  useVoiceAssistant,
  ParticipantTile,
  useTracks,
  TrackLoop,
} from '@livekit/components-react';
import { Track } from 'livekit-client';

interface VoiceAgentVisualizerProps {
  appName?: string;
  plain?: boolean;
}

export function VoiceAgentVisualizer({ appName = 'Langoedge', plain = false }: VoiceAgentVisualizerProps) {
  const { state, audioTrack } = useVoiceAssistant();
  const barCount = 180;

  // Check for agent video track (optional camera feed)
  const tracks = useTracks([Track.Source.Camera, Track.Source.ScreenShare, Track.Source.Unknown]);
  const agentVideoTrack = useMemo(() => {
    return tracks.find(
      (trackRef) =>
        !trackRef.participant?.isLocal &&
        trackRef.publication?.kind === Track.Kind.Video
    );
  }, [tracks]);

  // Extract multi-band volume thresholds for dynamic equalizer
  const volumes = useMultibandTrackVolume(audioTrack, {
    bands: barCount,
    loPass: 20,
    hiPass: 900,
    updateInterval: 26,
  });

  // Blend volume data with symmetrical mirroring for a balanced visualization
  const blendedVolumes = useMemo(() => {
    const len = volumes.length || barCount;
    const safeGet = (i: number) => volumes[(i + len) % len] ?? 0;
    const half = Math.floor(len / 2);
    const quarter = Math.floor(len / 4);

    return volumes.map((_, idx) => {
      const local =
        safeGet(idx - 2) * 0.1 +
        safeGet(idx - 1) * 0.2 +
        safeGet(idx) * 0.4 +
        safeGet(idx + 1) * 0.2 +
        safeGet(idx + 2) * 0.1;
      const opposite = safeGet(idx + half) * 0.35;
      const quarterMix = safeGet(idx + quarter) * 0.15 + safeGet(idx - quarter) * 0.15;
      return Math.min(1, local + opposite + quarterMix);
    });
  }, [barCount, volumes]);

  const statusLabel = useMemo(() => {
    switch (state) {
      case 'connecting':        return `Connecting to ${appName}...`;
      case 'pre-connect-buffering': return 'Preparing to listen...';
      case 'initializing':      return 'Initializing voice session...';
      case 'idle':              return `${appName} is listening...`;
      case 'listening':         return 'Listening to you...';
      case 'thinking':          return 'Thinking...';
      case 'speaking':          return `${appName} is speaking...`;
      case 'failed':            return 'Connection failed';
      case 'disconnected':      return 'Call ended';
      default:                  return 'Connecting...';
    }
  }, [state, appName]);

  const pulseScale =
    state === 'speaking' ? 1.06 : state === 'listening' || state === 'pre-connect-buffering' ? 1.03 : 1.01;

  const baseEnergy =
    state === 'speaking' ? 0.42 : state === 'listening' || state === 'pre-connect-buffering' ? 0.28 : 0.16;

  const rotationDuration =
    state === 'speaking' ? 18 : state === 'listening' || state === 'pre-connect-buffering' ? 26 : 36;

  return (
    <div className="relative isolate flex w-full flex-col items-center justify-center overflow-hidden rounded-2xl border border-zinc-700 bg-zinc-900/60 px-6 py-8 min-h-[260px]">
      {/* Background glow orbs */}
      <motion.div
        className="pointer-events-none absolute inset-[-24%] -z-20 rounded-full bg-[radial-gradient(circle_at_30%_35%,rgba(255,216,74,0.08),transparent_40%),radial-gradient(circle_at_70%_60%,rgba(16,185,129,0.06),transparent_45%),radial-gradient(circle_at_50%_50%,rgba(244,63,94,0.05),transparent_40%)] blur-3xl"
        animate={{ scale: [1, 1.04, 1] }}
        transition={{ duration: 14, repeat: Infinity, ease: 'easeInOut' }}
      />

      {/* Animated radial SVG equalizer */}
      <motion.div
        className="relative flex w-full items-center justify-center h-[160px]"
        animate={{ scale: pulseScale }}
        transition={{ type: 'spring', stiffness: 120, damping: 18 }}
      >
        {agentVideoTrack ? (
          // If agent has a camera feed, show as circular tile
          <div className="relative w-40 h-40 rounded-full overflow-hidden border-2 border-primary/30 bg-zinc-900">
            <TrackLoop tracks={[agentVideoTrack]}>
              <ParticipantTile className="w-full h-full object-cover rounded-full" />
            </TrackLoop>
          </div>
        ) : (
          // Otherwise render the 180-band radial equalizer
          <AnimatedRadialEqualizer
            blendedVolumes={blendedVolumes}
            baseEnergy={baseEnergy}
            rotationDuration={rotationDuration}
          />
        )}
      </motion.div>

      <p className="mt-4 text-xs font-bold uppercase tracking-widest text-emerald-400/80 animate-pulse">
        {statusLabel}
      </p>
    </div>
  );
}

Visualizer States

The visualizer adapts its animation to the agent's conversational state:

Step 5: User Interaction & Browser Policies

To prevent security warnings and ensure a smooth user experience, keep the following browser limitations in mind:

1. User Gesture Gate: Modern browsers block microphone access and audio playback unless initiated by a direct user gesture (like clicking a "Start Call" button). Do not attempt to initialize voice rooms automatically on page load.

2. Secure Context: WebRTC requires secure origins (https:// or localhost). If deploying to external staging environments, make sure SSL is configured.

Frequently Asked Questions