Voice-to-Text Sub-Routing: Async Tools for Voice Agents

Voice-to-text sub-routing is an architecture for voice AI agents that splits a live call into two cooperating parts: a fast conversational loop that keeps talking to the caller, and a separate asynchronous text graph that does slow backend work such as a database write, a CRM update or a third-party API lookup. The voice loop hands the work off as a tool call and never blocks waiting for it. When the text graph finishes, its result is folded into what the agent says next.

Key facts

• Real phone calls on major voice platforms already take 1.3 to 1.75 seconds median to first reply audio before any tool call runs (Openbenchmarks).
• A tool call run inline in the voice turn adds its full duration to the caller's silence.
• Sub-routing is a structural fix, not a speed fix: a faster model doesn't make a slow downstream API faster.
• In Langoedge, the text graph is its own LangGraph state machine with its own checkpointed state, so it can retry (for example, try the next free slot) without stalling the voice loop.

The problem: backend work on a live call

A voice agent has a timing constraint a chat agent doesn't. In a chat window, a two-second pause while a tool runs is fine. On a phone call, silence past a second or two sounds broken. Daily's Pipecat team puts the total budget for natural conversation at under 1,500ms voice-to-voice, with about 700ms already going to LLM time-to-first-token (Daily). There is no room left for a three-second calendar API call.

The naive design makes the voice loop wait. That's where the dead air comes from, the "let me just check that for you" loop, or the caller talking over a reply that hasn't arrived.

A single-loop agent has one option when backend work is slow: make the caller wait. A sub-routed agent can keep the conversation moving.

Why this isn't the same as "faster inference"

Most latency competition is about the conversational turn: faster speech-to-text, faster first token, faster text-to-speech. We cover that in voice AI agent latency. Those numbers describe speech generation. They say nothing about what happens when the agent needs to do something outside the conversation. A platform with excellent time to first audio can still stall badly the moment a tool call sits on the critical path.

How the split works

The voice graph (the live call) mounts a text graph as a callable tool. When the voice agent needs backend work, it calls that tool. The text graph runs the actual logic as its own LangGraph state machine, checkpointed independently, while the voice loop carries on. When the text graph resolves, the result lands in shared session state and the voice agent uses it on its next turn.

Two state machines. The voice loop reads the text graph's result when there is one; it never waits on it.

Here's the pattern reduced to plain Python asyncio. It isn't Langoedge's production code, just the shape of the idea:

import asyncio

async def check_calendar(slot: str) -> dict:
    # Stand-in for a slow third-party API call.
    await asyncio.sleep(2.5)
    return {"slot": slot, "available": False, "next_free": "Tue 3:00pm"}

async def voice_turn(say, session: dict, slot: str) -> None:
    # Kick off the backend work without awaiting it.
    session["pending"] = asyncio.create_task(check_calendar(slot))
    # Keep the caller engaged with something useful, not filler.
    await say("Let me check Tuesday at 2 for you. Is this for a new issue or a follow-up?")

async def next_turn(say, session: dict) -> None:
    task = session.get("pending")
    if task and task.done():
        result = task.result()
        if result["available"]:
            await say(f"Great, {result['slot']} is yours.")
        else:
            await say(f"2 o'clock has gone, but {result['next_free']} is free. Does that work?")
    elif task:
        await say("Still checking that, one moment.")

The important parts are that the tool runs as a task, the voice turn asks a question that's useful anyway (so the wait isn't wasted), and the next turn reads the result only if it's ready.

Retries belong in the text graph

Because the text graph is a real state machine rather than a linear script, it can loop. If a booking hits a slot that was just taken, it can try the next one before reporting back, instead of failing the tool call and leaving the voice agent with nothing to say. That's the behaviour we describe in stateful voice AI agents.

After-call work

The same mechanism can fire once a call ends, rather than mid-call. Writing a visit summary into a practice-management system is a good example: there's no reason to keep the caller on the line for it.

When you don't need sub-routing

If a call is purely conversational (answering a question, confirming details already known) the text graph never fires, and there's no added cost. And if a tool reliably returns in 100 to 200ms, running it inline is simpler and fine. Sub-routing earns its keep when tools are slow, variable, or need retries: calendars, CRMs, practice-management systems, anything behind a rate limit.

How other platforms approach it

Developer voice platforms such as Vapi, Bland and Retell are strong at the conversational layer. In most single-loop designs, a tool call either blocks the turn or leaves the developer to manage async state themselves. Retell's Workflows, announced in September 2026, move data in before a call and out after it, which covers the after-call case but not mid-call orchestration. Text-first graph builders such as Langflow, Dify and n8n have the opposite gap: real orchestration but no native telephony loop.

None of this is proprietary magic. LiveKit, Pipecat and LangGraph are open infrastructure, and anyone can build the pattern. The value is in having it as a first-class, tested part of the platform rather than something each team rebuilds.

FAQ

What is voice-to-text sub-routing?

Voice-to-text sub-routing is an architecture for voice AI agents in which the live conversation runs on a fast voice loop while backend tasks such as database writes, API calls and CRM updates run in a separate asynchronous text graph that the voice loop calls as a tool without waiting for it. The result is folded into the conversation when it's ready.

How is sub-routing different from a voice platform just being fast?

Speed metrics such as time to first audio byte describe how quickly the agent produces its next spoken reply. Sub-routing addresses a different problem: work that takes longer than a natural conversational pause. A platform can have fast replies and still produce dead air if a slow tool call runs inside the voice turn.

Does sub-routing add latency to simple calls?

No. The text graph only runs when the voice agent needs backend work. Purely conversational turns never touch it.

What happens if the backend task fails?

Because the text graph is a state machine, it can retry or take another path, for example trying the next available appointment if the first was just booked, before returning a result the voice agent can use.

Is sub-routing only for phone calls?

No. It applies to any real-time voice interface where backend work must happen without freezing the conversation. Phone calls are simply the most common case.

Sources

• Openbenchmarks, voice agent latency (TTFAB from real phone calls)
• Daily, "Benchmarking LLMs for Voice Agent Use Cases"
• Retell AI, "Introducing Retell Workflows" (September 2026)
• LangGraph documentation, persistence and checkpointers

About the author. Arnab Chakraborty is the founder of Langoedge, a Melbourne company building voice and text AI agents on LangGraph with direct SIP telephony. Disclosure: this article describes our own architecture alongside competitors'.