← Back to all guides

Web Search — Giving Agents Real-Time Knowledge

Langoedge Team4 min read

Why Agents Need Web Access

Large Language Models are limited by their training cutoff. To make an agent aware of recent events, look up live pricing, or research a company before a call, it needs a way to reach the open web at run time.

Langoedge gives you two routes, and which one you need depends on whether you already know where the answer lives.

You know the URL You don't know the URL
Use the built-in crawl_url tool. Add an MCP Connector for a search provider.
There is no built-in `search_web` or `exa_search` tool. Earlier versions of this page described those as native tools; they were never part of the shipped platform. Open-ended web search is available through an MCP Connector, as described below.

1. Fetching a Known Page (crawl_url)

crawl_url is a native tool — no account connection, no API key. It reads a live URL and converts it to clean markdown the model can ingest.

Use it when the agent knows exactly which page it needs: your own pricing page, a documentation URL, a competitor's homepage, a status page.

Tool Parameters

Parameter Type Required Description
url string Yes The page to fetch and convert to markdown.
url_patterns list No Restrict crawling to URLs matching these patterns.
allowed_domains list No Only follow links on these domains.
blocked_domains list No Never follow links on these domains.

See the Web Crawler guide for the full parameter set and crawl-depth behaviour.

2. Open-Ended Search (via MCP Connector)

For "find me pages about X", connect a search provider on the Connect page and expose it through an MCP Connector node. The catalogue carries the search engines built for AI agents — Tavily, Exa, Perplexity, Google Search, Bing and others — so you pick the one whose result style and pricing suits you rather than being locked to whichever one Langoedge happened to bundle.

1

Connect the provider

On the **Connect** page, search for the provider (for example Tavily), click **Connect Account**, and supply its API key when prompted.
2

Add an MCP Connector

On your Text Graph canvas, add an **MCP Connector**, pick the provider, and it loads that provider's live tool catalogue.
3

Attach it to the researching step

Select the connector from the tool list of whichever step should be able to search.

Because the catalogue is discovered at run time, the exact tool names and parameters come from the provider itself — the connector will show you what's available once the account is linked.

Best Practices for Search Integration

Query Optimization

Instruct your LLM node to write clear, target-oriented search queries. Avoid conversational filler like *"Can you find..."* — use exact keywords and parameters.

Result Summarization

Web searches yield long text snippets. Always place a summarization node after search operations to extract relevant points before feeding them back into the main state history.

Prefer crawl_url when you can

If the agent already knows the URL, fetching it directly is faster, cheaper, and more predictable than a search round trip.

Watch your latency budget

A search call adds seconds. On a voice agent, run it inside a Text Graph mounted asynchronously so the conversation doesn't stall.

Frequently Asked Questions

Is there a built-in web search tool?
Not for open-ended search. `crawl_url` is built in and fetches a URL you supply. For search, connect a provider such as Tavily or Exa through an MCP Connector — this also means you choose the engine and control its billing directly.
Do I need my own API key?
Yes, for a search provider reached via MCP. You supply the key when connecting the account, and the provider bills you directly. `crawl_url` needs no key.
What about academic or encyclopedia lookups?
Those are built in and need no connection — `arxiv_search` and `wikipedia_search`. See the [Academic & Reference Lookup guide](/guides/tools/academic-search).
How fresh are the results?
`crawl_url` fetches the page at the moment of execution, so it is always current. For MCP-backed search, freshness is whatever the provider offers.
LT

Langoedge Team

The Langoedge engineering team builds AI agent infrastructure that empowers businesses to deploy reliable, observable AI staff. Follow Langoedge Team on LinkedIn for product updates and architectural deep dives.