LangSmith Adds AI Search to Find AI Agent Traces by Behaviour

LangChain has added AI Search to LangSmith, its platform for building and debugging AI agents. The feature lets developers search their agents’ activity logs in plain language, for example asking “did the user get frustrated?” or “did PII leak?”. LangChain announced it on 13 October 2026.

What it does

When an AI agent runs, LangSmith records a “trace”: the user’s request, the agent’s reasoning, the tools it called and its replies. “Today, we’re adding AI Search, a new way to query traces using natural language,” LangChain wrote.

Developers describe the behaviour they are looking for, and LangSmith returns the conversations where it happened. LangChain says this could find cases where an agent said it finished a task but didn’t, or where a user tried to cancel a subscription. Questions like these are hard to answer with keyword search.

How it works

According to LangChain, “AI Search pairs decision model inference with SmithDB”, its database for trace data. A fast “decision model” checks each conversation against the question. LangChain says traces from different tools, including the OpenAI and Anthropic SDKs, LangChain, LangGraph, the Vercel AI SDK and OpenTelemetry, are converted into one standard format first.

Agent conversations can be very long. LangChain said that in a sample of 150 conversations from its own coding agent, Open SWE, “a couple of them ran past a million tokens”. In the median conversation, “about 2K of its 72K tokens came from the user’s messages and the agent’s replies. About 85% was tool output.” To keep searches fast, “AI Search fills an 8K token budget in priority order”, putting what the user and agent said first.

LangChain says it is still working on harder cases, such as telling apart a user who gave up from one who got what they needed.

Availability

“AI Search is in public beta on LangSmith Cloud in the GCP US, EU, and APAC regions, at no additional cost,” LangChain said. Organisation admins can turn it on in settings.

What this means

• Easier agent debugging. Developers can ask questions about agent behaviour instead of reading logs.
• Safety checks get simpler. Teams can search for problems like leaked personal data.
• It is still a beta. LangChain has not published accuracy figures.

Sources

1. LangChain blog, “Search agent traces by behavior with AI Search in LangSmith”, 13 October 2026. https://www.langchain.com/blog/agent-traces-ai-search-langsmith

Related stories

• DXC Partners With LangChain and Licenses LangSmith for Its OASIS IT-Operations Agents
• SolarWinds Unveils Avior, an Agentic AI System for Enterprise IT Operations
• Chainguard Launches Three Tools to Keep AI Coding Agents Contained, Checked and Hunting for Bugs
• OpenAI Says AI Agents Now Account for More Than Half of Its EU Enterprise Use, in an Analysis of Its Own Customer Data