Our chatbot had one way of thinking, and it used that one way for everything. This post is about the deep agent architecture we built to replace it.
Ask it “hi” and it would stop, reason about what it needed to know, decide to search, run the search, read the results, and reason again about whether to search once more. Ask it to compare five vendors on price, latency, and compliance and it would do exactly the same thing. Same loop. Same tools. Same number of sources pulled back. The only difference between those two requests was how many times the loop spun before the model decided it had enough.
We shipped it anyway, because it worked. People used it. But as the questions got harder, we kept hitting the same problem: slow on the easy things, shallow on the hard ones, and equally confident either way.
The problem was never that the model was not smart enough. The problem was that our architecture handed it exactly one routine and no way to choose a different one. A greeting and a research assignment went through identical machinery, and the machinery could not tell them apart.
This is the story of replacing that single routine with a deep agent: an architecture that first decides how much effort a question deserves, then brings only the machinery that effort actually requires.
The original design: one ReAct loop for everything
Our chatbot brain ran a ReAct loop. ReAct is short for Reason and Act. The model reasons about what it needs, takes an action (usually a tool call), observes what comes back, and reasons again. It repeats until it decides it can answer.
Think of hiring a research assistant. You ask a question. They pause and think about what they need to know. They go look something up. They read it. They ask themselves whether they have enough. They decide, and they repeat.
ReAct became the default for good reasons. It is a few dozen lines of code. It handles open-ended questions without anyone having to anticipate them in advance. And it degrades gracefully, because a model that does not need a tool simply answers.
But a single ReAct loop has four structural properties that only reveal themselves as problems once real traffic hits it.
- It has one routine. Nothing in the loop ever asks how hard the incoming question is. Every request enters the same machinery at the same depth.
- It trusts its tools. Whatever a tool returns goes straight into context as fact. There is no gate between retrieval and reasoning.
- It has one shared context. Every observation from every step piles into the same message history, and the final answer is written from that same pile.
- It has no plan. The model may reason, but nothing obliges it to, and nothing tracks which parts of a question are still unanswered.
What a deep agent actually is
A deep agent is not a smarter model. It is a harness built around the same tool-calling loop, with scaffolding that handles the things a bare loop handles badly. LangChain’s deepagents library is the reference implementation, and it organises that scaffolding into four groups.
- Execution environment. The agent gets tools, but also a virtual filesystem it can read and write, permission rules over which paths it may touch, and optional sandboxed code execution. Work can live outside the context window, in files.
- Context management. Skills load domain knowledge on demand rather than upfront. Memory files persist preferences and conventions across sessions. Summarization and offloading compress long histories and oversized tool results automatically. On supported models, the static parts of the prompt are cached.
- Delegation. The agent can maintain a structured task list as it works, and it can spawn subagents. A subagent runs in its own fresh context, works autonomously, and hands back a single distilled report.
- Steering. Sensitive tool calls can pause and wait for a human to approve, edit, or reject them before they run.
The unifying idea is simple. A plain ReAct loop has one context, one routine, and one level of effort. A deep agent has several of each, plus the ability to decide which one a given request should get.
We did not adopt all of it. We took the capabilities that mapped onto problems we actually had, and where the harness had no answer, we built our own on top of it, problem by problem, some fixes came free with the harness and some we added ourselves. Each section below says which is which.
Problem 1: Every question got the heavyweight treatment
The problem
A greeting should be instant. A factual lookup should be quick. A five-way comparison should be thorough. We gave all three the same routine: reason, search, observe, reason again.
The cost ran in both directions. Simple questions took seconds longer than they needed to and burned tokens on research that was never necessary. Worse, the loop’s bias toward acting meant that on easy questions the agent would go looking things up that it already knew, and come back with a weaker answer than if it had simply spoken.
The deep agent fix: triage before effort
Before any real work happens, one fast classification pass reads the question and sorts it into a lane.
- DIRECT. Greetings, follow-ups, clarifications. Answer from what the model already knows. No tools at all.
- TOOL. One clear factual question, like the capital of France or the height of the Eiffel Tower. One lookup which runs through the web search tool we built ourselves, one answer, stop.
- DEEP_RESEARCH. Genuinely open-ended work. Compare these approaches, trace the history of this, investigate this properly. This lane gets the full loop.
class Intent(str, Enum):
DIRECT = “DIRECT” # no lookups needed
TOOL = “TOOL” # one lookup
DEEP_RESEARCH = “DEEP_RESEARCH” # full investigation
Each lane only assembles the machinery it needs. Simple messages skip the heavy path and come back faster and cheaper. Hard questions still get the full investigation. They just stop subsidising the easy ones.
One rule governs the whole thing: when triage is unsure, it picks the most thorough lane. If we cannot tell how hard a question is, we spend more to be safe. Saving tokens never outranks being right.
This is a piece we built ourselves. The harness gives you the machinery to run work at different depths. Deciding which depth a particular question deserves is application logic, and you have to write it.
Problem 2: Bad sources went straight into the answer
The problem
When the agent looked something up, whatever came back went directly into its context. If a search returned four articles and one was only loosely related, all four arrived carrying equal authority. The model had no way to separate a strong source from a weak one, so it treated them alike and answered confidently from the mixture.
This failure is worth naming precisely, because it usually gets mislabelled. The agent was not hallucinating. It was faithfully repeating what retrieval handed it. The fault was upstream of the model entirely.
The deep agent fix: a grader between the tool and the loop
We put a checking step between retrieval and reasoning. Every time the agent looks something up, that step grades how relevant the results actually are before any of them are allowed into context.
If the sources are good, they pass through. If they are bad, the system rewrites the query and searches again from a different angle. If the second attempt also comes up short, it flags the gap so the agent answers cautiously instead of confidently. If the results are mixed, it keeps the good ones and searches again to fill what is missing.
The important detail is that the grader does not know what answer the agent is hoping for. It judges sources on their merits alone, which means it cannot talk itself into approving weak research just because the weak research would be convenient.
This is the second piece we built. A deep agent harness controls what enters context and when. Judging whether a specific retrieval result is good enough for your domain is a call only you can make. This is the same instinct we applied one level down, when we rebuilt our crawler to return only the parts of a page that match the query.
Problem 3: The agent searched instead of thinking
The problem
The original loop could reason. Nothing made it. Handed a hard question, it would fall into a searching rhythm: query, read, query, read, never pausing to ask whether the last result actually helped or what was still missing.
Multi-part questions exposed this most clearly. Asked four things at once, the agent would answer two well, mention a third in passing, and quietly drop the fourth. Nothing anywhere in the architecture was keeping track of the fact that a fourth part existed.
The deep agent fix: make planning a step, not a suggestion
This is where a deep agent’s task planning earns its place. The harness offers a task list the agent maintains as it works, with every item tracked as pending, in progress, or complete. A four-part question becomes four tracked items, and none of them can go missing without it being visible.
We paired that with a forced rhythm: reflect before searching, and reflect again after every result before deciding the next move.
It sounds trivial. Forcing the rhythm is exactly what turns frantic searching into deliberate investigation. It also brought a benefit we did not expect to value as much as we now do. The reasoning is written down as structured state, so when an answer goes wrong we can see precisely where the thinking went wrong instead of guessing at it.
Problem 4: Long investigations buried their own findings
The problem
A hard question needs many lookups. In the original design, the result of every lookup piled into one shared context, and none of it ever left.
The effect was perverse: answers got worse the longer the agent worked. Early findings, often the most relevant ones, ended up buried under later raw material. By the time the agent sat down to write, the good material from step two was competing for attention with fifteen pages of noise from step nine. More effort, worse result.
The deep agent fix: delegate to subagents with their own context
This is the capability that made deep agents worth adopting rather than just patching the loop.
For the heaviest questions, the work now splits between a manager and a specialist. The manager holds the question and the plan. It delegates the actual searching to a subagent that runs in a completely separate context, works through its subtask alone, and hands back only the distilled findings.
The manager never sees the raw pile. Its context stays small and focused on what it is actually responsible for, which is the shape of the final answer. The subagent can go as deep as it needs to, because all of its clutter dies with it.
Alongside that, summarization and offloading run automatically. History gets compressed as it ages, and oversized tool results are moved out of context and into files the agent can read back on demand.
This part is native. Isolated subagent context, automatic summarization, and offloading to a filesystem are exactly what the harness is built to provide.
Problem 5: Every lookup returned the same amount of material
The problem
Retrieval was configured once, globally. Every search pulled back the same fixed number of sources. That number was simultaneously too small for “list everything we know about X” and far too large for “what time does the gate open.”
The deep agent fix: scale depth to the intent we already know
We were already classifying every question in Problem 1, and that classification carries real information about breadth. So we let it set retrieval depth as well. Pinpoint questions get a small, precise set. Broad questions get a wide sweep.
Same retrieval machinery, effort matched to the need, and no second classification pass to pay for.
What changed
The original design was not broken. It was undiscerning. It spent the same effort and ran the same strategy for every request, regardless of what the request actually was. Every fix we made added a piece of judgment.
The part we are happiest about is what did not change. The original ReAct agent still exists in our system, untouched. The deep agent slots in as the default and calls the old loop when the old loop is the right tool for the job. We swapped the brain without rewiring anything around it.
