How we structure LangGraph agents in production

Production agents are workflows before they are personalities. We start by defining the state that must survive every step: the original request, trusted context, routing decision, tool results, review findings and final status. A typed state model makes the graph inspectable and gives every node a narrow contract.

Prompts then become implementation details inside those contracts. This prevents a model response from quietly becoming the database, router and user interface at the same time.

Routing should be deterministic wherever the business already knows the rule. We use model classification only where language genuinely needs interpretation, validate its output against a small schema, and keep a safe fallback. Each specialist receives the same task envelope, but only the context and tools required for its role.

An explicit graph also makes failures legible. Operators can see whether a task failed during classification, tool execution, validation or review instead of receiving one generic model error.

A reviewer should judge evidence rather than confidence. Our review nodes inspect diffs, test output, citations or structured results against task-specific criteria. A failed review returns actionable findings to the executor with a bounded retry count; it does not silently approve malformed output or retry forever.

Human approval remains a first-class node for consequential actions. The system can prepare a deployment, message or record change, but the graph pauses with enough context for a person to decide.

Every run needs correlation IDs, timing, model and tool metadata, token or cost signals, and a durable final status. Representative evaluation cases run when prompts, models or tools change. That discipline turns an impressive prototype into a system a team can debug, improve and trust.

A tool should expose the smallest useful operation, not unrestricted access to an entire system. A research agent may read approved sources but should not publish. A coding agent may edit inside one repository but should not control unrelated processes. A communications agent can draft a message while the actual send remains behind approval. These boundaries reduce the damage a plausible but incorrect decision can cause.

We also separate retrieval from action. Reading customer context and changing a customer record are different capabilities with different audit and permission requirements. Tool inputs use strict schemas, outputs are validated, and sensitive identifiers are carried in trusted state rather than reconstructed from model prose. The resulting graph is slightly more explicit, but significantly easier to secure and review.

Long-running workflows need to survive provider failures, process restarts and human pauses. We persist state after meaningful transitions so execution can resume from a known checkpoint instead of repeating completed work. The checkpoint records what ran, which inputs it used and whether an external side effect occurred.

Idempotency matters whenever a node can create a ticket, write a database row or send a notification. We assign stable operation keys and make the receiving integration recognise duplicates. A retry can safely reconstruct a calculation, but it must not charge a card or notify a customer twice. This is ordinary distributed-systems discipline applied to probabilistic software.

A single successful demo says very little about reliability. Before changing a prompt, model or tool, we run a set of cases that represent normal work, ambiguous requests, missing context, policy boundaries and known historical failures. Each case has observable acceptance criteria: the correct route, required evidence, prohibited action or expected structured fields.

Evaluation includes more than answer similarity. We measure tool selection, schema validity, groundedness, latency, cost and whether the graph escalated when it should. Failed production runs become new regression cases after sensitive information is removed. Over time, the evaluation set becomes an operational memory of what the system must not forget.

Not every node needs the strongest model. Classification and formatting can often use a smaller model, while architecture decisions or difficult synthesis justify a more capable one. We set node-level timeouts, cap retry counts and record token use so an apparently helpful loop cannot consume an unbounded budget.

Parallel work helps only when branches are independent and the merged result has a clear contract. Otherwise concurrency adds cost and makes traces harder to understand. We optimise after observing real runs: cache stable context, shorten repeated instructions, avoid sending irrelevant history and reserve premium reasoning for decisions where it changes the outcome.

A production launch includes owners, dashboards and a rollback path. Operators need to know how to pause new work, inspect a failed run, resume from a checkpoint and disable one tool without taking the whole graph offline. Alerts should distinguish provider outages from validation failures and business-rule rejections.

We release changes through a small pilot group, compare results with the existing workflow and expand autonomy only when evidence supports it. The best agent systems do not attempt to remove people from every step. They remove repetitive coordination while making the remaining human decisions faster, better informed and easier to audit.