The Protocol That Cleaned Up Our Agent Structure

Constructing Semantic Search with Transformers.js and Sentence Embeddings

GPU Time-Slicing for Concurrent LLM Brokers on Kubernetes

Just a few weeks in the past somebody from the information staff requested whether or not we may replace the database schema which was being populated by one of many instruments of our complicated agentic system. The replace is easy: two new columns are being added to the desk.

The instrument definition lived within the agent orchestrator. A second related model of it lived within the validation agent. A 3rd barely completely different and out-of-date model was in a utility module somebody had written three sprints in the past. The human-in-the-loop approval logic was wired immediately into graph edges, one customized implementation per instrument. Altering the schema meant touching 4 information, re-testing every agent individually, and hoping nothing downstream broke silently.

We fastened it however it raised one critical query: why did we construct it this fashion?

The trustworthy reply is that we had no various. Instrument calling in LangGraph is a neighborhood concern by design. You outline instruments the place you want them, you name them the place you name them and also you personal all of the plumbing. That is manageable when you might have solely two brokers however this turns into an issue when seven brokers are sharing overlapping instruments with a human gate.

After performing some analysis we determined that as a substitute of defining instruments domestically for each agent we must always use a shared useful resource that may host all our instruments and any agent can use them.

What’s MCP?

The Mannequin Context Protocol is an open commonplace revealed by Anthropic in late 2024. It standardises how an AI agent discovers and calls instruments. As an alternative of defining instruments contained in the orchestrator you run them on a separate server. The agent connects to that server at runtime, asks what instruments can be found, and will get a listing again.

A senior engineer studying this text will instantly ask: couldn’t I simply construct a centralised instrument registry and inject it into every agent at startup? I requested this to myself and used the instrument registry as a substitute of MCP in one other system.

Sure, you can, and if you have already got one thing like that working, MCP isn’t an emergency. What a bespoke registry doesn’t offer you is the interoperability boundary. MCP is a protocol, not a library. Any MCP-compatible shopper can connect with your server, LangGraph right now, a special framework subsequent 12 months. A TypeScript shopper can name your Python server with none additional integration work. A instrument registry doesn’t present this performance.

There’s additionally a staff possession level. In our case the ML staff owned the instruments, the appliance staff owned the graph. MCP gave them a clear contract with no shared codebase.

Constructing the MCP Server

An MCP server can expose three issues: Instruments (callable actions), Sources (read-only knowledge), and Prompts (reusable templates). For an agentic system that should take some actions, instruments are the first concern.

The Python SDK ships with FastMCP, which handles schema technology from kind hints and manages protocol lifecycle. You must write a operate and enhance it with a instrument decorator and the server takes care of the remainder.

One factor that catches folks out with stdio transport: by no means write to stdout. The MCP protocol makes use of stdout as its communication channel. Any stray print() name will corrupt the message stream in methods which can be very complicated to debug.

import sys
import logging
from mcp.server.fastmcp import FastMCP

logging.basicConfig(degree=logging.INFO, stream=sys.stderr)
logger = logging.getLogger("analyst-tools")

mcp = FastMCP("analyst-tools")

@mcp.instrument()
async def run_analysis(code: str, dataset: str) -> dict:
    """
    Executes a Python snippet towards dwell knowledge and returns the consequence.
    Use when the person needs to compute aggregates, filter information,
    or derive insights. The code should assign its closing output to a
    variable named 'output'.
    
    Args:
        code: Python code to execute.
        dataset: Considered one of 'gross sales', 'stock', 'pipeline'.
    """
    logger.information(f"run_analysis | dataset={dataset}")
    return await execute_in_sandbox(code, dataset)


@mcp.instrument()
async def write_to_db(desk: str, payload: dict) -> dict:
    """
    Persists a consequence report to the analyst outcomes desk.
    Solely name this after run_analysis has returned a verified output.
    
    Args:
        desk: Goal desk title.
        payload: Key-value pairs to jot down as a brand new report.
    """
    logger.information(f"write_to_db | desk={desk}")
    return await persist_result(desk, payload)


if __name__ == "__main__":
    mcp.run(transport="stdio")

The docstrings are utilized by the LLM to assist the agent determine which instrument to name. So, writing an excellent docstring is essential.

Stdio vs HTTP

This resolution comes up in each manufacturing deployment and most articles skip over it.

Stdio runs the server as a subprocess of the shopper. Communication occurs over commonplace enter and output. Latency is single-digit milliseconds, there’s no community concerned, and setup is minimal. The appropriate alternative for native improvement, single-machine deployments, or anyplace the server and shopper dwell in the identical course of tree.

Streamable HTTP runs the server as an unbiased service. Use this when the server must be shared throughout a number of purchasers or machines, while you wish to deploy it as a container, or while you want horizontal scaling. Serverless deployments like Cloud Run work effectively right here. Stdio doesn’t match the serverless mannequin in any respect as a result of it assumes a long-lived father or mother course of.

Switching between these in FastMCP is only one line:

mcp.run(transport="streamable-http", host="0.0.0.0", port=8080)

We simply have to alter the transport in mcp.run() and every part else stays the identical.

For knowledge residency necessities, an MCP server operating on-premise with instruments that by no means contact an exterior API offers you a clear story in your compliance staff. The protocol doesn’t care the place the server runs.

Connecting it to LangGraph

The langchain-mcp-adapters library manages the subprocess lifecycle, performs the instrument discovery handshake, and interprets MCP instrument schemas into LangChain-compatible instrument objects.

from langchain_mcp_adapters.shopper import MultiServerMCPClient
from langgraph.graph import StateGraph, MessagesState, START
from langgraph.prebuilt import ToolNode, tools_condition
from langchain_google_vertexai import ChatVertexAI

llm = ChatVertexAI(
    mannequin="gemini-2.5-flash",
    temperature=0,
    max_tokens=None
)

async def run(question: str):
    async with MultiServerMCPClient({
        "analyst-tools": {
            "command": "python",
            "args": ["./mcp_server.py"],
            "transport": "stdio",
        }
    }) as shopper:

        instruments = await shopper.get_tools()
        llm_with_tools = llm.bind_tools(instruments)

        def agent_node(state: MessagesState):
            return {"messages": [llm_with_tools.invoke(state["messages"])]}

        graph = StateGraph(MessagesState)
        graph.add_node("agent", agent_node)
        graph.add_node("instruments", ToolNode(instruments))
        graph.add_edge(START, "agent")
        graph.add_conditional_edges("agent", tools_condition)
        graph.add_edge("instruments", "agent")

        app = graph.compile()
        consequence = await app.ainvoke({
            "messages": [{"role": "user", "content": query}]
        })

        print(consequence["messages"][-1].content material)

tools_condition is a built-in LangGraph module that checks whether or not the final message incorporates instrument calls or not. If sure, path to the instrument executor and if no, we’re achieved. Utilizing it as a substitute of writing your individual routing operate issues as a result of it handles edge instances and implementation misses.

One behaviour price realizing: MultiServerMCPClient creates a brand new MCP session per instrument name by default. For a single request that makes 5 sequential instrument calls, that’s 5 handshakes. Positive for stdio on the identical machine, however noticeable on HTTP transport with a distant server. For manufacturing workloads with chained instrument calls, use async with shopper.session("analyst-tools") to pin a number of calls to at least one session.

Human-in-the-Loop on the Protocol Boundary

Earlier than MCP, our approval gate lived within the graph. We used interrupt_before on particular nodes, wired customized affirmation logic into graph edges, and up to date the UI each time a brand new delicate instrument was added. It labored however it additionally meant that including a instrument that required approval was a three-team co-ordination train.

After MCP, the gate strikes to a single layer between the LangGraph executor and the MCP shopper. Any instrument matching the sensitivity coverage hits the gate earlier than reaching the server. The graph has no data of it.

SENSITIVE_TOOLS = frozenset({"write_to_db", "send_notification", "trigger_webhook"})

async def gated_call(tool_name: str, arguments: dict, execute) -> dict:
    if tool_name in SENSITIVE_TOOLS:
        # In manufacturing: push to Slack / inner UI / audit queue
        print(f"nAPPROVAL REQUIRED {tool_name}")
        print(f"Arguments: {arguments}")
        resolution = enter("Approve? (y/n): ").strip().decrease()

        if resolution != "y":
            return {
                "standing": "rejected",
                "purpose": f"Operator declined '{tool_name}'."
            }

    return await execute(tool_name, arguments)

SENSITIVE_TOOLS is a single set, consulted for each instrument name no matter which agent triggered it. New delicate instrument added to the server? Add the title to this set. The graph doesn’t change. The approval UI doesn’t change. In our inner system we loaded this from a config file at startup. Product and compliance staff may replace it with no code deployment.

What can break in Manufacturing and Why?

Server crashes mid-execution. The shopper will obtain an error on the following instrument name. LangGraph’s ToolNode surfaces this again to the LLM as a instrument error message. Whether or not the mannequin recovers or loops in confusion is dependent upon your system immediate. At minimal, log the subprocess stderr individually so you may see what killed the server, with out it debugging is a guesswork.

The LLM calls the mistaken instrument. MCP doesn’t shield you from this. In case your instrument descriptions are obscure or overlap in which means, the mannequin will make the mistaken routing resolution. We spent appreciable time tuning the docstrings in our server particularly as a result of a poorly-worded description was inflicting write_to_db to get known as earlier than run_analysis had completed. Deal with instrument descriptions as a immediate engineering drawback.

Approval gate on long-running workflows. If a human must approve a instrument name and it takes 5 minutes, the agent graph is suspended ready. LangGraph helps persisting graph state through checkpointing, so you may let the method exit and resume when the choice arrives. That’s extra concerned than what’s proven right here however it’s the correct structure for workflows that may’t block a thread indefinitely.

Affect of MCP on our Agentic System

We migrated seven instruments on the server, three of them are approval-gated. The orchestrator that calls them has no data of what any of them do.

We fully eradicated the instrument duplication. Now, run_analysis is outlined precisely in a single place serving seven workflows concurrently. To replace the output schema we simply should make modifications within the server after which each shopper will decide up the change.

Including new capabilities grew to become quick. For instance we added a generate_visualisation instrument the next week and the agent was utilizing it the very subsequent day. No orchestrator modifications are made.

We ended up with one staff proudly owning the instruments, one other proudly owning the graph, and a transparent contract between them. When the analyst staff needs a brand new functionality, they discuss to the ML staff in regards to the server, not the appliance staff not the graph staff.

I wish to share one factor that MCP doesn’t repair: It received’t make unreliable instruments dependable. It received’t assist the LLM make higher routing choices in case your descriptions are dangerous. And it doesn’t change observability, you continue to must log instrument calls and hint execution paths. The construction makes these simpler to instrument, however the work continues to be yours.

Conclusion

By transitioning to MCP and shifting instruments out of our native agent orchestrator right into a devoted server, we cleaned up our codebase, decoupled our engineering constraints and made the entire agentic system simple to deploy.

Due to this transition our ML staff can now deploy and model instruments independently with out touching the appliance graph.

In case you loved this MCP deep dive, I’d encourage you to take a look at my ongoing sequence: The RAG for Enterprise Information Base at Hybrid Search and Re-ranking in manufacturing RAG.