I Constructed a Instrument-Calling Agent in Python. Right here’s How I Debugged It

is spectacular when it really works. However when it fails, the ultimate reply isn’t sufficient. You have to see the operate it selected, the arguments it despatched, and the consequence your code returned.

You probably have constructed a small agent round a product API, a retrieval endpoint, a database lookup, a climate service, or a Mannequin Context Protocol (MCP) instrument, you’ve gotten most likely seen the identical downside. The mannequin says it checked one thing. Possibly it did. Possibly it despatched malformed arguments. Possibly the API returned no consequence. Possibly the mannequin request failed earlier than your instrument ran in any respect. With out the message historical past, instrument arguments, returned payload, and remaining reply in a single place, you’re trusting a narrative as a substitute of inspecting a run.

OpenAI’s documentation lays out 4 steps: you describe a instrument, the mannequin requests it, you run the instrument, and you come back the consequence. That sample is beneficial, however the first demo typically skips the half that issues as soon as an agent touches actual work. Are you able to show what the mannequin requested, what Python executed, what failed validation, and whether or not the ultimate reply used the returned knowledge?

This text is constructed round that debugging downside. The agent calls public APIs, validates instrument arguments with JSON Schema, returns compact instrument payloads, information every instrument step, catches mannequin request failures, and might write the identical run to a Weights & Biases (W&B) Weave hint for evaluate. JSON means JavaScript Object Notation. On this article, JSON Schema is the contract that tells Python which instrument arguments are legitimate earlier than any instrument runs.

The tutorial downside is easy: construct an agent that may reply a climate query solely by calling actual instruments, then make each step inspectable. Climate is standing in for the service-backed duties builders normally give brokers: test a package deal, retrieve a buyer document, lookup stock, value an order, or name an inner API. The helpful query is whether or not the model-selected operate name truly occurred and returned usable knowledge.

The reader takeaway is direct: cease judging a tool-calling agent by the ultimate reply alone. Decide it by the mannequin request, schema validation, Python execution, compact instrument consequence, error path, and remaining reply collectively.

The loop is value constructing instantly as soon as earlier than adopting a bigger agent framework or wiring the identical instruments into MCP. Frameworks and MCP servers are helpful when you’ve gotten many instruments, routing guidelines, state, retries, or workforce conventions. The purpose right here is to grasp the message circulation earlier than abstraction hides it.

By the tip, you should have a script that may:

• outline instruments with JSON Schema for an OpenAI mannequin
• run a bounded instrument calling loop
• validate instrument names and arguments earlier than execution
• hold instrument outputs compact earlier than returning them to the mannequin
• return a structured error if the mannequin request itself fails
• seize run proof in console output and Weave

That’s the article’s edge. You get a runnable message loop you’ll be able to examine, break, confirm, and later change or wrap with MCP or an agent framework from a place of understanding.

The run ought to reply 4 questions

A tool-calling agent could reply, “I checked the API,” however the helpful questions begin after that sentence:

1. Which instrument did the mannequin request?
2. What arguments did it ship?
3. What did Python return?
4. Did the ultimate reply use the returned knowledge or conceal a failure?

The tutorial builds a small agent round these questions. The person asks whether or not to hold an umbrella in Lagos. The mannequin has to request a metropolis lookup, obtain coordinates, request climate, obtain a forecast, and reply from that returned knowledge. Each step is printed and will be traced.

In the event you can examine this small loop, the identical behavior carries into extra critical service-backed brokers. A refund agent ought to present the order lookup, coverage test, refund choice, and remaining message. A doc agent ought to present the search question, retrieved passages, and reply. An MCP instrument ought to nonetheless present the instrument identify, arguments, consequence, and error path.

What a instrument calling agent truly does

A instrument calling agent in Python is a loop pushed by a big language mannequin, or LLM. It lets the mannequin request named features, obtain their outcomes, and proceed with up to date messages.

That sounds near a chatbot, however the conduct is totally different. A chatbot receives textual content and returns textual content. A instrument calling agent receives textual content, could request motion, waits in your utility to execute that motion, reads the instrument consequence, after which decides what to say or do subsequent.

The necessary items are plain engineering objects:

• The mannequin decides whether or not it wants a instrument.
• The instrument is a Python operate owned by your utility.
• The schema specifies the arguments the instrument accepts.
• The messages are the operating document of the person request, mannequin instrument requests, instrument outcomes, and remaining reply.
• The agent loop is your Python code that retains the method operating till the mannequin stops requesting instruments.

A fundamental operate name is one request and one consequence. An agent loop is the repeated model. The mannequin can ask for one instrument, learn the consequence, ask for an additional instrument, and hold going till it has sufficient context.

The climate instance makes use of two instruments:

• geocode_city, which turns a metropolis identify into latitude, longitude, and nation.
• get_weather, which turns latitude and longitude right into a compact climate report.

In an actual utility, these features name exterior APIs. A package deal instrument would possibly name a delivery supplier. A flight instrument would possibly name an airline standing service. A purchasing instrument would possibly name a list system. On this article, the instruments use Nominatim from OpenStreetMap for geocoding and Open-Meteo for climate knowledge. These providers hold the instance actual whereas nonetheless being sufficiently small to learn. Additionally they don’t require API keys for this tutorial run, so the one key you want is the OpenAI key used for the mannequin name.

The message loop this text exposes

Your app sends the person message and the listing of accessible instruments to the mannequin. If the mannequin wants a instrument, it returns a structured request. Your Python utility reads that request, runs the matching operate, sends the consequence again as a instrument message, and asks the mannequin to proceed.

The newer OpenAI Responses API follows the identical thought. It additionally helps in-built instruments, together with internet search and file search, so the instrument will be supplied by OpenAI or by your personal utility.

The primary helpful thought is easy: the mannequin chooses, however your code executes.

That boundary issues. Your utility ought to nonetheless determine whether or not a requested instrument exists, whether or not the arguments match the schema, whether or not the decision is allowed, how a lot knowledge to return, what to log, and when to cease the loop.

Why begin and not using a framework

It’s value constructing one customized Python loop earlier than adopting a bigger agent framework. The primary model teaches you what is simple to examine. When you perceive the uncooked message circulation, you may make a greater choice about whether or not a framework removes complexity or hides it.

The identical thought applies to MCP. The official MCP documentation describes MCP as an ordinary means for functions to supply context to giant language fashions. An MCP server can expose instruments, sources, and prompts to an AI consumer, however the design questions stay the identical: What arguments are legitimate? What ought to the instrument return? What occurs when the mannequin request fails? What occurs when the instrument returns no consequence?

For this tutorial, the primary path makes use of the OpenAI Python software program growth equipment, or SDK. Native runtimes that help instrument calling, together with Ollama, observe the identical sample with totally different response codecs. The purpose right here is to construct the loop as soon as the place each object is seen.

Create one folder and arrange the setting

Create one working folder for this text, then run each setup and execution command from that folder. The folder will comprise the digital setting and openai_tool_calling_agent.py. Use Python 3.11 or newer. You don’t want a graphics processing unit, or GPU, as a result of the primary path calls a hosted OpenAI mannequin. The geocoding and climate instruments use public APIs that don’t require their very own keys.

On macOS or Linux, open a terminal in that working folder and run:

python3.11 -m venv .venv
supply .venv/bin/activate
python -m pip set up openai requests weave jsonschema

On Home windows PowerShell, open the identical working folder and run:

py -3.11 -m venv .venv
.venvScriptsactivate
python -m pip set up openai requests weave jsonschema

In the identical activated terminal, set your OpenAI key earlier than the mannequin run. On macOS or Linux, run:

export OPENAI_API_KEY="your_api_key_here"

On Home windows PowerShell, run:

$env:OPENAI_API_KEY="your_api_key_here"

In the identical activated terminal, if you need Weave tracing, log in to W&B:

wandb login

Confirm the Python model:

python --version

Anticipated output will look just like this:

Python 3.11.9

Save the entire runnable script

Save the next code as openai_tool_calling_agent.py in the identical working folder the place you created .venv. That is the one file readers have to create. The sections after the code clarify the design selections, however they don’t add any extra required code.

import argparse
import json
import os
from typing import Any

import requests
from jsonschema import ValidationError, validate
from openai import OpenAI, OpenAIError

strive:
    import weave
besides ImportError:
    weave = None


REQUEST_TIMEOUT = 10
USER_AGENT = "tool-calling-agent-python/1.0"
MODEL = os.getenv("OPENAI_MODEL", "gpt-4.1")


def geocode_city(metropolis: str) -> dict[str, Any]:
    response = requests.get(
        "https://nominatim.openstreetmap.org/search",
        params={"q": metropolis, "format": "jsonv2", "restrict": 1, "addressdetails": 1},
        headers={"Person-Agent": USER_AGENT},
        timeout=REQUEST_TIMEOUT,
    )
    response.raise_for_status()
    outcomes = response.json()
    if not outcomes:
        return {"error": f"Metropolis not discovered: {metropolis}"}

    first = outcomes[0]
    deal with = first.get("deal with", {})
    return {
        "metropolis": first.get("identify", metropolis),
        "nation": deal with.get("nation"),
        "latitude": float(first["lat"]),
        "longitude": float(first["lon"]),
    }


def get_weather(latitude: float, longitude: float, metropolis: str) -> dict[str, Any]:
    response = requests.get(
        "https://api.open-meteo.com/v1/forecast",
        params={
            "latitude": latitude,
            "longitude": longitude,
            "present": "temperature_2m,precipitation,rain,weather_code",
            "every day": (
                "weather_code,temperature_2m_max,temperature_2m_min,"
                "precipitation_sum,precipitation_probability_max"
            ),
            "forecast_days": 2,
            "timezone": "auto",
        },
        timeout=REQUEST_TIMEOUT,
    )
    response.raise_for_status()
    knowledge = response.json()
    present = knowledge.get("present", {})
    every day = knowledge.get("every day", {})

    def tomorrow_value(discipline: str) -> Any:
        values = every day.get(discipline) or []
        return values[1] if len(values) > 1 else None

    return {
        "metropolis": metropolis,
        "temperature_c": present.get("temperature_2m"),
        "precipitation_mm": present.get("precipitation"),
        "rain_mm": present.get("rain"),
        "weather_code": present.get("weather_code"),
        "tomorrow_weather_code": tomorrow_value("weather_code"),
        "tomorrow_temperature_max_c": tomorrow_value("temperature_2m_max"),
        "tomorrow_temperature_min_c": tomorrow_value("temperature_2m_min"),
        "tomorrow_precipitation_sum_mm": tomorrow_value("precipitation_sum"),
        "tomorrow_rain_chance_percent": tomorrow_value("precipitation_probability_max"),
    }


TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "geocode_city",
            "description": "Find latitude, longitude, and country for a supported city.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "City name, such as Lagos, London, or New York.",
                    }
                },
                "required": ["city"],
                "additionalProperties": False,
            },
        },
    },
    {
        "kind": "operate",
        "operate": {
            "identify": "get_weather",
            "description": "Get a compact climate report for a recognized location.",
            "parameters": {
                "kind": "object",
                "properties": {
                    "latitude": {"kind": "quantity"},
                    "longitude": {"kind": "quantity"},
                    "metropolis": {"kind": "string"},
                },
                "required": ["latitude", "longitude", "city"],
                "additionalProperties": False,
            },
        },
    },
]

TOOL_REGISTRY = {
    "geocode_city": geocode_city,
    "get_weather": get_weather,
}

SCHEMAS_BY_TOOL = {
    instrument["function"]["name"]: instrument["function"]["parameters"]
    for instrument in TOOLS
}


def compact_tool_result(consequence: dict[str, Any]) -> dict[str, Any]:
    if "error" in consequence:
        return {"error": consequence["error"]}

    allowed_keys = {
        "metropolis",
        "nation",
        "latitude",
        "longitude",
        "temperature_c",
        "precipitation_mm",
        "rain_mm",
        "weather_code",
        "tomorrow_weather_code",
        "tomorrow_temperature_max_c",
        "tomorrow_temperature_min_c",
        "tomorrow_precipitation_sum_mm",
        "tomorrow_rain_chance_percent",
    }
    return {key: worth for key, worth in consequence.gadgets() if key in allowed_keys}


def execute_tool_call(tool_name: str, tool_args: dict[str, Any]) -> dict[str, Any]:
    if tool_name not in TOOL_REGISTRY:
        return {"error": f"Unknown instrument: {tool_name}"}

    strive:
        validate(occasion=tool_args, schema=SCHEMAS_BY_TOOL[tool_name])
    besides ValidationError as exc:
        return {"error": "Invalid instrument arguments", "particulars": exc.message}

    strive:
        return TOOL_REGISTRY[tool_name](**tool_args)
    besides Exception as exc:
        return {"error": "Instrument execution failed", "particulars": str(exc)}


def maybe_trace(identify):
    if weave is None:
        return lambda fn: fn
    return weave.op(identify=identify)


@maybe_trace("run_agent")
def run_agent(user_prompt: str, max_turns: int = 4) -> dict[str, Any]:
    consumer = OpenAI()
    messages = [
        {
            "role": "system",
            "content": (
                "You are a concise weather assistant. "
                "Call tools only when they add facts needed for the answer."
            ),
        },
        {"role": "user", "content": user_prompt},
    ]
    transcript: listing[dict[str, Any]] = []

    for flip in vary(max_turns):
        strive:
            response = consumer.chat.completions.create(
                mannequin=MODEL,
                messages=messages,
                instruments=TOOLS,
            )
        besides OpenAIError as exc:
            return {
                "mannequin": MODEL,
                "user_prompt": user_prompt,
                "reply": "",
                "error": {
                    "kind": "model_request_failed",
                    "particulars": str(exc),
                },
                "transcript": transcript,
            }
        assistant_message = response.selections[0].message
        messages.append(assistant_message)

        tool_calls = assistant_message.tool_calls or []
        if not tool_calls:
            return {
                "mannequin": MODEL,
                "user_prompt": user_prompt,
                "reply": assistant_message.content material or "",
                "transcript": transcript,
            }

        for tool_call in tool_calls:
            tool_name = tool_call.operate.identify
            tool_args = json.hundreds(tool_call.operate.arguments)
            raw_result = execute_tool_call(tool_name, tool_args)
            tool_result = compact_tool_result(raw_result)

            transcript.append(
                {
                    "flip": flip + 1,
                    "instrument": tool_name,
                    "arguments": tool_args,
                    "consequence": tool_result,
                }
            )
            messages.append(
                {
                    "function": "instrument",
                    "tool_call_id": tool_call.id,
                    "content material": json.dumps(tool_result),
                }
            )

    return {
        "mannequin": MODEL,
        "user_prompt": user_prompt,
        "reply": "I couldn't end as a result of the agent reached its instrument name restrict.",
        "transcript": transcript,
    }


def confirm() -> dict[str, Any]:
    bad_arguments = execute_tool_call("get_weather", {"metropolis": "Lagos"})
    unknown_tool = execute_tool_call("lookup_package", {"tracking_id": "123"})
    schema_names = sorted(SCHEMAS_BY_TOOL)
    return {
        "standing": "okay",
        "mannequin": MODEL,
        "instruments": schema_names,
        "bad_arguments_check": bad_arguments,
        "unknown_tool_check": unknown_tool,
    }


def foremost() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--mode", selections=["verify", "run"], default="run")
    parser.add_argument(
        "--prompt",
        default="Ought to I carry an umbrella in Lagos tomorrow?",
    )
    parser.add_argument("--weave-project", default="")
    args = parser.parse_args()

    if args.mode == "confirm":
        print(json.dumps(confirm(), indent=2))
        return

    if args.weave_project:
        if weave is None:
            increase RuntimeError("Set up weave earlier than utilizing --weave-project.")
        weave.init(args.weave_project)

    consequence = run_agent(args.immediate)
    print(json.dumps(consequence, indent=2))


if __name__ == "__main__":
    foremost()

Run a preflight test earlier than spending tokens

Run this command from the identical working folder and the identical activated terminal. It checks imports, the instrument registry, JSON Schema validation, and unknown instrument dealing with with out utilizing the OpenAI API.

python openai_tool_calling_agent.py --mode confirm

Captured output from my preflight run:

{
  "standing": "okay",
  "mannequin": "gpt-4.1",
  "instruments": [
    "geocode_city",
    "get_weather"
  ],
  "bad_arguments_check": {
    "error": "Invalid instrument arguments",
    "particulars": "'latitude' is a required property"
  },
  "unknown_tool_check": {
    "error": "Unknown instrument: lookup_package"
  }
}

That is the primary purpose to construct the loop your self. Earlier than the mannequin is concerned, you’ll be able to test that the Python layer refuses an unknown instrument and catches a malformed get_weather name.

Run the agent in opposition to actual APIs

After setting OPENAI_API_KEY, run the complete instrument calling loop from the identical working folder and activated terminal:

python openai_tool_calling_agent.py --mode run --prompt "Ought to I carry an umbrella in Lagos tomorrow?"

The script prints JSON. It contains the mannequin identify, the person immediate, the ultimate reply, and a transcript of every instrument name. In my profitable climate run, the mannequin requested geocode_city first, then get_weather, then wrote the ultimate reply from the compact climate payload.

For readability, the identical captured run is formatted beneath as a step-by-step hint:

[MODEL]
gpt-4.1

[USER PROMPT]
Ought to I carry an umbrella in Lagos tomorrow?

[OPENAI REQUESTS TOOL turn 1]
{
  "arguments": {
    "metropolis": "Lagos"
  },
  "instrument": "geocode_city"
}

[PYTHON RUNS geocode_city]
{
  "metropolis": "Lagos",
  "nation": "Nigeria",
  "latitude": 6.4550575,
  "longitude": 3.3941795
}

[OPENAI REQUESTS TOOL turn 2]
{
  "arguments": {
    "metropolis": "Lagos",
    "latitude": 6.4550575,
    "longitude": 3.3941795
  },
  "instrument": "get_weather"
}

[PYTHON RUNS get_weather]
{
  "metropolis": "Lagos",
  "precipitation_mm": 0.0,
  "rain_mm": 0.0,
  "temperature_c": 27.7,
  "tomorrow_precipitation_sum_mm": 6.5,
  "tomorrow_rain_chance_percent": 84,
  "tomorrow_temperature_max_c": 28.9,
  "tomorrow_temperature_min_c": 24.6,
  "tomorrow_weather_code": 80,
  "weather_code": 3
}

[FINAL ANSWER]
Sure, it's best to carry an umbrella in Lagos tomorrow. There's a excessive likelihood of rain (84%) with about 6.5 mm of precipitation anticipated.

That output is the audit path. The mannequin didn’t magically know Lagos climate. It requested coordinates, your Python code fetched them, the mannequin requested a forecast, your Python code fetched a compact forecast, and the ultimate reply used that returned knowledge.

Run one messy immediate

A clear run proves the loop can end. It doesn’t show the loop is nice to debug when one thing goes flawed.

I attempted a messier immediate subsequent:

OPENAI_MODEL=gpt-4.1-mini python openai_tool_calling_agent.py --mode run --prompt "Ought to I carry an umbrella in Xqznotacity tomorrow? If that place isn't actual, inform me what failed."

The immediate was meant to train the geocoder path with a spot that ought to not exist. The primary failure appeared sooner than that: the mannequin request itself returned a server error. Earlier than including the OpenAIError handler, this crashed the script with a stack hint. After the change, the agent returned a structured failure:

{
  "mannequin": "gpt-4.1-mini",
  "user_prompt": "Ought to I carry an umbrella in Xqznotacity tomorrow? If that place isn't actual, inform me what failed.",
  "reply": "",
  "error": {
    "kind": "model_request_failed",
    "particulars": "Error code: 500 ... server_error"
  },
  "transcript": []
}

This can be a higher edge case than I anticipated. The primary boundary in a instrument calling agent is the mannequin request. If that fails, the appliance ought to return a helpful error as a substitute of hiding the issue behind a generic crash.

Add a Weave hint

To document the identical run in Weave, run this command from the identical working folder and activated terminal:

python openai_tool_calling_agent.py --mode run --weave-project wb-authors/tool-calling-agent-python --prompt "Ought to I carry an umbrella in Lagos tomorrow?"

The hint is beneficial as a result of it preserves the sequence that issues: person immediate, mannequin instrument request, validated Python name, compact instrument consequence, and remaining reply. I captured the output above from my very own run, and the identical run is logged on this Weave hint. The Weave tracing docs describe the identical evaluate sample for logged mannequin calls.

Earlier than you think about this primary model completed, evaluate the run output and the Weave hint. You need to see the mannequin request geocode_city, Python run the Nominatim lookup, the mannequin request get_weather, Python run the Open-Meteo forecast name, and the mannequin write a solution from the compact climate payload. That visibility is the baseline to protect earlier than including extra instruments, frameworks, evaluations, or dashboards.

How the script maps to the agent loop

The script has six items.

First, geocode_city and get_weather are slim instruments that decision actual providers. Nominatim turns a metropolis into coordinates, and Open-Meteo turns these coordinates right into a forecast. Nominatim additionally asks public purchasers to ship a transparent person agent string, which is why the script units USER_AGENT.

Second, TOOLS describes these features with JSON Schema. The schema is the contract the mannequin sees. It says which operate exists, what arguments it accepts, which fields are required, and whether or not additional fields are allowed.

Third, TOOL_REGISTRY and execute_tool_call hold execution inside your utility. The mannequin can request a instrument, however Python decides whether or not the instrument is understood, whether or not the arguments match the schema, and what structured error to return when one thing is flawed.

Fourth, compact_tool_result removes fields the mannequin doesn’t want. Instrument outputs are messages again to the mannequin, not full API dumps. Compact payloads make the reply cheaper to supply and simpler to examine later.

Fifth, run_agent retains a bounded loop. It sends messages and gear schemas to the mannequin, receives instrument calls, executes the matching Python features, appends instrument outcomes, and stops when the mannequin returns a standard reply or the loop reaches max_turns.

Sixth, the OpenAIError handler turns mannequin request failures into structured output. That retains server errors, connection failures, or authentication errors seen to the caller as a substitute of burying them in a traceback.

That’s the fundamental loop most groups want to grasp earlier than adopting a bigger framework. Frameworks are simpler to judge after you’ve gotten seen the uncooked message circulation as soon as.

Reliability begins the place the loop is seen

Manufacturing instrument calling brokers fail on the edges. A mannequin request can fail earlier than a instrument is chosen. Arguments arrive within the flawed format. A instrument occasions out. A mannequin picks the flawed operate. A name repeats with out including info. The entire script handles the primary guardrails instantly: mannequin request errors, schema validation, unknown instrument errors, request timeouts, compact instrument outcomes, and a loop restrict.

A helpful preflight test is already constructed into --mode confirm. It calls get_weather with solely a metropolis, despite the fact that the schema requires latitude, longitude, and metropolis. The script returns a structured error as a substitute of operating a damaged instrument name. It additionally checks that an unknown instrument identify returns a structured error.

That small preflight path issues. It lets readers test the Python layer earlier than they spend cash on mannequin calls, and it provides groups a spot so as to add extra checks later. The messy immediate provides the opposite facet of the reliability story: reside mannequin calls can fail too, so the agent ought to make that failure seen.

The place this leaves the agent

You now have the essential instrument calling agent sample in Python: outline slim instruments, describe them with schemas, let the mannequin request them, execute solely recognized features, return compact outcomes, deal with mannequin request failures, and hold the loop bounded.

The climate instance is just one use case. The identical loop can name a doc search instrument, a buyer database, a pricing service, a take a look at runner, or an inner workflow API.

Do one factor earlier than you add extra instruments: run the agent with a immediate that ought to fail. Ask for an unsupported metropolis. Break one argument. Return an outsized payload. Watch what the loop does.

That may inform you greater than one other clear, completely satisfied path.

The identical intuition applies past instrument calling. In a follow-up piece on debugging coding brokers, I apply the identical evidence-first strategy to an agent that edits UI code, and recording what it inspected, patched, and verified as a substitute of trusting its personal “fastened”.

The helpful lesson is greater than climate. Instrument calling begins as a loop you’ll be able to examine earlier than it turns into an structure choice. As soon as the loop is seen, a framework or MCP choice turns into simpler: undertake one when it removes repeated routing, state, retries, instrument packaging, or observability work, after you perceive what it hides. The ultimate reply is just one a part of the run.

Chosen Sources