In my newest submit, methods to get structured, machine-readable outputs as a response from an LLM, utilizing JSON Mode, perform calling, and structured outputs. In that submit, we briefly touched on the concept of perform calling, approaching it as a technique for acquiring structured responses. Nonetheless, perform calling is one thing that goes nicely past simply getting structured knowledge again from a mannequin, since it’s primarily the spine of agentic AI workflows. So, in right this moment’s submit, we’re going to take a more in-depth take a look at precisely this subject.
In the entire examples we have now lined up to now, the LLM is simply used as a passive responder, that means it receives a query after which generates a solution, and that’s it. However what if we would like the LLM not simply to reply with one thing however as a substitute to do one thing? Or to place it extra exactly, what if we would like an motion to be triggered based mostly on the mannequin’s response? This motion could also be something: lookup into dwell knowledge, ship a message, question a database, name an exterior API, and so forth.
That is made doable with software calling. Software calling is what transforms an LLM from a really sensible textual content generator into one thing that may truly set off actions and work together with the world round it.
So, let’s have a look!
What’s Software Calling?
Software calling (additionally referred to as perform calling) is the mechanism by which an LLM can request the execution of exterior capabilities or APIs as a part of producing its response. In different phrases, as a substitute of simply returning textual content, the mannequin can execute a selected perform with particular arguments, as a response to the consumer’s request.
The important thing factor to know right here is that the mannequin itself doesn’t execute the software. It solely decides which software to name and with what arguments. The precise execution of the chosen software occurs in our personal code, during which the request to the AI mannequin is included. We then feed the software’s end result again to the AI mannequin, which makes use of it to generate a ultimate response to the consumer.
That is the software calling loop, which incorporates the next steps:
- The consumer submits a message
- The AI mannequin takes the message as enter and produces an output, which is actually a call on which software to utilise and with which arguments
- The mannequin’s response containing the software choice and respective arguments for use is handed again to the code. The code – with no involvement of the AI mannequin – executes the chosen software with the chosen arguments. This execution produces some sort of end result (e.g., a calculation, data obtained from an API, and so on.), and this result’s then handed again to the AI mannequin.
- The AI mannequin takes as enter the results of the software and produces a ultimate response to the consumer based mostly on that.
Once more, the mannequin generates a software name, not a software execution. The 2 are very various things, and conflating them is without doubt one of the commonest sources of confusion.
However what precisely is a software name? In observe, it signifies that the mannequin returns a structured, machine-readable response utilizing Operate Calling, as we noticed within the earlier submit. On this response, content material is None; there isn’t any pure language reply, only a structured instruction indicating which software to name and with what arguments. It’s only after we execute the software and move the end result again that the mannequin generates an precise textual content response for the consumer.
However let’s see this in observe!
We’ll begin with a easy instance utilizing only one software and one name, after which progressively construct as much as some extra fascinating eventualities.
1. A single software: climate API
I believe that the commonest instance of software use with AI that involves thoughts is a climate API (the cornerstone of customized, dwell knowledge), so let’s think about we’re constructing a climate assistant. Specifically, we need to create a mechanism during which the consumer asks in regards to the climate, and as a substitute of simply letting the AI mannequin make one thing up (which the mannequin would very fortunately do 🙃), we would like it to name an actual climate perform and get precise knowledge in regards to the climate from some place else, exterior the LLM. To get the climate knowledge, I shall be utilizing Open-Meteo, a free, open-source climate API that fortunately requires no API key.
To make use of a software, we have now to initially declare it in instruments.
from openai import OpenAI
import json
consumer = OpenAI(api_key="your_api_key")
# Step 1: outline the software
instruments = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather for a given city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The name of the city, e.g. Athens"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to make use of"
}
},
"required": ["city"]
}
}
}
]Discover how the precise software for use (the climate API) is talked about nowhere up up to now. As a substitute, the mannequin decides which software to name based mostly on three issues: the perform description (“Get the present climate for a given metropolis”), the parameter descriptions (“The title of town, e.g., Athens”), and the enforced schema. It’s purely from this data that the mannequin figures out whether or not that is the proper software to name for a given consumer message and with what arguments. Thus, writing clear and correct descriptions when defining our instruments is of key significance for the mannequin to efficiently establish and name the proper software based mostly on the consumer’s enter.
So, after we have now outlined the instruments variable, we will then make a request to the AI mannequin:
# Step 2: ship the consumer message together with the software definition
messages = [
{"role": "user", "content": "What's the weather like in Athens right now?"}
]
response = consumer.chat.completions.create(
mannequin="gpt-4o-mini",
instruments=instruments,
messages=messages
)
print(response.decisions[0].message)Right here’s what occurs once we make this request. The mannequin reads the consumer’s message, “What’s the climate like in Athens proper now?”, and understands that the out there software get_current_weather may also help reply this question with actual, dwell knowledge. So, moderately than producing a textual content response instantly, it decides to name the software first. Extra particularly, the mannequin’s response at this level appears to be like like this:
ChatCompletionMessage(
content material=None,
position='assistant',
tool_calls=[
ChatCompletionMessageToolCall(
id='call_abc123',
type='function',
function=Function(
name='get_current_weather',
arguments='{"city": "Athens", "unit": "celsius"}'
)
)
]
)Discover how content material is None, as a result of the mannequin isn’t returning a textual content response, however a software name. Now it’s our job to truly execute the software, the mannequin chosen, and return the end result again to it. In our case, that is going to be making the API request to the climate API, utilizing the arguments (that’s, town and unit of measurement) supplied within the AI mannequin’s response:
# Step 3: execute the software utilizing the Open-Meteo API
import requests
def get_current_weather(metropolis: str, unit: str = "celsius"):
# geocode town title to coordinates
geo = requests.get(
"https://geocoding-api.open-meteo.com/v1/search",
params={"title": metropolis, "rely": 1}
).json()
lat = geo["results"][0]["latitude"]
lon = geo["results"][0]["longitude"]
# fetch present climate
climate = requests.get(
"https://api.open-meteo.com/v1/forecast",
params={
"latitude": lat,
"longitude": lon,
"present": "temperature_2m,weather_code",
"temperature_unit": unit
}
).json()
temp = climate["current"]["temperature_2m"]
return {"metropolis": metropolis, "temperature": temp, "unit": unit}
# extract the software name from the response
tool_call = response.decisions[0].message.tool_calls[0]
arguments = json.masses(tool_call.perform.arguments)
# name the precise perform
weather_result = get_current_weather(**arguments)we will then append the software’s end result to the message historical past after which ship every part again to the mannequin:
# Step 4: add the assistant's software name AND the software end result to the message historical past
messages.append(response.decisions[0].message) # necessary: append the software name first
messages.append({
"position": "software",
"tool_call_id": tool_call.id, # hyperlinks the end result again to the particular software name
"content material": json.dumps(weather_result)
})
# Step 5: ship every part again to the mannequin for a ultimate response
final_response = consumer.chat.completions.create(
mannequin="gpt-4o-mini",
instruments=instruments,
messages=messages
)
print(final_response.decisions[0].message.content material)And now, we lastly get a correct textual content response:
It is at present 29°C in Athens. Feels like a terrific day to be exterior!🍨 DataCream is a e-newsletter providing tales and tutorials on AI, knowledge, and tech. If you’re serious about these subjects, subscribe right here!
2. Letting the mannequin select from a number of instruments
Now let’s check out a extra real looking instance. In a real-world agentic software, the mannequin sometimes has entry to not one, however a number of instruments, and consequently, it wants to determine which one (or ones) have to be used based mostly on what the consumer is asking.
Let’s lengthen our preliminary climate API instance by including a further software for currencies. For this, we’ll use Frankfurter, a foreign money API offering European Central Financial institution every day charges, once more with no API key requirement. So, let’s replace our instruments variable by including a second software for changing currencies:
instruments = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather for a given city",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "The name of the city"},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
},
"required": ["city"]
}
}
},
{
"kind": "perform",
"perform": {
"title": "convert_currency",
"description": "Convert an quantity from one foreign money to a different",
"parameters": {
"kind": "object",
"properties": {
"quantity": {"kind": "quantity", "description": "The quantity to transform"},
"from_currency": {"kind": "string", "description": "The supply foreign money code, e.g. USD"},
"to_currency": {"kind": "string", "description": "The goal foreign money code, e.g. EUR"}
},
"required": ["amount", "from_currency", "to_currency"]
}
}
}
]And in addition arrange the precise convert_currency perform utilizing the Frankfurter API:
def convert_currency(quantity: float, from_currency: str, to_currency: str):
response = requests.get(
f"https://api.frankfurter.dev/v2/fee/{from_currency}/{to_currency}"
).json()
fee = response["rate"]
transformed = spherical(quantity * fee, 2)
return {
"quantity": quantity,
"from_currency": from_currency,
"to_currency": to_currency,
"converted_amount": transformed,
"fee": fee
}On this manner, the mannequin can deal with a a lot wider vary of consumer requests; it will probably now additionally reply about currencies, on high of the climate 😋. Now, if the consumer asks “What’s the climate in Athens?”, the mannequin ought to name get_current_weather. In the event that they ask “How a lot is 100 USD in EUR?”, it ought to name convert_currency. And if we ask one thing irrelevant to each climate and currencies for which neither of the out there instruments may also help, the mannequin will merely reply in textual content with out calling any software in any respect.
However let’s see this in motion:
messages = [
{"role": "user", "content": "How much is 200 USD in EUR?"}
]
response = consumer.chat.completions.create(
mannequin="gpt-4o-mini",
instruments=instruments,
messages=messages
)
tool_call = response.decisions[0].message.tool_calls[0]Let’s take a look on the response:
print(tool_call.perform.title) from which we get convert_currency. So, the mannequin understood that the query “How a lot is 200 USD in EUR?” is related to the convert_currency software. Let’s additionally check out the arguments:
print(tool_call.perform.arguments) from which we get
'{"quantity": 200, "from_currency": "USD", "to_currency": "EUR"}'
So, the mannequin appropriately identifies convert_currency as the proper software and fills within the acceptable arguments, with out us doing something apart from offering acceptable software descriptions, and the consumer offering an acceptable message. This actual decision-making mechanism is what makes tool-calling the muse of agentic techniques.
3. Calling a number of instruments without delay
One other fascinating software calling situation is that many fashions, like gpt-4o, can name a number of instruments in a single response when the consumer’s request requires it. This is called parallel software calling.
For instance, let’s think about a situation the place the consumer asks in a single request one thing that requires the usage of each the get_current_weather and convert_currency instruments to acquire the required data:
messages = [
{"role": "user", "content": "What's the weather in Athens and how much is 100 USD in EUR?"}
]
response = consumer.chat.completions.create(
mannequin="gpt-4o-mini",
instruments=instruments,
messages=messages
)
for tool_call in response.decisions[0].message.tool_calls:
print(tool_call.perform.title)
print(tool_call.perform.arguments)On this case, the response we get is the next:
get_current_weather
{"metropolis": "Athens"}
convert_currency
{"quantity": 100, "from_currency": "USD", "to_currency": "EUR"}Discover how each instruments are referred to as in a single mannequin response. We will then execute the respective instruments with the supplied arguments and move again the software outcomes to the mannequin collectively. That is rather more environment friendly than sequential calls, and it’s how extra superior brokers deal with multi-part requests.
On my thoughts: So, what makes this agentic?
One factor that has all the time gotten on my nerves is the time period “agentic” being slapped on every part. Brokers, agentic workflows, something originating from the phrase agent could be very attractive these days, however as you might have already found your self, not every part offered as agentic actually is.
So let’s take a step again and take into consideration what an agent truly is within the first place. At its core, an agent is one thing that perceives its setting, processes that data ultimately, has a purpose, after which decides what motion to take as a way to obtain it. Take into consideration what our software calling mechanism is doing: it perceives the instruments out there, decides which one is acceptable to handle the consumer’s request (if any), and passes that call on to the remainder of the code for execution. That, in its easiest kind, is company.
In real-world agentic purposes, the software calling loop runs not one however a number of instances, with the mannequin utilizing the outcomes of 1 software name to determine whether or not, and which, software to name subsequent. That is generally referred to as a ReAct loop (Motive + Act), and it’s what permits brokers to deal with advanced, multi-step duties that may’t be solved in a single name.
In the end, what I discover most fascinating about software calling is the way it adjustments the character of what an LLM is. Up up to now, a language mannequin was primarily a very refined input-output perform, which takes textual content as enter and generates textual content as output. However with the software calling, we acquire entry to an limitless assortment of further functionalities, which we will mix with the reasoning energy of the LLM to create techniques which can be much more succesful than both alone.
✨ Thanks for studying! ✨
Should you made it this far, you may discover pialgorithms helpful — a platform we’ve been constructing that helps groups securely handle organizational data in a single place.
Cherished this submit? Be a part of me on 💌Substack and 💼LinkedIn
All photos by the writer, besides talked about in any other case.
