You’ve most likely already seen the time period determination mannequin very often in your feed currently.
The concept of this new breed of AI is easy: you give the mannequin some context and a set of choices, and it picks one.
An LLM can do this too, however determination fashions are educated particularly for this job, and so they can do it a lot sooner and cheaper than LLMs.
Okay, however how do you truly put one to work in apply?
In my expertise, a hybrid workflow is an effective place to start out.
Principally, you let the choice mannequin deal with the closed-ended alternatives, the place the choices are already recognized, and also you let the LLM deal with the open-ended components, the place the reply needs to be reasoned out or generated from scratch. On prime of that, you’ve gotten Python tying every part collectively by managing the workflow and executing the operations.
Every part does what it does finest.
That’s what we’ll construct on this submit.
Particularly, we’ll create a browser automation workflow. After receiving a journey request in pure language, the workflow will navigate a hotel-search web site, choose an acceptable resort, and put together an inquiry.
We’ll see concretely the way to make that occur by combining a choice mannequin, an LLM, and Python into one workflow, and see whether or not utilizing a choice mannequin truly makes our instance sooner than utilizing an LLM for every part. (We’ll measure it!)
···
1. The Hybrid Workflow We Wish to Construct
1.1 What does browser automation contain?
Browser automation is likely one of the most sensible issues you are able to do with LLMs: an LLM that may function a browser can attain nearly something you’ll be able to.
You describe what you need in pure language, like “discover me a resort in Lisbon subsequent weekend beneath €150 an evening” or “obtain final month’s invoices from the provider portal.” The LLM system then interacts with the web site, like studying the web page, clicking buttons, selecting dropdown choices, and filling out types, to get it carried out.
The LLM system must carry out various actions. But when we have a look at them extra intently, we will broadly manage them into three classes:
-
Closed-ended choice. Which button to click on? Which merchandise to pick out? The choices are already there. The system wants to select the suitable one. This may additionally apply to selecting the subsequent workflow motion, when the out there actions are recognized prematurely.
-
Open-ended technology. What to write down in a message? The right way to clarify a request? Right here, the system must work out and generate a solution, which could possibly be a message, an evidence, a plan, or code.
-
Orchestration. To assist the above actions, we’d like a mechanism to assemble data from the web site, put together it for the fashions, take the fashions’ responses, and really carry out the browser operations, with exterior instruments.
Actually, if you happen to’ve constructed LLM functions earlier than, you’ve most likely handled all three already.
1.2 Who does what within the hybrid workflow?
Now, with these three classes clearly laid out, the division of labor follows naturally.
The choice mannequin can deal with closed-ended choice, the LLM can deal with open-ended technology, and Python handles orchestration.
Observe that closed-ended choice occurs at two ranges right here: inside an interplay, and throughout interactions.
Inside an interplay, the choice mannequin merely picks an possibility, like a date in a calendar. Throughout interactions, it as an alternative picks what to do subsequent, similar to whether or not to open the calendar, or submit the shape. Basically, each are the identical sort of drawback: choosing from a recognized set of choices already on the web page. So a choice mannequin can naturally deal with each. Nevertheless, this second stage is what lets it drive the workflow.
Now, we will put collectively a repeating cycle:
-
Learn the web page and put together the alternatives. Python gathers the present browser data and turns the out there controls into operations the choice mannequin can select from.
-
Choose the subsequent operation. The choice mannequin chooses what to do subsequent, similar to clicking a button, choosing an possibility, or filling a discipline. It may possibly additionally select to cease.
-
Name the LLM if wanted. Python checks the chosen operation. If it requires textual content, it calls the LLM to provide that textual content.
-
Perform the operation. Python makes use of the browser instruments to carry out the chosen operation. This consists of getting into any textual content equipped by the LLM.
-
Learn the up to date web page and repeat. The interplay adjustments the browser state; Python then fetches the brand new data and prepares new decisions for the subsequent cycle.
Subsequent, we put this workflow to work on a small hotel-search web site.
···
2. Case Examine: Discover a Lodge and Put together an Inquiry
2.1 What are we asking the workflow to do?
On this case examine, we work with the next journey request:
Can you discover us a resort in Lisbon for Nov 12–14, 2026? Two adults, max €150 an evening, refundable, breakfast included. If there’s a match, save an inquiry beneath Alex Morgan ([email protected]). Preserve it brief and well mannered: ask if we will verify in late, round 23:30, and if they’ve a quiet room. If nothing works, simply depart it.
For this case examine, I created a small native hotel-search web site with fictional inns in Lisbon and Porto.
The web site has the same old controls like a vacation spot dropdown, check-in/out calendars, and a visitor selector. The outcomes present resort costs, cancellation insurance policies, in addition to breakfast availability. One can even use the filters to slender the choice.
One factor to bear in mind: I designed inns to satisfy completely different combos of necessities. For instance, some are cheaper however non-refundable; others embody breakfast however exceed the price range. This manner, the workflow wants to contemplate the entire request when selecting a resort.
After choosing a resort, the workflow ought to open an inquiry type to enter the contact particulars and message. Lastly, it ought to open the overview web page, verify the affirmation field, and save the inquiry domestically.
2.2 How can we join the fashions to the browser?
Bear in mind the cycle we outlined in Part 1.2? Let’s undergo it once more, this time taking a look at how every step truly works.
1. Learn the web page and put together the alternatives.
To learn the web page, we use Playwright MCP, a Mannequin Context Protocol server from Microsoft that wraps the Playwright browser automation library.
With Playwright MCP, we will management a browser via a typical set of instruments. Observe that we do not work with web page screenshots. As an alternative, one in all these instruments captures the present state of the web page as a snapshot, which is a structured textual content description of its content material and controls. Every management will get a brief ID, which the opposite instruments use to know which factor to click on or kind into.
From the snapshot, Python turns the out there controls into a listing of concrete operations. To point out an instance:
Discover that this checklist covers each ranges of choice from earlier. A1 and A2 select a price inside one interplay, whereas A3 chooses which interplay comes subsequent. However to the choice mannequin, they’re all simply choices.
Behind every identifier, Python shops the browser instrument and arguments wanted to hold it out.
2. Choose the subsequent operation.
That is the place the choice mannequin comes into play. It receives the duty and web page data as context, in addition to the checklist of choices, after which it returns one identifier, say A3.
3. Name the LLM if wanted.
Typically the choice mannequin selects an operation that requires textual content, like “Fill Message to the resort from the duty.” The choice mannequin solely decides that this discipline needs to be stuffed now; it doesn’t write the content material. So Python passes the duty to the LLM, which writes the textual content.
4. Perform the operation.
Python appears up the instrument and arguments behind the chosen identifier and calls the corresponding Playwright MCP instrument. Utilizing the earlier A3 for example, this implies clicking the check-in button.
5. Learn the up to date web page and repeat.
As soon as Python executes the operation, the web page state adjustments. Python takes a brand new snapshot and builds a brand new checklist. This cycle continues till the choice mannequin selects a stopping possibility.
Now, let’s see the code that connects these items.
···
3. Construct the Hybrid Workflow
For our present case examine, we’ll primarily use Determination-1 from Microsoft as the choice mannequin and GPT-6-Luna because the LLM, each of that are deployed via Azure. We’ll additionally briefly cowl different options, like Jev and OpenAI’s Selections API. As talked about earlier, we use Playwright MCP for browser instruments.
Yow will discover the whole implementation within the companion pocket book right here.
3.1 Overview
We put the repeating cycle from Part 2 right into a operate referred to as run_browser_task(). This operate connects the fashions to the browser and retains the interactions shifting till the duty finishes.
The core of the operate appears like this:
The logic follows the supposed steps intently:
-
observe()reads the web page and prepares the alternatives, -
choose()calls the mannequin to decide on an operation, -
write_field()calls the LLM to provide textual content, -
Python executes the operation via
browser.call_tool()and repeats the cycle
Under, we unpack these items one after the other.
3.2 Flip browser controls into decisions
First, we have to learn the present web page and put together the operations. We wrap this in observe():
Within the code snippet above, the browser argument is our linked MCP shopper. Its job is to speak with Playwright MCP, which is a server that exposes Playwright’s browser capabilities as instruments Python can name.
Getting Playwright MCP working
To run the pocket book, you should have Python 3.11+, Node.js 20+, and Google Chrome put in. Node.js runs the Playwright MCP server, and Python connects to it via the MCP shopper.
Set up the Python MCP shopper and the Playwright MCP server:
python -m pip set up mcp==2.3.0
npm set up @playwright/[email protected]
You don’t must launch the MCP server individually. Once we name
run_browser_task(), it begins the server and opens the shopper connection. That connection is thebrowserobject handed toobserve().
The decision browser.call_tool("browser_snapshot", {}) asks the server to learn the present web page. The primary argument is the instrument identify, the second argument provides arguments for the instrument—none are wanted right here.
To parse the response, we ready a small tool_text() helper that extracts the textual content describing the web page and checks for reported instrument errors. The common expression then retrieves the snapshot from its YAML block.
Subsequent, we cross the snapshot to available_actions(). That is the helper operate we outlined to transform the noticed controls into operations:
Right here, buttons turn into clicks, dropdown choices turn into alternatives, and textual content fields turn into fill operations marked with a discipline label; the principle loop calls the LLM when one is chosen. We additionally add carried out and blocked as stopping decisions.
Every alternative pairs an outline for the choice mannequin with the instrument and arguments Python must execute it. For instance, an entry in its returned dictionary appears like this:
A3 is the operation’s identifier. The choice mannequin sees its description, “Click on Examine-in: Select date,” and may choose it by returning A3. Python then retrieves the execution particulars: name browser_click with {"goal": "e12"}. Right here, e12 is an instance reference equipped by the browser snapshot.
Since this operation wants no textual content, the discipline is None. For a text-field operation, it incorporates the sector label.
Okay, that is it for step one. With the 2 issues returned byobserve(), i.e., the web page snapshot and the operation dictionary, we will subsequent job the choice mannequin to pick out an operation.
3.3 Choose an operation with a choice mannequin
We wrap the API name in choose():
Within the code snippet above, http is the HTTPX shopper opened by run_browser_task(), which sends our mannequin requests. state incorporates the journey request, present web page snapshot, and up to date actions. decisions incorporates solely the operation identifiers and descriptions.
Within the request physique, the choice mannequin can see the context through state. We identify our query motion, set its kind to alternative, and supply the options via standards. That is the request format utilized by Microsoft-Determination-1’s determination endpoint on Azure.
We additionally want to inform the mannequin the way to make that choice. That’s the function of INSTRUCTIONS:
Lastly, request_json() is our pocket book helper that sends the JSON request and returns the parsed response along with request time.
The response makes use of the identical query identify, motion. Its alternative discipline incorporates the chosen identifier—for instance, A3. Python makes use of that identifier to retrieve the corresponding operation from actions.
We now have a particular operation.
3.4 Different methods to name a choice mannequin
Azure isn’t the one possibility. For instance, we will entry different determination fashions (e.g., gpt-6-luna-decisions) through OpenRouter:
To strive Jev via this endpoint, change the mannequin identifier to typesafe/jev-1.14.
If you wish to entry OpenAI’s Selections API immediately, the request construction is a bit completely different: context goes into enter, questions type a listing, and options use worth and description. Its solutions additionally arrive as a listing.
3.5 Provide textual content with the LLM
When the choice mannequin selects “Fill Message”, we’d like the LLM to provide the textual content for that message discipline.
For this half, we use GPT-6-Luna through the Responses API. We begin by configuring the Python shopper:
Then, we implement write_field():
For the LLM name, we provide the unique job, present web page, and chosen discipline label. That needs to be sufficient for the LLM to provide the specified reply.
With that, we now have the items wanted to learn the web page, choose an operation, provide textual content when crucial, and carry out the interplay.
We are actually able to run the workflow and examine the end result.
···
4. Run the Workflow and Examine the Outcomes
4.1 Run the hybrid workflow
Right here is the whole choice hint I obtained:
The entire run takes 41s. We are able to see that the development made by the choice mannequin is smart. Steps 13–15 every set off an extra Luna name to provide the chosen discipline’s worth.
In whole, the workflow made 19 choice calls. The saved inquiry reveals:
Riverlight Lodge value €145 per evening, was refundable, and included breakfast, satisfying the resort necessities. The inquiry contained the requested dates, November 12–14, 2026, two adults, and Alex Morgan’s identify and electronic mail.
The message was:
Might you please let me know whether or not check-in at 23:30 is feasible? I might additionally admire a quiet room, if out there.
So, our workflow accomplished the request accurately.
4.2 Evaluate with pure GPT-6-Luna
If we had been constructing this with a traditional LLM, we’d ask the LLM to do each choosing the subsequent browser operation and supplying textual content for the fields.
Now that we’ve outsourced these alternatives to a choice mannequin, can we truly acquire something?
To search out out, I changed the choice operate with one which calls GPT-6-Luna via the Responses API. It receives the identical job, present web page content material, out there operations, and up to date actions. It returns the chosen identifier via structured output, so the reply is at all times one of many out there operations. The choice directions are additionally the identical. Its reasoning effort is ready to none, with a 256-token output restrict.
So, what are the outcomes?
Properly, the pure Luna model additionally accomplished the duty accurately. Nevertheless, it took 156s, in contrast with 41s for Determination-1 plus Luna. That is 3.8 instances sooner!
I connected a brief recording under for each variations, performed at regular playback pace. You possibly can see how briskly the choice mannequin strikes via the search and inquiry steps, whereas GPT-6-Luna remains to be getting into the journey particulars:
···
5. Remaining Ideas
When constructing an LLM software, it helps to have a look at what every mannequin name truly must return. Typically we’d like a newly generated reply. Different instances, we already know the options and simply want the mannequin to select one.
That second kind of labor is an effective place to strive a choice mannequin.
Python prepares the context and decisions, and connects the chosen choice to an operation, whereas the LLM handles the components that require open-ended technology. Collectively, they provide us a fairly helpful sample for constructing hybrid workflows.
Though we used browser automation because the case examine, you’ll be able to positively look for a similar sample in your individual functions. If a variety step is dealt with by your LLM at present, strive a choice mannequin instead, and see what adjustments.
