Webwright: Why AI Net Brokers Ought to Write Code, Not Click on

1. gained’t maintain nonetheless

When you have constructed an online agent lately, you already know the failure sample. You give it a job like “pull each itemizing from this listing right into a spreadsheet” and watch it inch ahead. It reads the web page. It predicts a click on. It waits for the brand new DOM, the web page construction the browser sees. It reads once more, predicts once more, waits once more.

Then, someplace round step 40, issues collapse. A modal pops up unexpectedly. The “subsequent web page” button strikes. The agent errors one component for an additional. Any certainly one of these can derail the entire job. The deeper drawback isn’t the dangerous click on. It’s how the agent operates: take a look at the web page, determine on one motion, see what modified, then determine once more. It repeats this loop again and again, with no sturdy plan for the right way to full the duty from begin to end.

The sphere has tried a couple of other ways to make this loop extra dependable. Some brokers, like OpenAI’s Operator and Anthropic’s Pc Use, work from screenshots and work together with a web site very like an individual would. Others, like WebVoyager, use the web page’s DOM to know what parts can be found and determine which one to work together with.

Benchmarks resembling Mind2Web and WebArena made these methods simpler to match by giving brokers a regular set of actions—click on, kind, scroll, choose. And open-source instruments like browser-use, Skyvern, Stagehand, and LaVague have packaged these concepts into APIs that engineers can extra simply construct into actual purposes.

These approaches make the loop extra dependable, however they don’t change the way it basically works: the agent nonetheless takes one motion at a time, waits to see what occurs, then decides what to do subsequent. And when the duty is over, it hasn’t constructed something reusable—it has solely accomplished a sequence of clicks.

Webwright, a browser-agent framework from Microsoft Analysis and the College of Hong Kong, takes a unique method. Its tagline captures the concept: “A terminal is all you want for net brokers.”

As a substitute of asking the mannequin to determine the subsequent click on, Webwright has brokers writing and working code—utilizing bash and Playwright scripts to open browsers, examine pages, and perform the duty. The consequence isn’t only a lengthy sequence of browser actions. It’s a program engineers can examine, rerun, modify, and reuse.

This distinction issues most when the online is your information supply: dashboards, product catalogs, search outcomes, inside instruments, JavaScript-heavy websites, and workflows you anticipate to run greater than as soon as. In these circumstances, the query isn’t simply whether or not an agent can end the duty. It’s whether or not it ought to preserve clicking by means of the browser or write a reusable program to do the work.

We’ll begin with the 4 major approaches to constructing net brokers and the constraints they nonetheless share. Then we’ll look inside Webwright: how its three core parts work, how a framework of roughly a thousand traces performs on benchmarks, and what the outcomes say about price and reliability. Lastly, we’ll put the method to work on three frequent scraping issues—paginated pages, JavaScript-rendered content material, and infinite-scroll feeds.

2. Why net brokers preserve breaking

The shift to “write code” issues as a result of it addresses the underlying drawback, not simply the signs. At present’s net brokers differ in how they perceive a web page—some take a look at screenshots, others learn the DOM—however most nonetheless work the identical means: take one browser motion, see what occurs, then determine on the subsequent one.

That works for brief duties. However the longer the duty runs, the extra possibilities there are for one dangerous click on, a modified web page, or a misinterpret component to throw all the pieces off.

Imaginative and prescient brokers are simple to know: they take a look at the web page very like an individual does and determine the place to click on. That works properly for a lot of browser duties, however scraping calls for extra consistency. A small structure shift can transfer a button simply sufficient for the agent to click on the incorrect place—or nothing in any respect.

There’s additionally a value to repeatedly screenshots. Each new screenshot consumes tokens, and the agent has to hold sufficient context ahead to recollect what it already did. On a protracted job, that context turns into more durable and dearer to take care of.

DOM and accessibility-tree brokers keep away from a few of the issues that include screenshots. As a substitute of guessing the place a component relies on pixels, they will learn the construction of the web page and determine buttons, hyperlinks, kinds, and different parts straight. WebVoyager, for instance, reported about 59% job success throughout 50 real-world web sites, considerably higher than text-only baselines.

However this creates a unique drawback: an excessive amount of web page information. A posh web page can produce an accessibility tree bigger than 50KB. Because the agent strikes by means of a job, previous web page state accumulates in its context though a lot of it’s now not helpful. The references it makes use of to determine parts also can change when a web page lazy-loads content material, rerenders, or navigates someplace new.

So higher entry to web page construction doesn’t essentially make lengthy browser duties dependable. On VisualWebArena, main vision-language brokers accomplished solely about 16% of duties, in contrast with roughly 89% for people.

Mounted motion APIs made net brokers simpler to construct and benchmark. Give the mannequin a small set of actions—click on, kind, scroll, choose—then let it observe the consequence and select once more. The draw back is that the agent can solely specific one small step at a time. It will possibly’t naturally say, “preserve clicking subsequent till there are not any pages left,” “retry if this component doesn’t seem,” or “accumulate these 1,000 rows and save them to a CSV.” Every of these needs to be damaged into many particular person actions, with one other mannequin name in between.

Frameworks like browser-use, Skyvern, Stagehand, and LaVague make browser brokers a lot simpler to construct and combine. That’s helpful, however it doesn’t clear up one essential drawback for recurring information work: when the duty is completed, there usually isn’t something reusable left behind. The agent might have collected the info as soon as, however subsequent week it has to work by means of the browser yet again.

Analysis had already pointed towards one other method. Within the ICML 2024 paper Executable Code Actions Elicit Higher LLM Brokers, the researchers behind CodeAct changed predefined, JSON-style actions with executable Python. They reported as much as 20% greater success charges whereas utilizing roughly 30% fewer steps.

The reason being easy: code lets the mannequin do greater than take one motion at a time. It will possibly use loops, retailer variables, retry failures, write information, and examine errors—all inside a program it might probably run once more. Webwright brings that very same thought to browser automation.

3. Webwright’s distinctive level: make the workspace the state

Most browser brokers preserve their progress within the browser session. Shut the tab, and that state is gone.

Webwright flips this round. The browser is momentary; the native workspace is what persists. The agent writes scripts, logs, screenshots, and output information as it really works, finally turning a profitable run right into a reusable device. This shift has a couple of sensible advantages.

• Extra strong interactions — Playwright selectors and wait situations are extra dependable than pixel coordinates or momentary component IDs.
• Higher composition — Loops and capabilities can deal with tons of of repeated actions in a single program
• Seen state — Progress is seen in information and logs.
• Reusable output — As soon as the duty works, the code could be reproduced as a substitute of ranging from scratch.

The place the challenge’s 4 said benefits come from:

• Strong, reusable interactions — the agent acts by means of queries and wait-for-condition checks (web page.locator(...), wait_for_selector(...)) as a substitute of pixel coordinates or frozen component IDs, so a script survives structure shifts and re-renders.
• Environment friendly composition — loops, capabilities, and variables let a single flip say “do that for each row,” work a one-action-at-a-time agent has to spell out step-by-step.
• Workspace as state — progress lives in information, not in a fragile session or a context window bloated with stale web page dumps.
• Minimal by design — the entire system leans on 4 libraries (httpx, pydantic, playwright, typer) with no hidden framework beneath, and nonetheless posts state-of-the-art numbers.

3.1 Let’s do fast comparisons between Webwright and different choices in some eventualities.

3.1.1 Demo 1 · When clicking isn’t exact sufficient

The duty: Use Chase’s IRA calculator to match a Conventional vs. Roth IRA for somebody who’s 30, retires at 65, saves $300 a month, earns a 3% return, and has tax charges of 13% right now and 24% in retirement.

The problem is that the calculator makes use of six interactive JavaScript sliders.

End result:

• Webwright units the values straight in code by updating the DOM inputs and triggering the required occasions. The values are precise, the chart renders accurately, and the working resolution is saved as a reusable script.

• A imaginative and prescient agent has to govern the sliders visually. It will get shut, however not shut sufficient: the $300 contribution lands at $294.

3.1.2 Demo 2 · When the identical job comes again

The duty: Search Google Flights for a spherical journey from Seattle to San Francisco, together with the dates, and return the ranked outcomes.

End result:

• Webwright completes the search like different browser brokers would possibly. The essential distinction is what occurs afterward: it retains the working code.

• When the same flight search comes up later, the agent doesn’t have to determine each discipline, date picker, and click on once more. It will possibly reuse the earlier script, change the inputs, and run it once more.

That’s what Microsoft means by “your shopping historical past is code as a substitute of clicks.” A accomplished job turns into a place to begin for the subsequent one, slightly than a browser session that disappears when it ends.

3.2 How Webwright differs from different browser-agent repos

The alternate options are helpful, however they nonetheless put the browser on the middle of the workflow. Stagehand combines Playwright with natural-language instructions. agent-browser offers brokers a CLI for taking small browser actions. browser-use repeatedly reads the web page, chooses an motion, and executes it.

Webwright takes a unique method. As a substitute of selecting the subsequent browser motion, the mannequin can write a complete Python script. The browser is momentary; the code, logs, and outputs keep within the native workspace. And when the duty is solved, the agent leaves behind a program that may be run once more.

That’s the core thought: clicking completes the duty as soon as; code completes it and retains the answer.

3.3 Inside Webwright

So what does it take to construct an agent like this? Surprisingly little.

Most net brokers put a harness—the software program connecting the mannequin to the browser—between the 2. That harness normally offers the mannequin a set set of actions: click on this component, kind into this discipline, scroll the web page, learn the DOM, take a screenshot.

Webwright takes a unique method. As a substitute of giving the mannequin a menu of browser actions, it offers the mannequin a terminal and lets it determine what instructions to run.

That makes the system surprisingly small. The core harness is roughly 1,000 traces of code throughout three parts. The total repository is nearer to 1,500 traces when you embody the command-line interface and help for various mannequin suppliers. There isn’t any giant library of predefined browser actions. No customized DOM engine. The core system is simply three items:

• Runner (~150 traces) — Retains monitor of the duty and all the pieces that has occurred to this point: what the agent is making an attempt to do, the present state of its workspace, and the outcomes of earlier actions.
• Mannequin Endpoint (~550 traces) — Connects Webwright to the language mannequin. It offers backends for OpenAI, Anthropic, and OpenRouter.
• Setting (~300 traces) — Provides the mannequin a terminal linked to Playwright working Chromium. That is the place instructions really execute, browser interactions occur, and information created throughout the job are saved.

The interplay between these items is an easy loop.

1. The Runner offers the mannequin the duty and the most recent context.
2. The mannequin decides what to do subsequent and returns a shell command.
3. The Setting runs that command and sends again what occurred—terminal output, logs, screenshots, or error messages.
4. Webwright provides these outcomes to the context and asks the mannequin what to do subsequent.

In brief, the loop appears like this:

perceive the present state → select a command → run it → see what occurred → repeat

The method continues till the mannequin believes the duty is full and a ultimate self-check agrees. Webwright doesn’t attempt to encode each potential browser interplay into the harness. It offers the mannequin a general-purpose interface—the terminal—and lets the mannequin work out the right way to use it.

The benchmarks help the design. On On-line-Mind2Web, GPT-5.4 with Webwright scores 86.7%, the best amongst open-source AutoEval harnesses, whereas Claude Opus 4.7 reaches 84.7% and performs higher on the toughest duties.

The larger sign comes from Odysseys. GPT-5.4 utilizing coordinate-based browser management scores 33.5%. With Webwright, the identical mannequin reaches 60.1%—a 26.6-point acquire from altering the harness, not the mannequin.

Webwright’s challenge web page lists 60.8%; I exploit the 60.1% reported in its GitHub comparability for consistency.

One other consequence helps the thesis: as soon as Webwright has constructed reusable instruments, the mannequin can get smaller. Microsoft experiences that even a 9B open mannequin (Qwen-3.5-9B) performs properly on On-line-Mind2Web as soon as 5 or extra instruments can be found. The device doesn’t simply save work—it lowers the mannequin functionality wanted subsequent time.

There are tradeoffs. These are LLM-judged AutoEval scores, and the headline Mind2Web consequence makes use of 100 of 300 duties. It’s also not low-cost: about $2.37 per job with GPT-5.4 and $6.09 with Claude Opus 4.7. Webwright spends extra compute upfront to construct instruments which might be extra strong and reusable.

4. Experiments and End result

I examined Webwright on three more and more tough websites, utilizing a separate Claude Sonnet agent for every run. That is to see how strong Webwright is the place scraping normally breaks.

I used the Claude Code plugin slightly than the standalone benchmark harness. It retains the identical core setup—terminal + Playwright—however Claude Code runs the agent loop. That removes the necessity for a separate API key or per-task API invoice, although not the compute price.

The tradeoff is token utilization. In Microsoft’s instance, the Codex-hosted ability used ~3.3M tokens versus ~424K for the standalone harness—about 8× extra, largely from cached context. The price shifts into the host session slightly than disappearing.

Setup took one command:

playwright set up firefox   # the Claude Code ability drives headless Firefox, ~110MB one-time

4.1 🔧 Check 1 — static pagination · books.toscrape.com

books.toscrape.com is the simplest one amongst 3 circumstances: 50 numbered catalogue pages, 20 books every, served as plain HTML. The duty was to extract each e-book title, value, ranking, availability, and URL, then craft a reusable CLI with --pages and --out.

Earlier than writing the scraper, the agent inspected the location and examined its boundaries: web page 50 had no subsequent hyperlink, whereas web page 51 returned a 404. It then pulled selectors from an actual product card and constructed a easy pagination loop.

for n in vary(1, pages + 1):
    url = CATALOGUE_URL_TEMPLATE.format(n=n)
    await web page.goto(url, wait_until="domcontentloaded")
    playing cards = web page.locator("article.product_pod")
    rely = await playing cards.rely()
    log(n, f"loaded catalogue web page {n}/{pages} ({url}) -> {rely} e-book playing cards discovered")

A click-based agent might deal with this website, however code was cleaner. One refined subject was relative e-book URLs, which change throughout pages. As a substitute of manually establishing them, the scraper used the browser-resolved href values.

The consequence was not simply scraped information, however a standalone CLI device that might run once more with out an agent.

$ python final_script.py --help
utilization: final_script.py [-h] [--pages PAGES] [--out OUT]
Scrape all books listed on books.toscrape.com's catalogue pages.
  --pages PAGES  Variety of catalogue pages to traverse ... Default: 50.
  --out OUT      Output CSV file path ... Default: books.csv.

$ python final_script.py --pages 2 --out pattern.csv
-> 40 rows written to .../pattern.csv

End result: 1,000 books throughout 50 pages, zero empty fields, in ~37 seconds.

The verification step additionally caught a bug: the primary model unintentionally cleared its proof log when working --help. The agent discovered the facet impact, fastened it, examined the repair, and reran efficiently. Even on this easy website, the benefit was clear: a debuggable, reusable program as a substitute of a one-time click on hint.

4.2 🔧 Check 2 — JavaScript-rendered · quotes.toscrape.com/js

The second take a look at provides JavaScript. The quotes are not current within the uncooked HTML; they seem solely after the browser runs the web page’s JavaScript. The agent verified this primary: a direct HTTP request returned zero quote parts, whereas the rendered web page confirmed 10. A fundamental requests + BeautifulSoup scraper would silently return nothing.

Meaning each web page should be rendered earlier than extraction. There’s one other catch: web page 11 nonetheless returns HTTP 200, so standing codes can’t inform the scraper when to cease. As a substitute, this system checks the dwell DOM for the subsequent hyperlink and stops when it disappears on web page 10.

whereas True:
    url = BASE_URL if n == 1 else PAGE_URL_TEMPLATE.format(n=n)
    await web page.goto(url, wait_until="domcontentloaded")
    await web page.wait_for_selector(".quote", timeout=10000)   # watch for JS to inject the quotes
    ...                                                       # learn the ten rendered .quote playing cards
    has_next = await web page.locator("li.subsequent a").rely() > 0
    if not has_next or n >= pages:                           # cease on the DOM, not a standing code
        break
    n += 1

Once more, the output turned a reusable CLI with --pages and --out, in a position to run with out an agent.

The payoff — the crafted CLI. As on the books run, the working script turned a reusable scrape_quotes(pages, out) device with an argparse interface (--pages, default 10; --out, default quotes.csv) that re-runs standalone, no agent within the loop.

End result: 100 quotes throughout 10 pages in 8.9 seconds, with zero empty textual content or creator fields.

The run additionally uncovered a nasty assumption in my transient: I anticipated two pages to provide 40 rows, borrowing the 20-per-page rely from the books website. This website serves 10, so the right consequence was 20. The agent returned the true information and flagged the mismatch slightly than forcing the output to suit the spec.

4.3 🔧 Check 3 — infinite scroll · quotes.toscrape.com/scroll

The third take a look at removes pagination totally. Quotes load 10 at a time through AJAX because the web page scrolls, so there are not any web page URLs to iterate by means of. The scraper has to scroll, wait for brand new content material, measure the web page, and determine when loading is completed.

The agent first confirmed the location’s conduct: has_next turns into false at web page 10, and web page 11 returns no quotes. It then constructed a scroll-until-stable loop that stops when no new content material seems.

for i in vary(1, max_scrolls + 1):
    await web page.consider("window.scrollTo(0, doc.physique.scrollHeight)")
    await web page.wait_for_timeout(1000)
    rely = await web page.locator(".quote").rely()
    if rely == prev_count:          # no new quotes arrived
        stable_iters += 1
        if stable_iters >= 2:        # cease on stability, not a set rely
            break
    else:
        stable_iters = 0
    prev_count = rely

The DOM rely grew 10 -> 20 -> ... -> 100, held at 100 for 2 scrolls, and stopped at iteration 11. --max-scrolls was solely a security backstop. Forcing --max-scrolls 5 returned precisely 60 rows, proving the cease situation answered to the web page, not a hidden fixed.

End result: 100 quotes in 11 iterations, taking ~17 seconds.

Throughout the three assessments, the worth of code turns into clearer because the websites get more durable. Static pagination is easy; JavaScript requires an actual browser; infinite scroll requires this system to purpose about when to cease.

There’s additionally a helpful cross-check: Rungs 2 and three scrape the similar 100 quotes by means of two completely different interfaces—pagination and infinite scroll—and produce matching outcomes row for row. That provides us a completeness verify we will really diff.

There are tradeoffs. Setup required a ~110 MB Firefox obtain and a few Home windows surroundings fixes. The Claude Code plugin additionally makes use of headless Firefox and its personal screenshot capabilities slightly than Webwright’s normal Chromium setup. For a one-off click on, this method is overkill. The payoff seems when duties contain repetition, dynamic content material, or outcomes you should confirm and reuse.

5. Conclusion

Total, these three assessments present that browser-based code era is greater than a method to automate clicks. The ultimate result’s a reusable Playwright program that may be run once more with out the agent. Because the web sites turned extra advanced, the generated code additionally turned extra succesful. It moved from easy web page loops to rendering JavaScript and eventually to reasoning about when an infinite scroll had completed.

The experiments additionally present the worth of verification. The agent discovered bugs in its personal code, questioned incorrect assumptions within the job description, and confirmed that the extracted information was full. The matching outcomes from the paginated and infinite-scroll variations of the quotes website present additional confidence that nothing was missed. That is tough to realize with a one-time browser recording alone.

There are prices. Working Playwright requires a browser obtain and extra setup than easy HTTP scraping or click on recording. The generated packages are additionally longer and require some technical data to know. Nonetheless, these prices are outweighed when the duty must be repeated, maintained, or verified. Total, the assessments recommend that browser-based code era is a sensible method for constructing dependable net scrapers that may adapt to completely different web site designs whereas producing code that’s reusable, clear, and simple to check.

Webwright’s contribution isn’t an even bigger mannequin or a greater immediate. It’s a less complicated thought:

Give the mannequin a terminal, let it program the browser, and preserve the consequence as reusable code.

That adjustments the web-agent loop. As a substitute of fragile clicks and fixed replanning, the agent can write, run, debug, and reuse a program.

The concept extends past browsers. When a mannequin can code and its surroundings can execute that code, it might be higher to write this system that performs the duty than predict each motion one step at a time.

One of the best net brokers don’t simply click on. They write the device—and go away it behind.

6. Sources

Webwright (major)

The panorama

Diagrams: author-created (matplotlib). Household animations: author-created. Screenshots and generated CLIs