AI agents that browse the web, filling in forms, clicking through menus, and booking things, are notoriously unreliable. They repeat the same failed action, misread a page, or wander off course partway through a task. One way to catch these mistakes is to have a second AI model watch every step and grade it: is this click moving the task forward, or not?
The trouble is that using a large AI model as that grader is both expensive and unreliable: prompted this way, models like GPT-4o often misjudge which actions are actually helping. The researchers estimate that using GPT-4o to grade actions across a standard test set of 812 tasks would cost about $14,000 and take 40 hours on a single high-end GPU. That cost makes it impractical to use in most real settings, even though the grading itself genuinely helps agents perform better.
Grading a single step is harder than it sounds, because a step's value often depends on what happens later. Scrolling down a page might look like it accomplishes nothing, but if the information needed is one screen below, that scroll was the right move. The researchers found that AI models asked to grade actions in isolation often get this wrong, mistaking a necessary but invisible step for a wasted one.
There's also a timing problem specific to web tasks. In many other domains, an AI can try several full solutions and pick the best one after the fact. That doesn't work on the web: if an agent makes eight attempts to book a flight, the first successful booking already happened, and the other seven can't be undone. Judgment has to happen at each step, not just at the end.
The researchers built a model called WEB-SHEPHERD that grades agents in two stages. First, given an instruction like "find the warranty period for this laptop," it writes a short checklist of the subgoals needed to complete it, such as searching for the product, opening the right page, and locating the warranty section. Then, at each step the agent takes, it checks the checklist and scores whether each item is done, in progress, or not yet started.
(opens full size in a new tab)This checklist step matters more than it might seem. In tests, removing it caused accuracy to fall sharply. The checklist gives the grading model a fixed reference point, so it judges consistently across a long, multi-step task instead of grading each action in isolation. Because the checklist turns judging into a narrow, well-defined task instead of an open-ended one, a small model can do it well: WEB-SHEPHERD runs at just 3 to 8 billion parameters, far smaller than the frontier models it's being compared against.
To train the system, the team built a dataset of 40,000 step-level examples, drawn from real people completing tasks on more than 70 real websites. Each of their actions was compared against plausible wrong moves collected from other AI models. They also built a new benchmark, since no existing one measured how well a model grades web actions specifically.
(opens full size in a new tab)On that benchmark, WEB-SHEPHERD picked the correct action about 30 percentage points more often than GPT-4o did when simply asked to judge. When the researchers used it to guide a cheaper agent (GPT-4o-mini) as it browsed real websites, task success rose from 23.6% to 34.6%, an improvement of 11 percentage points. It did this at roughly a tenth the cost of using GPT-4o-mini itself as the judge, and about a hundredth the cost of using GPT-4o. Paired with the stronger GPT-4o as the policy, WEB-SHEPHERD still improved performance. Using GPT-4o-mini as the judge instead made things worse, dragging GPT-4o's success rate from 31.5% down to 26.7%.
(opens full size in a new tab)The system also performed well on WorkArena, a business-software benchmark it wasn't trained on, suggesting the approach generalizes rather than memorizing patterns from its training websites.
The approach hasn't yet been tested at larger scale, so it's unclear how much further accuracy could improve with a bigger model. It hasn't yet been tried as a training signal for reinforcement learning, and it doesn't yet handle instructions that mix text with images. The team has released the model, dataset, and code publicly. The checklist-based approach could extend to other step-by-step decision-making problems beyond web browsing, anywhere an AI system has to judge partial progress toward a goal it can't fully observe.
Paper: WEB-SHEPHERD: Advancing PRMs for Reinforcing Web Agents (arXiv:2505.15277)
Ocean's Halo ran its packaging supply chain across spreadsheets owned by three companies, retyped by hand at every handoff. One dashboard now shows what needs attention, and the plants and the supplier enter their own numbers.
ARGUS is a simulated 3D model of an outpost site that runs four specialized military tools as one live system, letting separate judgments about infrastructure and construction planning happen on one shared terrain and weather model.
A newsroom's international desk used to lose hours every morning figuring out what was new. Now a tool does that overnight, and flags which stories the desk is likely to run.
