We benchmarked trading agents on the action surface behind Robinhood Agentic Trading. Every model we tested holds the user's policy more reliably on SERV than the best frontier model does off the shelf, and the small, fast tiers do it at half to a tenth of the cost per decision.
| Model, on Robinhood Agentic Trading | Off the shelf | + SERV |
|---|---|---|
| gpt-5.4 | 75.7$4.52 /1k · 2.2s | 91.6$6.87 /1k · 4.5s |
| claude-fable-5 | 77.7$22.29 /1k · 6.0s | 89.5$39.51 /1k · 5.5s |
| gpt-5.5 | 71.8$8.72 /1k · 3.1s | 89.5$12.24 /1k · 3.8s |
| grok-4.5 | 75.0$2.44 /1k · 5.0s | 89.1$3.36 /1k · 20.2s |
| claude-opus-4.8 | 60.4$11.06 /1k · 4.0s | 89.0$16.02 /1k · 3.9s |
| gpt-5.6-sol | 73.7$6.84 /1k · 4.0s | 88.7$10.11 /1k · 3.7s |
| gpt-5.6-luna | 71.1$1.59 /1k · 2.8s | 87.8$2.32 /1k · 18.3s |
| gpt-5.4-mini | 50.7$1.30 /1k · 2.8s | 86.0$1.84 /1k · 2.7s |
| grok-4.3 | 63.0$2.37 /1k · 3.4s | 86.0$3.68 /1k · 7.0s |
| deepseek-v4-flash | 44.9$0.19 /1k · 14.1s | 85.4$0.30 /1k · 32.3s |
| gpt-5.6-terra | 69.6$3.53 /1k · 2.5s | 85.1$4.68 /1k · 2.6s |
| deepseek-v4-pro | 56.4$1.03 /1k · 10.7s | 81.8$1.58 /1k · 17.3s |
| gpt-5.4-nano | 0$0.37 /1k · 1.7s | 78.9$0.66 /1k · 2.7s |
Robinhood Agentic Trading connects an agent directly to a dedicated Agentic account, so the agent does both the financial reasoning and the execution: it researches, places and cancels equity and options orders, and rebalances, within the limits the user sets. The reasoning is the half that has to be trusted, because the wrong call moves real money. So the question is whether a small, fast model can hold those limits as well as a frontier model.
SERV is a reasoning layer that wraps a base model. You call the SERV API where an agent would otherwise call a model, and you point it at a smaller model instead of a frontier one. SERV runs that model through an extra structured reasoning pass before it answers, and the smaller model comes out reasoning more reliably than the big one, for less. On Robinhood Agentic Trading this is the financial-reasoning half: SERV works each decision through the user's limits, buying power, per-trade cap, watchlist scope, options approval level, pattern-day-trade and market-hours rules, and returns one action. Execute, resize to the binding cap, hold, escalate for the user's approval, or block. The Agentic account carries it out within those same limits. SERV reasons the decision; every investment decision remains the user's, exactly as Robinhood's disclosures require.
SERV runs in three modes. The per-call mode reasons through each decision as it happens. SERV Shadow does that reasoning once ahead of time: SERV writes the policy as an explicit decision graph, and a shadow agent audits the graph against the source policy, repairing anything that drifted, so each later decision costs less. SERV Kronos, the newest mode, is the strongest of the three on this benchmark: it holds the user's gates while cutting needless refusals, and it leads the board for most models. All three run against the same tools and the same grading. Which mode leads depends on the model, so the comparison above shows each model's best; per-call remains the fit when the policy changes from request to request.
Routing gpt-5.4-mini through SERV lifts its score from 50.7 to 86.0 out of 100, past every model off the shelf. The gain holds across the board: all thirteen models clear the off-the-shelf ceiling on SERV, and the strongest configuration reaches 91.6.
The reliability comes cheap. Small and mid models on SERV reach 87 to 89 for $2 to $4 per 1,000 decisions, the same scores that frontier-class configurations post at $12 to $40. They also beat the off-the-shelf frontier, which runs $4.52 to $8.72.
On its own, the smaller model breached hard controls. On SERV it cleared all 186 hard-control checks (restricted symbols, account-scope boundaries, the user's trading pause), in both its per-call and Shadow configurations. On a frozen set of unseen scenarios, the SERV Shadow configurations hold at 94 to 98% adherence, with one hard-control miss across 477 decisions.
The tiny model makes the sharpest case. Off the shelf, gpt-5.4-nano scores 0 with 32 hard-control breaches; through SERV it reaches 28.5; on SERV it clears all 186 hard-control checks and scores 78.9, above every frontier model off the shelf, at $0.66 per 1,000 decisions.
Every figure here comes from the real order schema behind the Robinhood Trading MCP, graded by a deterministic rule engine. The score tracks one thing: whether the agent holds the user's mandate. Profit is out of scope.
We built this benchmark on the tool surface of the Robinhood Trading MCP. The actions are the ones an agent runs against an Agentic account, review_equity_order, place_equity_order, cancel_equity_order, the options suite (review_option_order, place_option_order), and an escalate path that maps to Robinhood's approval mode, with the platform's own order schema: shares or fractional quantity checked against get_equity_tradability, limit orders with an explicit limit_price, and sizing grounded in the buying power that get_portfolio reports. Restricted or halted symbols, options approval levels, and pattern-day-trade limits are hard, non-overridable blocks. The benchmark simulates this surface offline, so no live account is touched, which also makes it the paper-trading sandbox the platform itself does not ship.
In the agent's loop, SERV is the model call itself, not a step after it: the agent hands SERV the task and the user's limits, SERV returns the action, and it executes on the Agentic account. Swapping a frontier model for a smaller one on SERV is a one-line change, and it leaves the Agentic account, its limits, and Robinhood's own approval flow exactly where they are.
Reliability here is one number, the SERV DeFi Score, on a 0 to 100 scale where 100 is a flawless policy. It is built from the angles that matter when money moves under a mandate, not raw accuracy.
Execute, resize, hold, escalate, or block: the agent is credited for the action the policy actually calls for, judged against a deterministic rule engine, never another model.
Refusing something the policy allows is safe but wastes the user's opportunity, so it carries a small penalty. That alone stops a "refuse everything" agent from scoring well.
Acting when the policy said to wait, resize, or escalate is a real error, penalized well above caution.
A trade through a restricted symbol, an options-level gate, or the user's trading pause is the failure a user cannot recover from, so it costs an order of magnitude more than an ordinary error. Catastrophic safety is also reported on its own, so a rare disaster cannot be averaged away.
Cost and latency stay off the score and sit beside it, so price is weighed against behavior. The ranking is stable across a wide range of the severe-error penalty, so it doesn't hang on one chosen weight. Only an order that actually fires counts as a trade; a refusal mislabeled in its own record earns no credit, but it is not priced as a trade.
The comparison at the top is the whole finding. Off the shelf, the best any model reaches is 77.7 and the mainstream frontier tops out at 75.7; on SERV, every model clears that bar, the strongest reaching 91.6, and each “+ SERV” result is clean across all 186 hard-control checks. What matters for Robinhood is score read against cost. A small, fast model on SERV reaches the top of the board for a few dollars per thousand decisions: grok-4.5 scores 89.1 at $3.36 and gpt-5.6-luna reaches 87.8 at $2.32, while frontier configurations at the same scores run $12 to $40.
The “+ SERV” column is each model's strongest SERV configuration. SERV runs in three reasoning modes, per-call, SERV Shadow, and SERV Kronos; which one leads depends on the model, and the full mode-by-mode matrix is available on request. Latency tracks the mode rather than the model: most SERV configurations answer in 2 to 7 seconds, and where a slower mode tops a model's score, a nearly equal configuration runs in that band.
SERV is the reasoning layer a strategy runs on. In our benchmark, the strategy is the trade itself: a smaller model on SERV makes the call. ThoughtProof, an external team, put a different strategy on the same layer, a safeguard that checks each trade before it executes, and ran it with live capital on a major exchange. The safeguard held up, at a fraction of the cost of the bigger-model setup it replaced.
Every SERV tier from the mid range up scored above every general-purpose model on the panel, at the same cost tier, and none of them let a dangerous call through.
ThoughtProof Research, Rasch calibration (the psychometric method used to score standardized tests), 150 cases across 13 industries and 10 models, person reliability 0.91. Models on SERV are verification-tuned; the entry tier is the first-pass filter in a cascade, not a standalone judge.
The trading model varied, and the test ran both live and on paper, but the pattern held: when the safeguard on SERV cleared each trade before it executed, the account kept more of its capital. Agents fabricated balances and price moves the data didn't support, and the safeguard caught them before the trade went through.
ThoughtProof, live and paper A/B with Kimi K2.6, GPT-5.4, and Gemini 3.5 Flash as the trading model; the live arm ran 14 days, ~980 cycles, on Advanced Trade. Small-capital experiment; the safeguard runs on SERV's Nano-to-Pro cascade.
Two strategies, the trade and the safeguard around it, both ran on smaller models through SERV and stayed reliable at a fraction of the bigger models' cost.
Robinhood Agentic Trading puts an agent's financial reasoning and execution on a real brokerage account, within the limits the user sets, across seventy thousand agentic accounts and growing. The disclosures are candid that agents can err; two things have to hold anyway: the agent keeps the user's limits when real money moves, and the per-decision cost holds as usage grows.
SERV holds both. A small, fast model on SERV keeps the mandate more reliably than the frontier on the real account actions, at half of the per-decision cost down to a tenth.
The summary above rests on a full evaluation.
173 scenarios, 62 of them catastrophic-eligible, meaning an action exists that would breach a hard control. Each is run three times to absorb model variance, so the headline figures are averages. The 186 hard-control checks come from the catastrophic-eligible cases across runs.
Scenarios call the actions behind the Robinhood Trading MCP, with the platform's own rules: order review before placement, share and fractional sizing checked against tradability, limit orders with an explicit limit price, and restricted symbols, options approval levels, and pattern-day-trade limits as hard blocks. The benchmark simulates this surface offline, which doubles as the paper-trading sandbox the platform does not provide.
Every decision is scored by a rule engine that computes the uniquely-correct action. A breach counts because a coded control fired, the numbers are reproducible from the scenario logs, and no model grades another model's output.
Per-call reasoning, SERV Shadow, and SERV Kronos all run under the same scenarios, the same three runs, and the same deterministic grading as every other row. SERV Shadow was additionally re-checked on a frozen held-out set of unseen scenarios: adherence 94 to 98%, with one order through a hard control in 477 decisions.
Degenerate strategies set the floor: an always-refuse and an always-execute agent both score far below every real configuration on the same metric, which confirms the score rewards judgment and not a fixed reflex. A separate multi-agent audit reproduced all 173 oracle labels with zero mismatches.
This measures policy adherence, not profit or alpha. It ran paper and sandbox only, at three runs per scenario. The clean catastrophic result (four breaches to zero) is specific to gpt-5.4-mini; other models shift by different amounts, and the comparison at the top shows each model's before and after.