What a Verification Gate Is Actually Worth
Everyone building agents agrees, in the abstract, that you want a check before an autonomous action fires. Almost nobody can tell you what that check is worth — because measuring it means running the counterfactual: the same agent, same market, same moment, without the gate.
So we ran it. Live. With real capital.
The setup
Two agents. Both run Kimi K2.6. Both see the same live USDC spot markets on a major US exchange — a rotating set of liquid pairs (ETH, SOL, NEAR, XLM and others). Both started on 2026-06-15 with $1,035.
The only difference: one agent trades directly. The other must clear ThoughtProof Sentinel before every action — an independent reasoning gate that never sees the agent's own confidence, only the plan and the market. The gate is fail-closed: both BLOCK and UNCERTAIN stop execution. On high-stakes actions the design can escalate to an adversarial multi-model panel, though in this run Sentinel resolved every cycle on its own.
The result
As of cycle 1,987 (28 days in, still running — of which 1,107 had a comparable trade the gate ruled on; the rest were cycles where the two portfolios had diverged too far to compare):
| Arm | Equity | Return | Executed | Blocked |
|---|---|---|---|---|
| Verified (Sentinel gate) | $796.38 | −23.1% | 458 | 733 |
| Unverified (counterfactual) | $24.82 | −97.6% | 1,294 | — |
| Delta | +$771.56 | +74.5pp |
The market went down over these four weeks. The gate did not make the agent profitable — that's not what it does. It kept the portfolio recoverable. Unverified: −97.6%, effectively wiped out. Verified: −23.1%, still in the game.
What the gate actually caught
The equity gap is the headline, but it's not the proof. A gate that blindly blocked 80% of trades would also lose less in a falling market — you don't need reasoning to be cautious, you just need to trade less. (We ran that control too: a rate-matched random blocker. Trading less does protect capital.)
So the equity number isn't the claim. The claim is what the gate blocked, and why. That's the thing a coin-flip can't do. Across the run, every stop came with a specific, criterion-level objection. Real examples, verbatim:
These aren't rate-limits. They're reasons — each one points at the exact place the agent's plan diverged from its own evidence. And a reason is something the agent can act on. Which is what happened:
- 507
ALLOW_AFTER_REPLAN— the agent's first plan failed the check, it read the objection, and produced a better one. The gate working as a correction loop, not just a filter. A random blocker produces zero of these — it has nothing to hand back. - 413
UNCERTAIN— the reasoning couldn't be confidently cleared, so execution stopped and a human would decide. No silent auto-execution. - 179
BLOCK— the reasoning didn't hold; the action was stopped outright, each with its objection. - 8
ALLOW— sound on the first pass.
The 507 successful replans are the part no throughput-matched placebo can reproduce. A coin-flip that blocks a trade leaves the agent stuck. An objection tells it what's wrong, and it fixes itself before capital moves. That is the difference between trading less and reasoning better.
"You just picked a weak model"
Kimi K2.6 is the base model in the headline run, and the obvious objection is that a stronger model wouldn't need the gate. So we checked. We ran the same harness with GPT-5.4 and Claude Opus 4.8 — and both of them still produced plans that cited data not present in their own evidence. Same failure class, verbatim from the log: "justifications citing data not in the evidence are fabricated." Opus triggered that objection 57 times; GPT-5.4, 43.
Frontier models drift and hallucinate too — less often, but they do, and "less often" is not "never" when real capital is on the line. That's the point of a gate: it isn't there because your model is bad, it's there because every model occasionally acts on a premise it made up. The gate catches that failure regardless of which model produced it.
Cheap enough to run on everything
A gate you can only afford on the big decisions isn't a runtime safety layer — it's an occasional audit. Each Sentinel check here costs under a cent. That's possible because Sentinel runs on OpenServ's SERV Reasoning models, tuned for exactly this kind of fast, structured judgment. The economics are the point: when verification costs a fraction of a cent, you run it on every action, not just the ones that scare you.
What this is and isn't
This is not a trading-alpha claim. Whether the market moves your way is orthogonal to whether your agent's reasoning was defensible. This experiment measures the second thing: reasoning integrity under real stakes.
It's also not a benchmark you have to take on faith. The gate that produced these verdicts is public and callable — Sentinel is live at sentinel.thoughtproof.ai, and anyone can send a plan plus market context to the same endpoint this experiment uses and get the same kind of structured verdict back. The numbers update daily on the live experiment page. The A/B harness that drives the two agents is internal (it holds live-capital credentials) and we don't publish trade-level data for capital privacy — but the part that does the judging, and the fail-closed policy, are fully inspectable.
Limitations — what we're not claiming
We'd rather state these than have you find them:
- One market regime. This ran over four weeks of a broadly falling market. A gate that protects capital in a downturn might look like a brake in a rally — the honest test is running it across regimes, which we're still accumulating.
- The unverified arm is a paper counterfactual. Only the verified agent trades live capital; the unverified twin is simulated on the same market feed. Once their portfolios diverge, they're no longer holding identical positions, so the later cycles compare two increasingly different books. The equity gap is directional evidence, not a controlled p-value.
- Not a same-ticks head-to-head. The random-block control and the frontier-model runs happened over different windows, not the exact same market ticks as the headline run. A tick-for-tick comparison is the next thing we're building.
None of these touch the core, model-independent finding: across every model we tried, the agent sometimes acted on premises its own evidence didn't support, and the gate caught it with a specific, reproducible objection. That part doesn't depend on the market, the window, or the counterfactual.
Why it matters for anyone shipping agents
The hard part of agent autonomy isn't the automation. It's knowing when an action is good, bad, or dangerous — and knowing when to stop and ask. A verification gate is the layer that answers that, per action, at runtime. This is what it's worth: in a bad month, the difference between a −23% drawdown you can recover from and a −98% one you can't.