Microsoft ThinkingBox: How to Test If Your AI Agent Actually Works

Your agent booked the refund. You watched it happen. The client watched it happen.

Run it twenty more times and there is a one in four chance it does that correctly every time.

That is not a guess. Microsoft published ThinkingBox on 19 August 2026, along with a benchmark of 507 real business workflows, and tested 12 proprietary and open-weight models against it. The strongest model scored 65.36% pass@1 and 25.25% pass^20.

Those two numbers, and the distance between them, are the most useful thing anyone has published about selling agents this year.

Read the second number carefully

pass^20 is not pass@20, and the difference matters enough to spell out.

pass@k usually asks: given k attempts, did at least one succeed? It goes up as k rises, because more attempts means more chances.

pass^k asks the opposite: did all k attempts succeed? It goes down as k rises, because one failure in twenty is enough to fail the whole thing.

So 25.25% pass^20 means the best available model handles a task correctly on all twenty runs for only a quarter of tasks. On the other three quarters, it will get it right sometimes and wrong sometimes, with no warning about which run you are on.

That is exactly the behaviour a demo cannot reveal, because a demo is n=1.

A demo is a single sample. The paper's whole point is that a single sample tells you almost nothing about a system that runs a thousand times a month.

What ThinkingBox actually does differently

Most agent benchmarks grade the agent's own account of events. Did it say it issued the refund. Did it emit a tool call that looks like issuing a refund.

ThinkingBox checks the database.

It runs isolated, MCP-compatible tool sessions, records the full execution trace, and then evaluates the terminal backend state. Task-specific executable checks compare the final state against the starting state and accept any valid path to the right outcome, while rejecting three separate failure modes:

• Wrong effects: it changed the wrong thing
• Missing effects: it did less than the task required
• Extra effects: it did the job and also something nobody asked for

That third one is worth pausing on. An agent that issues the correct refund and also cancels the subscription has completed the task by any self-report measure. It has also created a support ticket and possibly lost you a customer. Only a state check catches it.

The finding that should worry you most

From the paper: many failed trials show clean termination and valid state-changing actions.

Read that again. The agent finished tidily. It called real tools. It modified real records. Every signal your monitoring is probably watching looked healthy.

And the task was not done.

The paper's conclusion is blunt: response-level and tool-call-level signals are not clear proxies for end-to-end task completion. If your observability shows tool calls, latency and error rates, you are measuring whether the machinery moved, not whether the job got done.

The benchmark, in numbers

Five domains, and every one of them is a stateful business workflow rather than a coding puzzle. That is the point. Refunds, bookings, claims and IT tickets all leave a record you can check, and getting them almost right is worse than not attempting them.

How to borrow this without running the benchmark

You do not need ThinkingBox to apply its idea. Three things transfer to any agent you are shipping.

1. Run the same task twenty times before you call it working. This is the whole lesson and it costs you an afternoon. If it passes 20 out of 20, you have something. If it passes 17, you have a system that will fail roughly three times in twenty for a client, silently, and you found out before they did.

2. Assert on state, never on the transcript. After each run, query the database. Is the row there, does it hold the right values, and did anything else change. An agent that reports success while leaving the wrong state is the failure mode that reaches production, precisely because it looks fine.

3. Check for extra effects, not only missing ones. Most teams test that the agent did the thing. Almost nobody tests that it did only the thing. The benchmark rejects extra effects for good reason, and in a client's live system an unrequested change is the one that generates the angry call.

For anyone selling agent work, there is a commercial version of this too. Twenty consistent runs on the client's own data is a far better proof than a live demo, and it is a claim a buyer can verify. It also protects you, because you find the 17-in-20 case during the build rather than in month two of a support agreement.

Common mistakes

Treating a successful demo as evidence. A demo is one sample from a distribution the paper shows is wide. The strongest model in the field is fully consistent on a quarter of tasks.

Monitoring tool calls and calling it observability. Failed trials showed clean termination and valid state-changing actions. Those signals were green while the task failed.

Testing that the agent did the job, but not that it did only the job. Extra effects are a distinct failure category and they are the ones that damage trust with a client.

Assuming a better model closes the gap. 65.36% and 25.25% are both from the best model tested across 12. This is a property of stateful agent work today, not a gap you buy your way out of with a bigger context window.

Reporting pass@1 to a client. It is the flattering number and it describes a single attempt. If the workflow runs daily, pass^20 is closer to what they will actually experience.

Key takeaways

• Microsoft released ThinkingBox on 19 August 2026 with a benchmark of 507 policy-conditioned workflows across five business domains, testing 12 models at 20 trials each.
• The best model scored 65.36% pass@1 and 25.25% pass^20. pass^k asks whether all k attempts succeeded, so it falls as k rises.
• Grading is done on terminal database state rather than the agent's self-report, and it rejects wrong, missing and extra effects.
• Many failed trials terminated cleanly and made valid state-changing tool calls, so tool-level signals are not a proxy for task completion.
• Extra effects are their own failure category. Test that the agent did only the task, not merely that it did the task.
• The transferable practice is cheap: run any agent twenty times on the same task and assert on state after each run.
• Twenty consistent runs on a client's own data is stronger proof than a demo, and it is verifiable by the buyer.

We build and hand over agents for other people's workflows, so this is the standard the work has to clear before it ships. If that is the kind of thing you need built properly rather than demoed, that is what we do. On the layer underneath, our comparison of the agent frameworks covers what wraps the model, and Skills versus MCP servers covers how it reaches your systems in the first place.