Wednesday 28 October 2026
Separate critic review lifted agent accuracy up to 9.5 points
Five bullets
The one thing
Rethinking Verification in Agent Harnesses compared 17 verification designs across seven frontier coding models. Independent review by a separate critic gave the best accuracy for the cost. On Terminal-Bench it added 5.9 to 9.5 points at 1.13 to 1.25 times baseline cost.
The paper sorts verification into four layers. Initial instructions and mid-turn reminders gave modest gains that depended on the model. Completion gates helped more but could raise cost sharply. On SWE-bench Verified, a critic added 3.9 to 6.6 points at nearly baseline cost.
Extra review rounds or more critics added up to 3.5 more points on Terminal-Bench. The practical lesson is simple. More rules in the prompt help a little. A second agent reviewing finished work helps more, and the skill that grows is designing that review.
Before adding another rule to your prompt, add a separate review pass and compare the results.
Try this · 15 minutes
See what a separate critic catches in your agent's diff
- Pick a small task your agent finished today and save its diff.
- Open a fresh agent session with none of the original conversation.
- Give it the task and the diff, then ask for concrete defects only.
- Send that critique to the original agent and compare the revised diff.
Stories
- TRYCodex 0.165.0 lets you change reasoning effort mid-conversationCodex shipped version 0.165.0 with reasoning effort changes during a conversation on by default for supported models. Memory v2 is now the default for fresh profiles, with imports from other agents. The release also adds GPT-6.1 Terra and a built-in Vertex AI provider, set with model_provider = "gcp". It matters to daily Codex users who want to tune effort turn by turn.
- TRYSilent truncation cut a third of one agent's tool outputsContext Is a Pipeline, Not a Window measured silent truncation across 720 agent runs. Prompts never used more than 5.75% of the window, yet 3,827 of 11,189 tool results were cut. The harness sliced JSON at a fixed length, so every cut result reached the model broken. Two of seven surveyed frameworks cut output the same way, so anyone who builds a harness should check theirs.
- WATCHCoding agents change their hidden instructions between releases, often unexplainedHidden in Plain Sight studied the instruction stacks of 1,886 releases of Claude Code, Codex, Gemini CLI, Qwen Code and OpenCode. Occasional large revisions changed the rules for tool use, attribution and coding style. Only 48% of examined changes explained why they were made. It matters to anyone whose agent behaves differently after an upgrade.
- TRYJetBrains Air runs several coding agents side by side in the IDECodex, Gemini, GitHub Copilot, Claude Agent and Junie are supported. Sessions open as tabs with diffs and session-level cost estimates.
- WATCHMemory Tax: many agent memory systems lose to plain RAGBuilding memory can drop or obscure evidence, which the authors call memory tax. Worth recalling as Codex turns memory v2 on by default.
- WATCHWriting down claims before feedback cuts sycophantic agent editsThinking-in-Bets has the agent record its claims, checks and confidence first. Writing the same record after a complaint recovered little of the gain.
- WATCHAnthropic separates the agent harness from the sandbox that runs codeIn Managed Agents, a failed sandbox does not end the session. Credentials stay outside the sandbox and attach through a proxy.
- SKIPClaude Code v2.1.312 stops overwriting large, partly read CLAUDE.md filesWrite now asks Claude to read the whole file or edit in place. Update and move on.
Safe to ignore
53 sources read. 150 items. 14 shortlisted. 5 bullets.