Daily Eval

Wed 4 Nov 2026

A weekday email for software engineers working alongside AI agents. Five minutes: what changed, whether it matters for you, and one thing to try.

Sample.

AGENTS.md rules phrased as conditions were followed less often

  1. 001AGENTS.md study found conditional rules followed less often than direct ones
  2. 002Codex users report their training opt-out switched itself back on
  3. 003GitHub Copilot now runs as an agent inside JetBrains Air
  4. 004Silent Instrument Errors traced a 13-point leaderboard gap to the grader
  5. 005Caper approves shell commands after running them in a sandbox

How.

  1. 001Every weekday, a pipeline reads 57 sources: 53 feeds, Hacker News, Hugging Face daily papers, Lobsters and new GitHub repos.
  2. 002Today, 173 items became 14 shortlisted, then 5 bullets you can skim.
  3. 003Each story gets TRY, WATCH or SKIP. The one thing leads; Safe to ignore names the hype.

Who.

Written by John Young, a CTO who runs coding agents in production.

He builds flywheel, an open-source agent orchestrator.

jyoung.dev  /  github.com/johnayoung/flywheel

Join.

Agentic AI, evaluated every weekday. Five minutes.

2026-11-04.md

Wednesday 4 November 2026

AGENTS.md rules phrased as conditions were followed less often

Five bullets

The one thing

Direct rules beat conditional ones in agent instruction files

Do Coding Agents Follow AGENTS.md? tested one policy written in four styles. The researchers inserted it into the agent context files of 86 benchmark tasks. They ran each task across four harness-model combinations.

Rules phrased as conditions, in the form 'if X, do Y', were followed less often. Prescriptive rules give a direct command. Explanatory rules say why the rule exists. Both beat conditional phrasing, but which one won depended on the harness and model.

Your AGENTS.md is input the agent may or may not obey. That makes wording a variable you can test, not a matter of taste. The skill that grows is checking your instructions against your own setup.

Rewrite your conditional rules as direct commands, then check which wording your own agent follows.

Try this · 15 minutes

See which rule wording your agent actually follows

  1. Open your AGENTS.md and find one rule written as a condition, 'if X, do Y'.
  2. Write two new versions: a direct command, and a command that explains why.
  3. Run the same small task three times, once with each wording.
  4. Note which runs obeyed the rule. Keep the wording your harness and model followed.

Stories

Safe to ignore

57 sources read. 173 items. 14 shortlisted. 5 bullets.