Most companies run agents the way they run interns, one ticket at a time. Draft this post, then answer whatever came in overnight. The output is fine and the gains are small, because the expensive judgment still happens upstream, inside a human deciding what the next ticket should be.
That was the right model when the models were weak. It stopped being right sometime in the last year. A smart enough agent treats your project brief as a ceiling, executing your plan faithfully instead of finding the better one you didn't see.
You don't have to take my word on the timing. METR, an independent lab that measures what AI agents can actually finish, tracks the length of software task, counted by how long a human expert takes, that the best public model completes half the time. In 2019 that was about three seconds, and in early 2023 it was still four minutes. This spring it passed sixteen hours. Since 2023 it has doubled roughly every four months.
The other line of evidence is what keeps happening to the tests. The standard exam for fixing real software bugs went from 49 percent to 81 in thirteen months, and the company that co-created it stopped reporting it in February, saying scores no longer track real ability. An exam experts assembled specifically to stump AI went from single digits to 55 percent in nineteen months. In December, on ordinary computer work, an agent passed the human baseline for the first time, and twenty-four models now clear ninety percent on GPQA, a graduate-level science exam, so a top score there no longer separates anyone. There is a reason the newest of these tests is named Humanity's Last Exam.
Think about the best people you've hired. You never managed them by ticket. You gave them a goal and the ground rules, plus enough context to make calls without you, and then you reviewed outcomes. Agents are ready for that same contract. It has three parts.
- A goal with a number on it. Grow organic traffic to a million visits a month, or cut support resolution to four hours. A direction and a finish line, with the route left open.
- A constitution. The operating rules: what the company believes, what it never does, where the money limits sit, when to stop and ask a human. Each function writes its own on top of a shared base, because a marketing agent's judgment shouldn't read like a support agent's.
- A loop. The agent runs on a daily cycle. It reads its own results and adjusts course. You review outcomes and rule-adherence on a cadence, like you would with any senior hire.
I ran this at Flip Education. By the end it was two people operating in eighteen countries and eleven languages, with seventy-plus scheduled agents underneath, watched by automated quality gates and a cost ledger that closed every morning. The charter and the gates were the constitution. What running it taught me is that the rules did more work than the prompts ever did. Prose instructions decayed within weeks; the encoded ones didn't.
The hard part is the constitution itself. Which failure modes to catch before they burn a week, and what good looks like per function, written precisely enough for a machine to execute. Ours got a spending clause the hard way: a stretch in which four cost estimates in a row missed by three times taught us that any sizeable job states its expected cost and gets an approval recorded before the tools will run it. None of that judgment comes in the box with the agent, and writing it down is slow.
Whoever writes the best constitution in a category is going to run the cheapest, fastest company in it.
The tooling behind this contract is on GitHub.