A daily loop that tests upgrades and keeps what agents use

Why this is worth building

AI moves fast. Even a specialist's excellent skill, meaning a written method an agent reads when a task needs it, will be outdated within one to three months and won't use the best tools anymore. Meanwhile people publish what works on the web every day, and a setup can learn from that. So I built a system around the setup, which is everything around the model, the tools and instructions it works with, what it is shown and the checks it runs. Scripts run the daily scan. AI agents read what is worth reading and test it on our own work. A tool saves a copy before each change goes in, so it can be undone. A week later, a check shows whether the agents use it. And a few rules stay with me. Nothing changes until it has passed a test, and only what the agents use stays.

Why it has to upgrade itself

Before I built this, I sent my agents a link to a design gallery, and 41 minutes later what they learned from it was a rule in the design briefs every agent reads, but it happened only because I found and sent that link myself. New methods appear daily; no one can find and send them all. In fields where you're not a specialist you wouldn't know where to start.

How much the setup matters

Princeton researchers tested one model, with the same settings, on repeating published scientific work. Moving it from a third-party research setup to the one built by its own maker took it from 51% to 95% of the tasks. In a second study, a Canadian university and an AI lab had a research system rewrite its own tools, testing every version on real tasks and keeping a growing archive of versions that still worked, and its success on software repairs went from 20% to 50% with the model unchanged. It matched the best human-built systems. The setup can matter as much as the choice of model, and what fits a model changes as models change.

The method, from scan to kept change

Every morning, with no person involved, scripts collect new tools and methods from public sources and from my own social media feed, and read the release notes of the tools I already use, so nobody has to keep up by hand. A small AI model answers a few factual questions about each item and fixed rules sort it into worth a look, worth watching or skip, so the scan decides only what gets read, never what gets adopted.

From here on the work happens in my working sessions with the agents. An AI agent reads the links I send and the items marked worth a look where they were published, and writes a verdict with a short note, so a claim is checked at its source before anyone spends time testing it. Then AI agents test what passed on our own past work, against a pass mark written before the test runs, and for a new tool a separate AI compares the answers without knowing which is which. Anything risky comes to me. A test that would spend more than the setup may spend alone, or that needs a new install or account, waits for my yes before it runs, and a larger project comes to me with its test result, so tested internal changes go straight in and the bigger decisions stay with me.

Once a change passes, an agent puts it in place through a tool that first saves an exact copy, so one short command undoes it, and builds it into a tool the agents already run. A week later an agent checks the sign of use named before the change went in, and a change nobody uses is removed, so the setup keeps only what the agents use.

Like a person, and not

In some ways it's like a person learning on the job. It learns from its mistakes and judges its own work first. A manager asks a person to check their own work before bringing it, then gives feedback, and working with an agent is similar. That self-check alone gets the work to a good level, and human judgment adds the last part of the quality. In other ways it's better. No person can research how the best in the world do something almost at once; that takes months of courses, sometimes years. The scan works overnight, what's worth a look is on my desk in the morning, I give feedback, and the agents iterate the same day. In one way it's worse. With a senior hire, a CMO for example, I'd give only the top-level goals and leave the how to them.

Lines that stay with a person

Every company has to decide what always needs a person's approval, what needs approval for a given role or department, and what the governance is. It depends on the company and on how mature the company is with AI. What models can do today matters too. These lines move as the models get more capable. I'm giving the agents more and more room.