← Notes

August 2026 · Raian Pollock

Nobody at my company writes the reports

Two of us ran Flip Education across 18 countries and 11 languages. For most of that time, the way we found out something was broken was that a customer wrote in to tell us. That sentence is worth sitting with, because it's how most companies your size still work, whether they admit it or not.

Here's what changed, in the order a CEO would care about it.

What I can see now

A synthetic user tries to log in, generate something, and load the home page every 5 minutes, around the clock. If login breaks at 3am on a Sunday, I know at 3:05. Before, I knew when the first customer got annoyed enough to email, which is usually hours later, and only from the ones who bothered. The rest just left.

The login and password-reset emails get probed every 10 minutes with a test address. That one matters more than it sounds. When the path breaks nothing errors anywhere, so the user requests a reset, waits, never gets the mail, and quietly goes away. There's no angry ticket. There's just a person who never comes back, and no record of why.

Every hour, something reads the support conversations and picks out any customer who has sent two or more messages and is still stuck. That's the customer who was about to churn and tell nobody. Before, catching them depended on one of us happening to scroll the inbox at the right moment.

And every hour, something checks whether the core product feature is producing worse results than it did yesterday. Not broken. Worse. A feature that silently degrades is the hardest thing in a company to catch, because every dashboard says it's up. This is the one I couldn't see at all before. Now it's a number I get told about.

Same feature, same fortnight day 1 day 14 What the status page says: up, up, up, up What customers actually get nothing here errors
The hardest failure to catch is the one that never fails. Nothing alarms. The feature still answers, just worse than last week, and no part of that is an error.

How much faster

The old lag was measured in customer patience. Something broke, some unknown number of users hit it, one of them eventually wrote in, and then we started looking. Call it hours on a good day. Days on a bad one. Never, for the silent failures.

The new lag is 5 minutes for anything a user touches, 10 minutes for email delivery, an hour for a struggling customer or a degrading feature. In the 100 days to 24 August 2026, this produced 750 named problems across 142 alerts. Named, as in: here's what it is, here's where, here's when it started. Not a vibe that something seemed off.

What gets checked Now Before Can someone log in and use it every 5 min a customer wrote in Do password reset emails arrive every 10 min a customer wrote in Is a customer stuck after two messages every hour nothing watched this Is the main feature quietly getting worse every hour nothing watched this The bottom two have no manual version. Nobody rereads every support thread hourly, and nobody reruns the product against yesterday.
The top two replaced a slow way of finding out. The bottom two replaced nothing at all, because before this existed there was simply no way for either of those things to reach anybody.
The top of a real daily report dated 16 August 2026. Under the company header, a highlighted verdict explains that a wave of 72 automated visitors flooded one part of the app, that none of them got through, and that the zero percent success rate this produced is not an outage.
The top of one real daily report, 16 August 2026. That highlighted block is the report's verdict, and it sits above the numbers rather than under them. It's explaining that a success rate of zero isn't an outage, because a wave of bots had inflated the count it's measured against. On its own, that number would have ruined somebody's Sunday for no reason at all. Nobody wrote this one.

What it removed

No role got hired. That's the cleanest way to put it. The work of watching the company, the part-time ops person, the Sunday evening inbox sweep, the weekly "is everything actually fine" meeting, none of it exists. Two people ran the whole thing.

It also removed things we'd built that turned out to be waste. 24 agents have been retired. Reporting that nobody read got switched off. An audit found two agents everyone assumed were working that weren't, and five more exist as code that nothing schedules, so they never run at all. That last one taught us something. The system watches itself now, because the first version of "we have monitoring" turned out to be partly fictional.

What it would cost to switch off

If you turned it off tomorrow, I'd go back to learning about outages from customers. I'd lose the stuck-customer warning entirely, because no human rereads every support thread every hour. I'd lose the degradation check entirely, because no human reruns the product against yesterday's output every hour either. Those two don't degrade gracefully into manual work. They just stop existing, and the company goes back to being surprised.

The machinery underneath is boring on purpose. 56 agents are live. 54 of them each query one outside system and write down the answer; 2 write the actual reports and never go back out. Every outside system gets queried exactly once, so two reports about the same week can't disagree. 30 of the 56 use no AI model at all, because counting and comparing don't need one. None of that is the point. The point is the four answers above.

The part that isn't solved

An 1910 patent drawing of a watchman's time detector: a large pocket-watch case opened flat, a clock dial in the lid on the left, and a numbered paper recording disc on the right.
A watchman's time detector, patented 1910. The watchman carried it on his rounds and turned a key at each station, and the paper disc inside recorded whether he had actually been there. Automate a check and you inherit his problem. Something has to prove the check ran. U.S. Patent No. 973,383, 1910.

Here's the honest cost. Of the 56 agents, 28 email the founders, and only 2 stay silent when nothing is wrong. Which means I read a lot of mail from my own company. Most of it says some version of "checked, fine." That's the price currently being paid for the visibility, and it's a real price. Attention is the thing two people have least of, and the system spends it carelessly.

The fix is obvious in shape. Only tell me when something's wrong, or when something changed. Getting there without losing the reassurance that the checks actually ran is harder than it looks, and we haven't finished it. You should know that before you buy anything from me. The systems worth trusting are the ones whose failure modes somebody told you about in advance.

This is what I'd install first at your company. It changes what you're able to notice, and everything else builds on noticing. If you want to know what your company looks like when it reports on itself, email me.