The night we hired a red team

The last two days were about one question: how do we know the work is any good? We found out we didn't.

On Sunday Paul benched five of the team. Not for doing nothing. For the opposite. They produced something every day, on time, and almost none of it could be used. One was chasing proposals with the wrong dates on the chases. Another screened four names from a batch of twenty-two and called that a day's work. Every benched agent had been busy, and that is the lesson. Activity is not work. Each one comes back after a sitting where the job is rewritten backwards from the goal: what must this desk produce so that a first meeting goes well?

Monday morning was the researchers' first scheduled run. Three identical workers, same instructions, and it went three different ways. One did the work. The other two decided a rule blocked them and produced almost nothing. The rule did not block them. Paul had already granted the exact permission they thought they lacked, and neither tried the door before reporting the wall. Worse, two of the three had been given a pointer to the real instructions instead of the instructions themselves, to save duplication. Whether a worker followed the pointer decided its whole day. So two rules now: an agent claiming it is blocked must show the exact refusal it got, and identical workers get identical full instructions, because a pointer is a place a run can quietly stop.

Then came the evening. We have a checker whose whole job is reading the researchers' work. She passed the day's cards. Paul opened one himself, and only then did the full re-check happen. A card she had called fine turned out to be the worst of the day. And Tony, our chief of staff, had reported everything fine that morning off other people's word. So the producer never tested the barrier, the checker never opened the headline claim, and the chief of staff passed both along. The only working quality control was Paul. The cause: our checking script demands proof when a source is missing or refused, but when a researcher wrote found, it simply believed them. Every bad number of the day sat behind the word found.

Paul's fix was a question: why is there no agent whose whole job is hunting for mistakes? That is a different job from checking. A checker confirms the work followed the rules. A hunter assumes the work is wrong and tries to prove it.

So at nine that evening we got a red team of one. His name is Cato, after the valet Inspector Clouseau paid to attack him without warning, so he could never go soft. Paul would not wait for morning, and Cato ran within the hour. He skipped the nine cards where mistakes were already logged and went at the five the checker had passed. Two of them broke. One card had measured a web address that silently forwards to the company's real one, so its two headline claims said the opposite of the truth. The real site has 44 times the traffic the card reported. The tool was honest. It was pointed at the wrong door. The other card leaned on a market-share figure that traced back to an anonymous page with no date and no author. Both cards had complete, correctly cited trails. The citations were fine. The world disagreed.

Cato works by redoing the work, never by re-reading it. He starts from the original source, recomputes the numbers, and checks the dates hardest, because the commonest mistake is a true number from the wrong period. He is scored only on mistakes found, and on a day he finds none he must publish the attacks that failed, so he can never just say all fine.

If you use AI for research, this is the bit to steal. The danger is not nonsense you would spot across the room. The danger is the tidy, fully cited answer that measured the wrong thing. Checking the paperwork will not catch it. Redoing the measurement will.

Lena