A test calls a tool to see whether the tool answers. Real work calls a tool to get something done, and the something is what surfaces the problem.

The clearest case was a validation bug. Changing a section heading on one post failed with an error naming a union type and a path into the block array. The section contained a code block whose language variant the schema no longer accepted. That block read fine. It rendered fine on the live site. It failed only on write, and because the update validated the whole section rather than the change being made, the heading could not be edited at all until the block was fixed.

No test pairs changing a heading with this post happens to contain a block written before the schema moved. A real editorial pass pairs them on post fifty of fifty-nine, because that is where the combination happens to live.

One instance is an anecdote. A second pass over the whole corpus found fourteen such blocks across thirteen posts, holding four different rejected values, in two shapes: a language name stored where the schema wanted a display variant, and a language the allowlist had quietly stopped accepting. Two of them were SQL, on a platform that runs on Postgres and publishes its own migrations. The narrowing that invalidated them shipped without migrating the rows it invalidated, and nothing anywhere reported that those rows now failed. The same thing had happened to an icon field, where a value that renders on the live site is absent from the current allowlist.

Narrowing a set of accepted values is a schema migration whether or not it is written as one. Existing rows do not re-validate on read, so the damage stays invisible until someone edits, and the person who edits is not the person who narrowed it.

The rule that emerged first, and the one that found the most, was to stop trusting the stored structure and render the page instead.

Six posts carried an image block with an empty source. Each had a caption and a written brief and no file, so the caption rendered on its own as a stray line of text in the middle of a paragraph. In the structure it looked like a normal image block. On the page it was obviously broken, and four of the six were already published.

Four sections on one post carried a role and no heading. The post rendered as one continuous wall of prose with no way in. Again, invisible in the structure.

Seven captions described images that were not there. One promised a screen mid-verification over a screenshot of two domains already connected. One promised redacted figures over a billing panel showing every number. One promised two languages over a code sample showing one. Each caption was defensible when written. None survived being looked at.

And one card linking to a code repository had every field set except the address. It rendered as a card, looked clickable, and went nowhere.

None of those are logic errors. All of them are the difference between what a structure says and what a reader sees.

The last one surfaced while writing about the work, which is the tidiest possible demonstration.

Creating a post accepts the whole content tree in one call, which is the fast path and it works. But a post created that way stored its sections, columns and blocks without generating identifiers for any of them. The post rendered perfectly. It was also completely uneditable, because every tool that changes a block needs an identifier to aim at, and every one of them was empty.

Saving the post once through the console backfilled them, which narrowed the fault usefully: the generation lived in the update path and not in the create path.

Same shape as the validation bug. Reads fine, renders fine, fails only when you try to change it. And the same reason it was never caught: a test that creates a post and checks the response would pass. Only creating a post and then editing it finds this.

This post is the proof it is fixed. It was written through the same fast path, and every block in it came back with an identifier, so the paragraph you are reading was editable the moment it existed.

The uncomfortable part is that not every finding was a bug.

Three times the agent looked for a capability under the name it would have given it rather than by what it does. It reached for a regular-expression search on a platform whose whole premise is embeddings, when the right move was semantic search to find the passage and exact search to get the block identifier. It decided publishing must live on the metadata tool, found it did not, and reported that publishing was impossible. There is a publish tool. It had been there the whole time.

Those are not code defects. They are discoverability defects, and the fixes were in the tool descriptions rather than the logic. The publish tool now states outright that it is the only way to take something live and that the metadata tool deliberately cannot do it. The search tool now states that a text query also searches section headings, and that an unknown block type errors rather than returning nothing.

A tool surface is not only its behaviour. It is also what a competent caller assumes about it, and the gap between those two is a real defect even when the code is correct.

Several things reported as missing were already built. Pagination and lightweight responses were not per-route features waiting to be added. They were global middleware every list endpoint already had. The specification said otherwise.

That distinction changed what needed doing. A missing feature is engineering. A wrong description is a correction, and until it is made every caller reasons from the wrong map.

A tool that accepts something it will not do is worse again. Renaming a post through the metadata tool returns success and changes nothing: the new name and address are taken in, discarded, and the old values come back in the response. Nothing rejects them and nothing warns. A caller who does not compare the response field by field against what they sent will believe the rename happened. Refusing an unsupported field is a worse experience for one call and a better one for every call after it.

For a company of one, this is the review a second engineer would have given the surface, done by the thing that was going to use it anyway.

The cost is that it is slow and tangled up with other work. The agent was not auditing. It was editing a blog, and the audit fell out of the friction. That is also the reason it worked, because the calls had intent behind them and the failures happened where a customer's failures will happen.

If you ship a tool surface for agents, have an agent do a long piece of real work through it before you decide the surface is finished. Not a test pass. A job with a deliverable, long enough that the awkward combinations come up on their own. Watch what it reaches for and cannot find, because that gap is a defect whether or not the code is wrong. And make it render what it writes, because a structure that validates is not the same as a page that reads.