What makes a testing agent different from a coding agent?

I've been in a lot of discussions lately about independent testing agents, why you'd want one separate from your coding agent, and what makes it different. It isn't really about the model. Part of why coding agents feel so autonomous is that the verification problem was mostly solved for them. They inherit compilers, linters, test runners, CI, source control. Structured feedback that tells them, precisely, whether they're right or wrong. The model is impressive, but it's standing on decades of verification infrastructure it never had to build.

An independent testing agent inherits almost none of that. What it acts on isn't source code, it's a running app: non-deterministic, stateful, changing under it, and nothing volunteers whether it succeeded. There's also a reason you want it independent. A coding agent checking its own work is grading its own homework. You want a separate agent whose only job is deciding whether the product behaves. So its harness has to be built from scratch, and I've been working out what's actually in it.

Start with acting. A testing agent has to do what a user does. It clicks through a web app, taps through a mobile app, calls an API, so the harness has to give it real hands on the product across every surface customers use. But that part is getting commoditized. Any capable model can drive a UI or hit an endpoint now. Executing a test isn't the hard problem anymore.

The hard problem is verification. How does the agent know a login actually worked, a checkout actually completed, a page rendered the way it should? Generating the action is easy-ish; deciding whether the result was right is the whole job. Without a verification layer you haven't built an autonomous tester. You've built an autonomous clicker.

Verification starts with observing, deeply, the way a user would and a tester would. The agent needs screenshots, DOM state, network activity, logs, traces, the runtime behavior. But collecting evidence isn't enough. The harness has to compare what it saw against what was expected and decide whether the behavior was correct, and that judgment is the hard, valuable part.

It also has to do that efficiently. Easy to wave away until you run it at scale. The naive version, where you hand the whole DOM, every screenshot, and all the network traffic to the model on every step and ask "did that look right?", costs enormous tokens and time for one verdict. A good harness runs the cheap deterministic checks deterministically, saves the model for the judgments that need it, and uses what it already knows about the app so it isn't reasoning from raw pixels each run. At the scale a real suite runs, that's the difference between viable and not.

And verification compounds. Every run produces knowledge: which selectors are stable, which flows matter, which failures are expected, which recoveries work. A real harness keeps that and hands it to the next run. Without it, the agent shows up as a brand-new tester every time it opens the browser. It's most of why pointing a general-purpose agent at a browser only gets you so far.

And none of it matters unless people trust the verdict. The agent touches credentials, environments, and real data, so it has to run inside the same controls a person would, and explain itself: what it did, why it decided what it decided, what it saw and concluded. The harness isn't only constraining the agent; it's making its work auditable.

So here's where I've landed. Coding agents got a head start because software already had a verification harness. Compilers, tests, CI, and version control all tell an agent when it's right and wrong. Testing doesn't come with one, so it has to be built. And the hard part was never getting the agent to act. It's getting it to know what happened, judge whether it was right without spending a fortune to do it, and leave behind evidence the rest of us can trust.
share X LinkedIn

More posts

Claude and the mabl MCP now build my test scenarios, merge conflicts included

I use Claude with the mabl MCP to create mabl tests. A lot.

New CLI feature? Sure, I write the integration test. Then I also ask Claude to create a mabl test that proves the feature works for real.

This week I'm improving the flow merge page. To work on that page I need a merge conflict, and a merge conflict takes some setup. Create a test with a flow. Branch it. Edit the flow on the branch. Go back to master and change the same flow there. That's a lot of clicking just to reach the screen I want to fix.

So now I type "Claude, use mabl and do that for me!" My job is to fix the merge page, not to make merge conflicts by hand.

98% Improvement, or Bust

Time to optimize your cloud estate. "Ship it" is no longer the end of development.

Dust off the 12 optional optimizations you never shipped because they died in Jira. Feed the fallow tickets into the agent forge. Now is the time for the work we never had bandwidth to do.

Last week, I dusted off an old ML pipeline—long considered too complex to replumb. No problem: select the fanciest LLM model, crank it to MAX effort, and feed it some good ideas.

Flatten and optimize queries? Done.

Materialize all views? Done.

Cache expensive calls in Redis? Done.

Vectorize computations and push down to machine instructions? Done.

Build a comprehensive end-to-end validation harness and suite to ensure these changes break nothing? Check.

OK... 98% cheaper? Pencils down.

It's time to revisit everything you've ever shipped. Juice that lemon until it turns to diamond. Capacity and expertise are no longer the limit. With agent weights and your horse sense, 98% optimization is table stakes.

LFG.

Lean into the Discernment Horizon

Lean in. Today you can ask harder questions and produce more thorough analyses than ever before. You're the most capable you've ever been. Time to live your best coding life. Let's say you want to optimize your org's 100 CI workflows? You better instrument disk, memory, CPU, and network usage — OK... done. Now analyze the last 10K builds: what's the optimal build instance shape for _each individual_ workflow? Done. Now shadow-roll five custom instance shapes, raise dozens of PRs to consume them, and run 300 test builds. Done. Data? Reviewed. Statistical analysis, completed. Resources saved: 40%. Builds broken: zero.

You wouldn't have allocated the month it'd take to do this kind of work in 2024, but it's 2026 and your agent fleet just did it for you over the weekend.

Lean in. LFG.

Personal backlogs now churn fast, mostly deeper into what people already knew.

It's always been true that if you don't give a team enough scope, they'll fill whatever scope you do give them, usually further than you anticipated. I've watched that at the team level for years. What's new is watching it happen to individuals. The people who were already deep in devops are building more devops, and better devops. Same with the framework people, the networking people, the ones who care about UX polish. Everyone leans into what they already know. Personal backlogs are filling up and getting churned through faster than they used to. This is largely a very good thing. But it does create some interesting challenges around how we think about architecture, prioritization, and team commitments, and we're still working through those.

Deleting 400 stale i18n keys and the Bootstrap we'd carried for years

I've been deleting a lot of code lately and I cannot tell you how good it feels. I cleared out about 400 stale translation keys — the kind that accumulate over years and that nobody dares touch, because who still knows what references them? The agent knows; it checks every reference in seconds, so the sweep I'd always filed under "someday" turned into an afternoon. And Eugene and Jeff finally pulled out Bootstrap (BOOTSTRAP! the framework we've been trying to get rid of for years!) — thank you both, truly. I re-ran our UI library market share report afterward and that's when the impact really landed: Bootstrap is finally gone.

What surprised me is how the agents change the math. Migrating class components to function components with hooks, ripping out an old framework, pruning dead i18n — these were always "we should do this eventually" jobs that lost every prioritization fight. The cost of removing code and cleaning up our codebases dropped enough that now we just... do it. I keep catching myself opening diffs hoping they're mostly red.