Over the last few years we've rebuilt our test-authoring agent five times. In normal software, rewriting a core feature that many times is a flashing red light; when the models shift under you every few months, it's mostly just what staying current costs. The useful part is what survived every rewrite — five things that held no matter which model we were running on.
Work with the model's grain, not against it. Back in 2023 I tried to get PaLM to pick the single most similar word to a target in the DOM, for smarter auto-healing. It failed every way I framed it — but the outputs showed it clearly understood the task. It was bad at picking one similar word and surprisingly good at grouping words by meaning. So we rebuilt auto-healing around semantic grouping instead of arguing with it, and a hard limitation turned into a reliable feature. These models have a grain, like people do; you get further shaping the system around what they're already good at.
An agent is only as good as its input. Most of the agent failures I've chased turned out to be upstream of the model — a planning session stuffed with base64 screenshots the model can't read, a tool returning a vague error the agent couldn't act on, the right context buried under noise. The model usually wasn't the problem; the signal we handed it was. We've gotten far more mileage out of cleaning up what goes in — tighter tool definitions, scoped context, error messages written for the model to actually use — than out of swapping the model itself.
Embrace non-determinism on purpose. Testing is supposed to remove uncertainty, so deliberately putting a probabilistic model in the middle of it felt like a category error. It wasn't. When we leaned into letting a model judge whether an application state was right — in plain language, the way a person would — it expanded what automated testing could even cover. The trick was using the model where judgment beats a brittle assertion, not everywhere.
Measure trust with data, not vibes. For a while we tested our AI features with small hand-curated sets and spot-checks, and it left us blind to silent regressions. The fix was building evaluator suites — the same judge idea, pointed inward — that grade our agents across many runs. That's how we caught a regression hiding inside an upgrade: moving to Gemini 3 cut reasoning loops by something like 2–4x, but the same report flagged hardcoded values going up, because the model was now finishing harder tests that older ones gave up on. I'd never have spotted that by eye.
AI amplifies whatever foundation you've built. When we scaled coding agents across our repos this year, the biggest thing slowing them down wasn't the AI — it was build times, shaky CI, thin test coverage. The same fundamentals that slow people down. Fixing them helped the agents and the humans in equal measure. A non-deterministic tool doesn't make error handling, reusability, and tests matter less; it runs your weak spots over and over until they show.
What strikes me looking back is how little of this was about the model getting smarter. In 2023 we couldn't get a model through a login screen; today the model is rarely the bottleneck. The harder questions now are fit and cost — whether an agent has the context about what you care about, works inside the tools you already use, and earns its keep on latency and token spend instead of taxing the team. The bar I keep coming back to is whether it behaves like a good teammate.
I use Claude with the mabl MCP to create mabl tests. A lot.
New CLI feature? Sure, I write the integration test. Then I also ask Claude to create a mabl test that proves the feature works for real.
This week I'm improving the flow merge page. To work on that page I need a merge conflict, and a merge conflict takes some setup. Create a test with a flow. Branch it. Edit the flow on the branch. Go back to master and change the same flow there. That's a lot of clicking just to reach the screen I want to fix.
So now I type "Claude, use mabl and do that for me!" My job is to fix the merge page, not to make merge conflicts by hand.
Time to optimize your cloud estate. "Ship it" is no longer the end of development.
Dust off the 12 optional optimizations you never shipped because they died in Jira. Feed the fallow tickets into the agent forge. Now is the time for the work we never had bandwidth to do.
Last week, I dusted off an old ML pipeline—long considered too complex to replumb. No problem: select the fanciest LLM model, crank it to MAX effort, and feed it some good ideas.
Flatten and optimize queries? Done.
Materialize all views? Done.
Cache expensive calls in Redis? Done.
Vectorize computations and push down to machine instructions? Done.
Build a comprehensive end-to-end validation harness and suite to ensure these changes break nothing? Check.
OK... 98% cheaper? Pencils down.
It's time to revisit everything you've ever shipped. Juice that lemon until it turns to diamond. Capacity and expertise are no longer the limit. With agent weights and your horse sense, 98% optimization is table stakes.
Lean in. Today you can ask harder questions and produce more thorough analyses than ever before. You're the most capable you've ever been. Time to live your best coding life. Let's say you want to optimize your org's 100 CI workflows? You better instrument disk, memory, CPU, and network usage — OK... done. Now analyze the last 10K builds: what's the optimal build instance shape for _each individual_ workflow? Done. Now shadow-roll five custom instance shapes, raise dozens of PRs to consume them, and run 300 test builds. Done. Data? Reviewed. Statistical analysis, completed. Resources saved: 40%. Builds broken: zero.
You wouldn't have allocated the month it'd take to do this kind of work in 2024, but it's 2026 and your agent fleet just did it for you over the weekend.
It's always been true that if you don't give a team enough scope, they'll fill whatever scope you do give them, usually further than you anticipated. I've watched that at the team level for years. What's new is watching it happen to individuals. The people who were already deep in devops are building more devops, and better devops. Same with the framework people, the networking people, the ones who care about UX polish. Everyone leans into what they already know. Personal backlogs are filling up and getting churned through faster than they used to. This is largely a very good thing. But it does create some interesting challenges around how we think about architecture, prioritization, and team commitments, and we're still working through those.
I've been deleting a lot of code lately and I cannot tell you how good it feels. I cleared out about 400 stale translation keys — the kind that accumulate over years and that nobody dares touch, because who still knows what references them? The agent knows; it checks every reference in seconds, so the sweep I'd always filed under "someday" turned into an afternoon. And Eugene and Jeff finally pulled out Bootstrap (BOOTSTRAP! the framework we've been trying to get rid of for years!) — thank you both, truly. I re-ran our UI library market share report afterward and that's when the impact really landed: Bootstrap is finally gone.
What surprised me is how the agents change the math. Migrating class components to function components with hooks, ripping out an old framework, pruning dead i18n — these were always "we should do this eventually" jobs that lost every prioritization fight. The cost of removing code and cleaning up our codebases dropped enough that now we just... do it. I keep catching myself opening diffs hoping they're mostly red.
