Deep Thought Ran Locally

For the past few weeks I have been running a coding experiment on my own hardware. Qwen3.8-27B, running entirely locally on a Mac M4 Max. No cloud. No API calls. No data leaving the machine for the actual code generation.

This was a real test. The case was a corporate change request against a production SaaS application, a mature codebase with years of accumulated decisions sitting in it. The work touched a multi-tenant permissions layer, required schema changes with a backward-compatible migration path, and rippled out across a couple dozen files and their tests. Writing the code is the easy half of a request like that. The hard half is leaving intact the three things nobody remembers depend on it.

This is the exact class of work that the industry has spent two years filing under “frontier cloud model only.” Local models get the autocomplete jobs. Local models get the boilerplate, the unit test stubs, the throwaway script. Anything with real blast radius goes to the API, because that is where the capability lives.

It finished the change request. Under adversarial review, at quality I put my name on, on a machine that has never once phoned home.

The model turned out to be the least interesting part of the experiment. The review loop is where the finding lives.

Models grade their own homework generously

Ask a model to write code. Then ask that same model whether the code is any good. It will tell you the code is good.

The easy explanation is context. It wrote the thing, the entire justification chain is still sitting in its context window, and every design decision in there already survived its own scrutiny once at the moment it was made. Asking a second time re-runs the same reasoning over the same priors and arrives at the same verdict. The model is being consistent. Consistency is exactly the problem.

That explanation is true and incomplete, and the incomplete half is the one that matters.

Clear the context. Open a fresh session. Hand the same model the diff cold, with no memory of having produced it and no justification chain to defend. The review gets better. It still comes in weaker than a review from a different model, and I have run this comparison enough times to stop being surprised by it.

The reason lives in the weights. A model’s failure modes are a property of how it was trained. When it produced a subtly wrong permission check, it produced that check because its priors said that shape was correct. Show the same weights the same code an hour later in a clean session and those priors are unchanged. The model reads the defect and sees a reasonable implementation, because a reasonable implementation is exactly what it believed it was writing. It cannot see past the edge of its own training, and clearing the context window moves nothing about where that edge sits.

Context hygiene fixes the flattery. Architecture diversity fixes the blind spot. You want both, and a single model can only give you one.

Human engineering organizations solved this decades ago. They invented code review and handed the job to somebody who did not write the code. Nobody considers that an insult to the author. We built the practice because we understood that the person closest to the work has the worst view of it. Everyone who has ever stared at their own typo for twenty minutes knows the feeling from the inside.

We dropped that lesson the moment we started letting models review themselves.

Adversarial review, in full

Adversarial review splits the two jobs on purpose and refuses to let one system hold both.

One model builds. A second model receives the finished work with an explicit hostile mandate: find what is broken in here. The framing does more work than people expect. “Review this code” produces a courteous summary and two naming suggestions. “Assume there is a defect in this diff and prove it exists” produces a list. Same model, same code, wildly different output, because the second prompt makes finding nothing the failure condition rather than the happy path.

The critic reports findings. The implementer fixes them. The cycle repeats until the critic stops surfacing anything real.

That last word carries the weight. A critic operating under a hostile mandate will always return something, because you told it to. The termination condition is the point where the findings stop being defects and start being preferences. Recognizing that boundary is a judgment call, and it stays a human one. I make that call myself on every loop, and I have written before about why every agentic workflow needs a second agent whose only job is to break the first one’s work.

Three details separate a review loop that works from one that performs the motions.

The critic gets a different architecture. This is the load-bearing detail, for the reason laid out above. Two instances of the same model share training data, share failure modes, and share blind spots, whether or not they share a context window. Point a model at its own family’s output and you get agreement dressed up as scrutiny. In this setup the local model builds and Claude Opus attacks. Different architecture, different training set, different set of things each one fails to notice. Defects survive review inside the overlap between two systems’ blind spots, and the way to shrink that overlap is to make the two systems less alike.

The critic does not receive the implementer’s reasoning. It gets the diff, the original change request, and the surrounding code it needs to understand the blast radius. It does not get the explanation of why the implementation went the way it did. Withholding the rationale is deliberate. Rationale is persuasive, and a critic reading a confident justification tends to grade against that justification rather than against the requirement. Give it the artifact and the requirement, then let it work.

The loop runs at phase boundaries. I decomposed the change request into four phases: schema and migration, permissions layer, API surface, test coverage. Review fires at the end of each phase instead of once at the end of everything. A permissions defect found at the end of phase two costs one fix. The same defect found after phase four has three phases of code built on top of it. Catching problems while the surrounding code is still soft is most of the value.

The result surprised me

Claude typically converges in one to two review cycles. Qwen needed nine to ten to reach the same point.

It reached the same point. The finished implementation was equivalent quality by my read: same migration safety, same permission boundary handling, same test coverage, same behavior against the change request. I would ship either one.

The gap shows up in the shape of the cycles rather than the destination.

What the loop actually caught

If you have ever walked a strong junior developer through five rounds of pull request feedback, you already know the shape of this. The early rounds are surface. The middle rounds are where you find the actual bugs. The last rounds are where somebody notices the thing that would have paged you at 3am in March. Ten review cycles with a local model follows the same arc, compressed into an afternoon.

Cycles one and two were noise. Rename this variable. Extract this into a helper. Add a null check on a value the framework already guarantees. One finding objected to a pattern the codebase uses deliberately in forty other places. This is the stretch that makes people give up on local models, because round one genuinely looks like a machine flailing at a wall. Every one of those findings cost me a few seconds to dismiss.

Cycles three through five found real defects. Three that mattered:

The migration ran its backfill and added the new constraint inside a single transaction. Correct on my dev database with four hundred rows. On the production table, that shape takes a lock long enough to time out every request in flight.

The new permission check pulled the tenant identifier out of the request payload where the surrounding code pulls it from the session. A caller who edits one field in a JSON body reads another tenant’s data. That defect is in the top five causes of real multi-tenant breaches, and it looked completely ordinary sitting in the diff.

One of the generated tests asserted the new behavior while seeding its own fixture with an administrator account. The test passed. It would have passed with the permission layer deleted entirely.

Cycle eight is the one I keep telling people about. The permission set gets cached, and the generated code built the cache key from the user identifier alone. The existing codebase prefixes every cache key with the tenant. Users who belong to exactly one tenant behave perfectly, which is nearly everyone, which is why every test passed. A user who belongs to two tenants gets whichever permission set landed in the cache first and carries it across the boundary until the entry expires.

That is the argument for this entire technique. Every test in the suite passes, because no test in the suite was written to look for it. It ships clean. It surfaces eleven months later as a support ticket that takes two engineers a week to trace, filed by your largest customer, who is exactly the kind of customer with users in more than one tenant.

Cycle ten is how I knew it was done. Three findings came back: a naming suggestion, a request for a docstring, and a proposal to extract a helper. Zero of them changed behavior. The critic was still working hard, and it had run out of defects to find. That is the signal.

Iterations are free, so spend all of them

Every review cycle against a frontier API is a metered event. Input tokens for the diff, input tokens for the surrounding context, output tokens for the findings, then the entire exchange again for the fix pass. Ten cycles is a line item somebody in finance eventually asks about. I have written about the token tax nobody budgets for, and review loops are one of the fastest ways to run that meter.

On a machine I already own, iterations are free.

(Somewhere a reader is drafting a comment about the electrons. Yes. There is a Mac drawing power and a utility company that noticed. The difference between two cycles and ten would require a lab-grade meter to detect and a personality I do not have to care about. Consider the pedantry acknowledged and the argument unchanged.)

Free rewrites the strategy, and it rewrites more of it than people expect. Run the review loop until the critic stops finding defects instead of until the budget flinches. Regenerate an entire phase from scratch when the critic finds something structural, because patching is only preferable when regeneration has a price. Generate three implementations of the module you are most worried about and let the critic pick the survivor. Every one of those techniques is indefensible at API prices. All of them are free on hardware sitting three feet away.

Wall-clock time is the only cost that survives, and wall-clock time detaches cleanly from my attention. The loop runs while I write documentation, review a colleague’s pull request, or eat lunch. Nine unattended cycles on my own machine belongs in a different category of expense from nine billed API round trips.

Free attempts break the leaderboard

Model comparisons measure convergence speed almost exclusively. How good is the first draft. How few shots to a working answer. Every benchmark rewards arriving fast, and arriving fast is the right thing to reward when you pay per attempt.

Free attempts break that metric.

What matters on owned hardware is the ceiling. Does the model converge at all, and where does it land when it does? A model that reaches production quality in ten cycles and a model that reaches production quality in two are the same model to me. I own the machine. I can wait.

The failure mode worth watching for is different too. Some models never converge. They oscillate, fixing the critic’s finding by breaking something adjacent, then fixing that by reintroducing the original defect. I have watched smaller models ping-pong for six rounds without net progress. Convergence is a real property and a more binary one than the benchmarks suggest.

Qwen converged. It took the long way. The destination was the same address, and the address is the part that was supposed to be out of reach.

Don’t Panic, and read the fine print

One developer. One change request. One codebase. One local model against one cloud critic.

This is an anecdote with a methodology attached rather than a benchmark. I did not run the request N times, vary the prompt structure, or control for the fact that I have been shaping how I write change requests for agents since the first useful ones showed up. A single trial from a practitioner who wanted a specific answer is exactly the kind of evidence you should hold loosely.

The specific numbers will also rot fast. Local model quality is moving quickly enough that the nine-to-ten figure is a snapshot of August 2026 and nothing more durable than that.

What I will defend is narrower and much sturdier. A 27B model running on laptop-class hardware completed a genuinely difficult change request in a mature production codebase, under adversarial review, at quality I would put my name on. Eighteen months ago that was a thing people said would happen eventually. It happened on my desk this month, and the proof required no permission, no procurement, and no vendor.

The long way to the same answer

Deep Thought spent seven and a half million years computing the Answer to the Ultimate Question of Life, the Universe, and Everything. The mice who commissioned it had opinions about the timeline. The answer was still correct.

Convert that into the only unit this industry accepts. My Mac generates about 30 tokens per second running this model. Seven and a half million years at that rate comes to roughly 7 quadrillion tokens, which is somewhere around four hundred times the entire training corpus of a frontier model, spent on inference, to produce an answer that fits in a single token. At mid-tier API pricing that reasoning trace bills out north of twenty billion dollars, which tells you everything about why the mice went with an on-premises deployment.

The dismissal of local models has hardened into received wisdom across a lot of engineering organizations. Too small, too dumb, fine for autocomplete, useless for real work in a real codebase. That assessment was accurate for a while, and it stayed accurate long enough to calcify into something people repeat without retesting.

It is worth retesting. Especially if you have data that cannot leave your building, a compliance boundary you cannot cross, or a finance team that will not absorb metered inference at production volume. Take a real change request, run it on your own hardware, and put a review loop on it that sincerely tries to break the work.

The ceiling is higher than you have been told. The path there is longer, and on hardware you own, the path costs nothing.

First published August 22, 2026 on 42 Insights.