JohnnyCode.ai Blog

Slop Is an Operations Problem

I kept failing to experience the thing everyone complains about. So I audited my own setup to find out why.

Published

Illustration for Slop Is an Operations Problem

Every week someone tells me AI writes garbage. Slop. Confident nonsense that reads fine and falls apart the second you look at it. Bland prose, invented APIs, code that compiles and lies. The complaint is everywhere, and the people making it are serious engineers I respect.

I kept nodding along and quietly wondering what they were talking about.

That gap bothered me for months. I ship production agentic systems. I have been writing code for forty-one years. If the tools were producing garbage at the rate everyone describes, I would be drowning in it. I was getting output I would sign my name to. Something about my situation differed from theirs and I could not name it, which is an uncomfortable place for someone who claims to understand this stuff.

So I did the only honest thing available. I audited my own setup. Every project memory file, every workflow definition, every hook, every review gate, every standing rule. I treated my own infrastructure as a hostile codebase and read it end to end.

The answer was four layers deep, and the model was never the variable.

The Somebody Else’s Problem field

Douglas Adams gave us the perfect instrument for this. The Somebody Else’s Problem field is cheaper than invisibility and infinitely more effective. It works because the human brain will not look at something it has decided is somebody else’s responsibility. You can park a spaceship at a cricket match under an SEP field and thirty thousand people will look right past it.

Slop sits under an SEP field. The industry has collectively decided that output quality belongs to the model vendor. So the complaint always terminates at the same place: the model is bad, the model hallucinated, the model got lazy this week. The vendor will fix it, or the vendor will fail to fix it, and either way the matter rests with somebody else.

That framing is why the problem stays unsolved for so many smart people. The thing sitting in plain view is the operating environment around the model, and almost nobody looks at it.

Here is my claim, stated plainly so you can argue with it.

AI slop is what model defaults produce in a context vacuum with no verification loop. I never operate in a context vacuum. Nothing I produce ships without verification. Slop still gets generated in my sessions, regularly. My system makes it unshippable.

You are reviewing raw output. I am reviewing the survivors of a pipeline.

The receipts, before the argument

Numbers from the audit, so you know what scale of apparatus we are discussing:

  • 85+ project workspaces under management
  • 25+ projects carrying curated persistent memory
  • 34 memory files in the primary repository alone
  • 2,300 lines of workflow definitions in that same repository
  • 40+ skills and slash commands on call
  • 3 models on the code review bench
  • 6 scheduled routines running unattended

That is the apparatus. Now the four layers it implements.

Layer 01: the agent never starts ignorant

My agents carry persistent memory across dozens of projects. These files are distilled facts with a deliberate structure I have never seen anyone else impose: a one-line description for recall, a Why section explaining the reasoning behind the rule, a How to apply section naming every code path the rule covers, and provenance back to the exact session where the lesson was learned.

One real example, lightly redacted:

---
type: feedback
origin: session 8dbef4da · 2026-04-29
---
Every work item comment carries two sections:
Developer Notes, then QA Notes in plain language.

Why: QA reads these. Technical notes alone are
unreadable to them.

How to apply: any path that posts a comment.
The skill, the CLI, anything. No exceptions.

A correction I made in April, still enforcing itself in August without my involvement.

The project instructions go further than facts. My primary repository’s playbook contains a production post-mortem converted into standing policy, including the flawed reasoning that caused the incident in the first place. The rule reads: never reason about framework defaults from memory, diff a real response body. The model reads a senior engineer’s institutional knowledge before it writes a line of code.

Compare that to the typical session. A developer opens a fresh window, types a request, and hands the model a task with zero grounding. The model has to fill the gap with something, so it fills the gap with its priors. Generic architecture. Average naming. The statistical center of every codebase it ever saw.

That vacuum-filling is the slop. People call the output generic and blame the model for being generic when the request contained nothing specific to be faithful to.

I wrote about the hygiene side of this in Prune Your Agent. Memory that accumulates without curation goes bad the way any dataset goes bad. Structure and pruning are the whole game, and both are manual labor nobody wants to do.

Layer 02: tasks run through engineered workflows

One repository of mine carries twenty slash commands and over 2,300 lines of workflow definitions. My global library adds 800-line skills for project kickoff, structured brainstorming, and phase-by-phase builds. Call this process engineering, versioned and refined the way code gets versioned and refined. Prompting is what you do when you have none of it.

The clearest artifact is my fully autonomous pull request reviewer. It runs unattended, on a schedule, against real repositories, and its instruction file includes a section titled “non-negotiables, repeated here so they cannot be missed.”

The non-negotiables:

  • Never ask the user anything. This runs unattended.
  • Enforce the creator gate before all else. Match on account identity id, never display name.
  • Never switch, pull, or stash the local checkout. Use a throwaway git worktree.
  • Wait for the upstream review to post before reviewing. Thirty minute hard cap.
  • Findings post only after evidence checks. Twenty thread cap, overflow disclosed.

Read those again and notice what they are. That is tighter operating procedure than most human code review processes I have worked under in four decades. An identity gate matched on immutable id rather than a display name that anyone can spoof. A disposable worktree so the agent physically cannot corrupt my working copy. A time cap so a stalled upstream review fails loudly instead of hanging. Mandatory disclosure when findings overflow the thread limit, because a silently truncated review is worse than no review.

Slop thrives where every task is a fresh improvisation against model defaults. Nothing in my environment gets improvised twice. The second time I do something, it becomes a workflow. The tenth time, it has non-negotiables.

Layer 03: nothing counts until it survives an adversary

This is the layer that carries the weight, and it is the one almost nobody implements.

Every substantial piece of work in my environment faces an opponent before it ships. Code review runs a multi-model bench: a fresh reviewing agent with no memory of writing the code, plus a rival vendor’s model, running in parallel and independently. A third model sits on that bench right now marked excluded until calibrated, because I ran it against known-bad code and its findings were noise. I calibrate my reviewers. I have met very few people who do.

Build workflows spawn an independent adversarial agent at every phase boundary. Its only job is to re-read the plan and the diff and poke holes. It has no stake in the work being good. A machine-generated dependency map turns impact analysis into a data lookup instead of a guess, so “what does this change break” gets answered with a query rather than an opinion. Test loops boot the real services and drive a real browser, because a mocked test suite is an agreement between two pieces of code that they will lie in the same direction.

The standard extends past code and into investigation. My memory corpus is stamped the way lab work is stamped. Findings carry verdicts: verified, proven, refuted. One root-cause analysis in there was corrected when new evidence arrived, and the correction is preserved alongside the original conclusion. One triage refuted its own premise, and that refutation is recorded as proudly as any proof. Team-shared facts stay invisible to other agents until a gatekeeper validates them against the actual code.

Investigations carry verdicts. Vibes do not enter the corpus.

I made the general case for this in The Adversarial Audit: every agentic workflow needs a second agent whose only job is to break the first one’s work. What the audit of my own setup showed me is how far the principle had spread without my noticing. It is in the code review bench, the build phases, the investigation standards, and the memory gatekeeper. Four independent implementations of one idea.

The people drowning in slop have zero implementations of it. They generate, they skim, they ship. The first adversarial read of their output happens in production, performed by a customer.

Layer 04: every mistake becomes permanent law

For most people a correction lasts exactly one session. You tell the model it got the comment format wrong, it apologizes, it fixes it, and tomorrow morning quality resets to baseline. You are Sisyphus with a context window.

Mine ratchets. The system converts feedback into standing law, and the law survives the session that produced it.

That comment formatting rule from Layer 01 is enforced at five separate points: the comment template file, the posting skill, the skill’s prompt file, the slash command, and the persistent memory itself. Any invocation path hits the same rule. Redundant enforcement is the difference between a preference and a law.

A workflow gotcha that burned me twice is annotated recurring in memory. It has never burned me since.

This is the quiet reason the gap keeps widening between my experience and everyone else’s. The model is not improving faster for me than for them. We are running the same weights. My system remembers every correction I have ever made and theirs forgets by morning.

There is a longer post in this, and I will write it.

Appendix: the tells are stripped at the config level

Slop is partly an aesthetic signature, a cluster of habits the defaults keep reproducing until readers can smell them from the first paragraph. My agents operate under standing house rules that strip that signature, so whatever survives gets judged on substance.

The banned list:

  • Em dashes. Anywhere. In any output. Ever.
  • Emojis and AI attribution in anything that gets posted publicly.
  • Native browser alert boxes. Real themed modals only.

The required list:

  • The house component library, never generic HTML inputs.
  • Every comment written for the audience that actually reads it.
  • Findings shown to a human before anything posts, unless the workflow is explicitly gated for autonomy.

This article was written under those rules. The em dash ban alone eliminates the single most reliable tell in machine-written prose, and it costs me nothing, because commas, colons, semicolons, and periods have handled English fine for several centuries.

The honest version

My going-in theory was that the difference came down to tooling, skills, or training. The audit says all three, plus one thing I had failed to name: nothing unverified ever leaves the pipeline. Findings are presented before they post. Database changes require a change-request form. The creator gate stops out-of-scope work cold, unattended, at three in the morning, with nobody watching.

So here is the precise claim, and it is sharper than the polite version.

People experiencing slop are consuming first drafts from an amnesiac model with no playbook, no process, and no checker. That is a rough description of an intern on their first day with no onboarding, no code standards, and no code review. You would not ship that person’s first draft to production either, and you would not conclude from the experience that hiring juniors is a scam.

My first draft is an input. It goes into a pipeline that grounds it, structures it, attacks it, and remembers everything it got wrong last time. What comes out the far end is what I show people.

The model is the same one everybody else has.

Slop is an operations problem. It has your name on it.

First published August 28, 2026 on 42 Insights.

← All posts