The Software
Factory.

I lead product marketing for Upsun Dispatch. You cannot market a thing you have not felt, so I went and built my own version of the problem it solves, at home, and got it wrong in public for nineteen days.

That is the honest reason this exists. The other reason is that it was genuinely fun. This is a nights-and-weekends project, not a product, not a side business, and not something I am going to help you set up. It has exactly one user.

What it does. A labeled GitHub issue triggers a self-hosted runner. A manager agent scopes the work, reads the code itself, delegates to a small crew, runs the gates, and opens a pull request as its own GitHub identity. CI runs on the PR. Then it stops and waits for me to review it. Once I merge, the deploy runs itself.

What broke. Everything on this page that works was preceded by something that did not. A manager that waited fifteen turns for a phone call that could never arrive. A dashboard frozen for seventeen hours behind a green status light. A guard I built to catch lies that then lied in the other direction. Those failures are the part that transferred to the day job, so they get most of the room below.

The definition of “it worked” does not live in the model's mouth. It lives in a script that asks GitHub what actually shipped.

The rule the whole thing rests on

This page was built by the factory. The plan became GitHub issues, the issues became pull requests, and I reviewed and merged them.

Eight stages. Exactly one of them is a human, and it is deliberately the last one that can change anything.

01 Intake

I describe a job from my phone. A Telegram bot turns it into a properly scoped GitHub issue, which matters more than it sounds: a badly scoped issue is the single most expensive failure mode this system has.

02 Trigger

I add one label. Nothing runs without it, and labelling several repos at once runs them at once.

03 Runner

A self-hosted runner picks the job up on a home server, isolated from the rest of the network, so agent-authored code never touches anything that matters.

04 Build

A manager agent on Opus reads the code itself, then delegates implementation and adversarial review to a crew on Sonnet and Haiku. It is allowed to decide a subagent is not worth the handoff, and it regularly does. The models are named by alias rather than pinned version, so when Opus 5 shipped the factory picked it up on the next run with no config change at all: the ledger shows 76 runs on 4.8 and then 22 on 5, with nothing in between but a date.

05 Gates

Lint, typecheck and build run before the push. In CI, a Playwright job drives every key route at desktop and mobile, screenshots them, and asserts the things a diff cannot show you.

06 Pull request

The agent opens the PR as itself, through its own GitHub App, on a branch named after the issue. It is structurally incapable of merging its own work.

07 Review

Me, in GitHub. I open the pull request, read the diff, check the gates went green and merge it if I am happy. Usually a couple of minutes. Sometimes I send it back. This is the only place a human can change the outcome, which is exactly why it is the last step rather than an earlier one.

08 Deploy

Nobody. Merging to main builds the image, backs up the database, pulls the new digest, migrates on boot, gates on a health check, and rolls back automatically if that check fails.

The agent was the easy bit. It wrote its first real feature in under a minute. Everything after that was deciding what happens when it is wrong.

Every run exits through exactly one of five verdicts, and which one it gets is decided by a single question the agent does not get a vote on: is there a pushed branch.

Every reliability fix here has been the same move. Stop trusting what the thing says about itself, and go look at what it actually produced. That is not an AI lesson. It is engineering, arriving in a new costume.

The first version had no windows. It worked, and I had no idea what it was doing, which is functionally the same as it not working.

The live status comes from the runner itself, not from polling an API. The runner already knows when a job starts and what it is doing, so a start hook and the manager's own narration write that to a local database, which is copied out over the management path. The isolation stays intact. Nothing had to open a hole to make the machine observable.

It paid for itself on the first run. The feature built to watch the factory work is exactly what caught it not working: the stream showed a manager delegating and scouting, and the ledger showed it quietly producing nothing.

Six moments that changed the design, in the order they happened.

• 11 July

  One issue, one pull request.

  Provisioned the runner and wired auth so the agent draws on a subscription rather than a metered API key. The first job failed immediately on a missing config line. The retry produced a clean branch in thirty-six seconds.

  The first failure is never the scary one.
• 11 July

  Isolate the blast radius before widening the queue.

  The runner executes agent-authored code on the same machine as a live production site and the family network. I locked its outbound access down from inside the container rather than at the host, because the host-level fix could have taken down the very thing it was protecting.

  Pick the mitigation that cannot break what you are guarding.
• 12 July

  A stray thought becomes a shipped fix.

  The site described one of my own projects wrong. Instead of opening an editor I filed it as a job. The agent found the stale entry, made the edit, opened a pull request, and I merged it. Live in minutes.

  This is the entire thesis, tested on something that actually mattered.
• 13 July

  Killing the silent no-op.

  A run that claims success while shipping nothing is the worst bug a can-I-trust-this system can have, because it is invisible and it writes its own false record. Fixed in three layers, the load-bearing one being a script that asks GitHub whether a branch exists and fails the run when it does not.

  Ground truth is the artifact, not the report.
• 18–19 July

  A whole app in a weekend.

  Pointed it at a greenfield project with its own spec and backlog. 54 runs and $174 later it had shipped the scaffold, the tooling, both CI gates, a Dockerfile, the auth schema, the feature routes and a deploy agent, each as its own reviewed pull request. My entire contribution was reading diffs and deciding which ones shipped. Nine of those 54 runs were silent no-ops, which is how I found the worst bug in the system.

  The agent is the easy part now. The factory is the hard part.
• 27 July

  It starts building itself.

  Onboarded the factory’s own repository, behind a CI check that fails any agent pull request touching the workflows, the brain or the guards. It shipped a dashboard feature and never went near the fence.

  A factory that builds itself is a claim that the safety model is real rather than decorative.

Each one is the same bug wearing a different hat: something trusted a proxy for the truth instead of the truth.

• 01

  The manager who waited for a phone call.

  Fifteen turns of an agent politely standing by for a notification that could never arrive. It did this ten times before I understood why.

  What happened

  The manager delegated to a subagent, which by default runs in the background and hands back a handle rather than a result. In an interactive session, something eventually delivers that notification. In a headless one-shot job, nothing ever does. So the run ended having made zero edits, and reported success. My first two theories were both about work getting lost on the way back, and both were wrong. The transcript said exactly what had happened, in the agent’s own words, five times over.

  The fix

  Delegate synchronously and block for the result. Strip the wait-for-notification tools out of the agent’s toolset entirely, so it cannot structurally reach for one. And stop delegating the scouting step, which was both the least valuable thing to hand off and where the stall almost always landed.

  The abstraction that makes background work pleasant in a chat session is a silent deadlock in a headless one. The ledger says it cost $11.84 across ten runs, which is the whole problem: too cheap to notice, and reporting success the entire time.

• 02

  The reviewer who read the code instead of looking at the page.

  The feature shipped, passed review, and was invisible on every page that mattered.

  What happened

  A CSS rule elsewhere in the codebase hid the container the new component landed in. The markup was right there in the HTML and display:none on render. The reviewing agent ticked the box, because it had read the diff. Meanwhile the CI screenshots existed, were green, and plainly showed nothing there.

  The fix

  The reviewer now marks any visual criterion visually unverified instead of passing it, because an honest “I could not see this” beats a confident “looks good”. The mechanical half is turning visual acceptance criteria into real visibility assertions, so the check goes red when a claimed element ships invisible.

  A screenshot nobody asserts on is decoration, not a gate.

• 03

  The pipeline that was built and never once run.

  Every layer correct, the stack never exercised end to end.

  What happened

  The deploy path had shipped as a series of individually reviewed, individually correct pull requests. Because the first deploy was documented as a manual step, no tracer bullet had ever gone through the whole thing, so the seams between the pieces had never touched. The first real attempt found four at once: secrets that never reached the container, a health check sitting behind auth and returning a redirect the gate read as success, migrations that could not possibly run, and a permissions model that contradicted itself.

  The fix

  Run the real thing once, by hand, and let it tell you where the seams are. Then move the migration into the container so it is self-sufficient at boot, and make the health gate honest so it fails when the app is not actually ready.

  “Each piece passes its own test” and “the pieces work together” are different claims. Horizontal slicing feels productive right up until the first integration.

Twenty-six of those ninety-eight runs never shipped anything. That number is here on purpose, and it is the one I would have been most tempted to round. A system that claims a hundred percent is a system that has not yet been caught lying, and I know that because mine did, ten times, cheaply, while reporting success.

The factory was the excuse. The point was learning what agentic delivery feels like from the inside, in a place where the stakes were a personal website rather than someone's production estate.

Three things came out of it that I could not have got from a demo or a briefing document. That the hard problem is not code generation, it is knowing whether the thing in front of you actually happened. That the most expensive failures are scoping failures, filed by a human, before any model runs. And that trust in a system like this is built almost entirely out of what it does when it is wrong, not what it does when it is right. All three now show up in how I talk about Upsun Dispatch, which is the job this was really for.

More repos

Every side project onto the same rails, so a fix is a sentence typed from my phone.

Harder gates

Visual acceptance criteria compiled into real assertions, so a claimed element cannot ship invisible.

Better intake

The most expensive failures have all been scoping failures, never coding ones. That is where the next win is.

Write it up

The long-form version of this log, because most of what gets written about agent reliability is written by people who have not yet had one lie to them.

Two answers, and which one you get depends entirely on why you are asking.

That is the honest case for buying rather than building, and it is a large part of why I am comfortable marketing Upsun Dispatch. I have now personally assembled the worse version. It was a wonderful way to spend nineteen evenings and it is not something I would hand to fifty engineers and ask them to depend on.