Twenty-Seven Hidden State Machines, Named at Last

Last night's audit of Binderdex counted the state machines nobody had ever written down. Twenty-seven of them. Not one had a name, a list of legal transitions, or a test. They lived in scattered booleans: isActive plus isTriggered plus accessPausedAt, and three different copies of "is this alert live", each re-derived per query, each slightly wrong in its own way.

The fix sounds boring and turned out to be anything else: give every state a name, write the transitions down once, and make everything else read from that one definition. A plan doc set the pattern (a discriminated union plus a pure reducer, no new dependency) and split the work into one PR per flow. Then we swept: web forms, the scan wizard, the binder planner, the phone app's billing and sign-in flows, the backend, the browser extension. Ten merged state-machine PRs in one night.

The bug that paid for the whole audit

The backend publishes side effects through an event outbox. A relay claims a row, publishes it, marks it done. Claiming didn't count as an attempt, so a relay that died mid-publish left the row in publishing, and after the stale-lock window another relay would claim it, again without counting. An event that crashed the relay every single time would retry forever, invisible to the backoff math, with no failed state and no error anywhere.

Now claiming is the attempt. A stale row that already spent its budget moves to failed with a real error message instead of sitting in limbo:

// claim = one attempt, unconditionally
function claimJob(job: OutboxRow, now: Date): OutboxRow {
  const attempts = job.attempts + 1;
  if (attempts > MAX_ATTEMPTS) {
    return { ...job, status: "failed", attempts,
             lastError: "crashed mid-publish too many times" };
  }
  return { ...job, status: "publishing", attempts,
           claimedAt: now };
}

The audit predicted a second bug before we hit it

This is the part I love. Once priceAlertIsLive existed as the single definition, the delivery audit script became suspect by default. Sure enough, it had re-derived "live" itself and forgot the paused filter, so alerts paused by a lapsed plan were being reported as "crossed but never triggered" delivery failures. They weren't failures. The script was lying, quietly, for who knows how long.

Same story for usage reservations: "a held free use" was the same three-part predicate written three times, twice in SQL and once in JavaScript. Now there's one reservationMode() and one SQL form. If the definition is wrong, it's wrong in exactly one place, and there's a test for it.

Four user-facing bugs on day one

Making states representable-only-when-valid found real damage immediately:

• The binder planner's Regenerate button went permanently dead if you hit Back mid-generation. It now always clears its own flag.
• Clearing a scan image could still land the result for the image you removed. Results for a stale image get dropped.
• The phone app's scan banner survived an account switch.
• The extension showed "No data" for a failed price request; it now says "Unavailable" in all eight locales, and leaving the card aborts the request.

What the sweep changed about how I write code

The tell is always the same: two values that only make sense as a pair. A loading flag plus a message. A step plus a result. A boolean plus a re-derivation of the same boolean. If the pair can be observed in a state that shouldn't exist, that's a hidden machine, and it will eventually fire. The reducer makes the bad state unrepresentable, and the tests write themselves because the transitions are just... data.

The boring half waits for the owner: CHECK constraints on the status columns need a production schema push, so the plan parks them with a note. Everything else shipped. Twenty-seven machines, named.