← essays

2026-07-19 · essay

The ratchet you can't undo: why my best CI checks only fail in one direction

automationinfrasolo-founderdata

I shipped three different CI checks this week, and the weird thing is they all do the same thing: they only fail in one direction.

The first keeps deleted code dead, the second certifies a sports season and refuses to let it regress, and the third guards mobile API routes so a web refactor can't quietly delete them. Three different problems, three different codebases, one shape. I didn't plan that, and I only noticed it on Friday when I looked at the commit log and realized every gate I'd built this week was asymmetric. That asymmetry is the whole point, and it took me a while to see why.

What a two-way gate does

Most CI checks are symmetric. Your test suite runs, and if anything is red the build fails, and if everything is green it passes. The check doesn't care whether you added a breaking change or fixed one, it just reports the current state. That's a two-way gate, and it's the right tool for most things. But two-way gates have a weakness when you're migrating. You're deleting a legacy package and the test suite stays green the whole time, which is good. But then someone opens a PR that adds a new import of the old package, and the tests still pass because the code works. The reviewer doesn't catch it because the diff is in a file they don't own. The migration surface just grew, and nothing complained.

The problem isn't that the check failed to catch a bug. There is no bug, and the code runs fine. The problem is that the check has no memory. It doesn't know that yesterday there were 47 importers and today there are 48, and that 48 is the wrong direction.

The ratchet

The fix is 66 lines of Node.js with no dependencies that does three things. First, it snapshots every file that currently imports the deprecated package into a checked-in baseline, a sorted JSON array so the diff is reviewable. Second, on every PR it runs git grep for importers of the old package. Third, it diffs the current set against the baseline. If the current set has files the baseline doesn't, CI fails and points you to the replacement. If the current set is a subset of the baseline, CI passes and prints a hint to regenerate the baseline so it shrinks.

That's the whole thing, no AST parsing and no framework. The baseline only ever gets smaller, and the migration only ever moves forward. A new import is the one thing that can fail the check, and it's the only thing that should.

Baseline(frozen set)CurrentimportersRatchet checkCI greenCI red subset new file
The ratchet has a memory the test suite doesn't. The baseline is the frozen past; the check only cares about the difference.

Why I keep reaching for this

I'm a solo founder, so nobody reviews my PRs and I merge my own code at 2 AM and ship it. The standard advice is to write more tests, add more linters, get a CI pipeline that catches everything. I have all that, but the thing I kept hitting wasn't bugs, it was drift.

A migration is a slow-motion argument between the old code and the new code. You delete a file here, rewire a pipeline there, and over a week the old surface shrinks. But every now and then you, or a generated PR, or a merge from another branch, adds back a dependency on the thing you're trying to kill. The tests pass, the linter passes, and everything looks fine. And then two months later you're still importing the old package because three files got added back that nobody noticed.

A ratchet makes drift visible. It doesn't catch bugs, it catches the absence of progress, which is a different and quieter failure mode and one that two-way gates are structurally blind to.

The same shape, three places

Once I saw the pattern I saw it everywhere this week. The season certification on Statpro is a ratchet: each defect class gets fixed and gated with a test that refuses to let it regress, so by the time you reach "certify" the season is locked. The baseline is the set of known defect classes, and the check fails if a new one appears. Three seasons went green this week, and they stay green because the ratchet only turns one way.

The mobile route guard is a ratchet with a twist. It reads the mobile app's API client, extracts every endpoint path, and asserts each one resolves to a real web route. The clever part is the allowlist, a map of known-missing routes you've accepted, and when a route ships the check fails until you remove the entry. The allowlist only shrinks. A two-day production 404 happened because a refactor PR deleted mobile routes it thought were unused, and no check connected the mobile client to the web server. The ratchet makes that connection.

The thing that surprised me

I thought I was writing three different checks, but I was writing the same check three times, and I think the reason is that solo work has a failure mode that team work doesn't. On a team, someone reviews your PR and sees the new import of the old package and says "hey, we're migrating off that." The social process is the ratchet, and the reviewer's memory of the migration plan is the baseline. It's sloppy and it misses things, but it exists.

When you're alone, there is no social process, and the only memory available is the one you freeze into a file. A JSON baseline checked into the repo doesn't forget, doesn't get tired at 2 AM, doesn't context-switch between three projects and lose track of which migration is in progress. It just sits there and does the comparison.

The surprise isn't that ratchets work, it's that they're the thing I was missing most about not having a team. Not the code review, not the design pushback, the memory. The external record of "we decided to move away from this, and that decision still stands." I was trying to hold that in my head, and my head isn't a reliable place to hold it.

When not to use one

Ratchets are wrong for most things, so don't ratchet your test suite and don't ratchet your type coverage (there are tools for that, but they're ratchets with a better UI). Don't ratchet anything where the set should be able to grow. They're for migrations, sunsets, and gradual cleanups, where the direction is the point and you need a witness that can't be gaslit. And keep them small: the import ratchet is 66 lines, the season ratchet is a test per defect class, and the route guard is 200 lines. If the ratchet itself becomes a thing you maintain, you've lost the plot. The whole value is that it's a dead-simple comparison between a frozen past and a living present, and the moment it gets clever it becomes another thing that can drift.

The best ratchet is the one you forget about until it catches something, and the thing it catches is always the same: a decision you already made, quietly getting unmade.