The guard that wasn't wrong
Last Friday my build pipeline refused to run. The error was clean, almost polite: Certified question capacity exceeds safe bound: 1000001. One row over a million. The kind of number that makes you suspicious of an off-by-one before you suspect anything real.
It wasn't an off-by-one. A capacity stop I'd written months earlier, a runaway-enumeration guard capped at one million rows, had been sitting exactly at the ceiling of a single question family. The playerOpponents dimension legitimately saturates at 999,999 rows (floor of a million divided by three leagues, times three). So one dimension alone consumed the entire safety budget, and every other family got zero headroom. The guard fired not because anything was broken, but because the production source had grown to fill the exact hole I'd left.
Here's the part that took me a day to see clearly: the guard was correct when I wrote it. It was correct yesterday, and it will be correct tomorrow. It was never a wall. It was a countdown timer, and the countdown was set by the growth rate of data I don't control.
The instinct that misleads
When you write a guard, you reach for a number that feels safe. You look at the current row count, maybe you double it for headroom, maybe you round up to a clean power of ten. A million feels generous when your table has 19,000 rows. The instinct is to size the guard against what you can see.
But the production source isn't a snapshot. It's a river. Historical seasons get backfilled and new leagues get added. A dimension that sat at 12,000 rows last quarter can hit 999,999 the next time you run a full compile, because somewhere upstream a backfill job finished and nobody told you. The guard you wrote against Tuesday's data is answering a question about Tuesday. The system asking it has moved on.
I think this is the part of solo engineering that's hardest to internalize: most of your code is fine, and most of your bugs aren't bugs. They're time bombs planted by a version of you who didn't know the data would grow this fast, or grow at all. The failure isn't a logic error so much as a coupling you didn't notice, between a constant you hardcoded and a variable you don't own.
Size against the design ceiling
The fix sounds obvious in retrospect. Size the guard against the design ceiling. The design ceiling is the answer to a different question: what is the maximum this system is built to hold, ever, even on its worst day? For my question catalog that's not a million. It's the cross product of every player, every opponent, every season, every league, every stat slice the schema can express. That number is closer to eight million, and it's the right shape because it's derived from the schema rather than measured from a row count that changes every week.
The difference matters because of what each number is coupled to. A measured stop is coupled to the source. A design ceiling is coupled to your own schema. You control your schema; you don't control the source. When the source grows, a measured stop silently shrinks in real terms, and one day it fires on a system that's working exactly as designed. A design ceiling only moves when you change the design, which is the only time it should move.
Where else this hides
Once I saw it in the capacity stop, I started seeing it everywhere I'd reached for a "safe" number. The cache TTL I set to an hour because the data felt stable, and then the data started updating every five minutes. The batch size I capped at 500 because the table was small, and then the table got large and the batch became the bottleneck. The retry count I set to three because failures were rare, and then a flaky dependency made three feel like none.
Each of those numbers was reasonable the day I wrote it. Each of them was a countdown I couldn't see, because the thing they were sized against was moving and I'd treated it as fixed.
The pattern I'm trying to internalize now, slowly, is to ask of every constant: what is this number measuring, and is that thing actually stationary? If it's measuring my own design, fine. If it's measuring the world, the world will move, and the number will quietly stop being safe long before it stops looking safe.
The receipt that saved me
The reason this story has a clean ending is the same reason most of my week had clean endings. The migration pipeline is fail-closed by design: every step produces a verifiable receipt, and every receipt gets checked. The capacity stop didn't silently damage my catalog. It refused to run, printed the exact number that tripped it, and waited. I woke up to an error message instead of a quietly broken database.
That's the part I'm most grateful for, and the part I'd pass along to anyone building alone at night. A guard that fires loudly is a gift. A guard that fires silently is just a slower kind of bug. The difference between the two is not the guard itself but whether anything downstream is listening.
The stop is at eight million now. I'm under no illusion that eight million is permanent. It's just coupled to the right thing this time. When it fires again, it'll be because I changed the design, not because the river rose while I was asleep.