Until last night, Statpro refreshed its NFL data the way most side projects do: one big scheduled job that collected everything, then wrote everything. That works fine until the injuries source hiccups at 2am and takes the whole schedule down with it. Last night I split the pipeline into independent refresh lanes, each bound to a certified release, and the 11k-line diff is what it took.
One parser token nearly froze the whole board
The night started with a bug, as these nights do. Games that ended in overtime were arriving with the status FINAL OVERTIME, and the strict parser stored that status as unknown. It sounds harmless. It is not: a game whose status is unknown cannot be finalized, cannot publish its score, cannot leave the board. One missing token would have quietly frozen a whole week of results.
The fix itself was five lines. The interesting part is the guard around it:
const exactToken = value.trim().toUpperCase();
const token = exactToken.replace(/[ -]+/g, "_");
if (token === "FINAL_OVERTIME" && exactToken !== token) return "unknown";
return statusTokens()[token] ?? "unknown";Only the exact spelling FINAL_OVERTIME maps to final. FINAL-OVERTIME or FINAL OVERTIME stay unknown, fail closed, and the test suite now treats that as a regression. The strictness is not pedantry: if sloppy variants mapped to final, a mid-game LIVE banner could masquerade as a finished game, and a wrong final is worse than a delayed one.
Independent refreshes, each bound to a certified release
Injuries and games come from different sources, update on different rhythms, and fail in different ways. When they share one schedule, one broken source stalls both. So each lane now runs its own daily refresh with its own deployment switch, and every write is gated by a certified release check: before persisting anything, the job verifies the payload belongs to the expected source release. Wrong or stale release, no write. That is the difference between "the data changed" and "the data can only change when the source provably advanced".
The games lane goes further: publication is atomic per game. Each game commits inside one transaction under an advisory lock, so a game either publishes complete or not at all. There is no state where readers see a score but no final status.
The shape of the whole thing, as a topology:
flowchart TD
T["Nightly trigger"] --> I["Injuries refresh"]
T --> G["Games refresh"]
I --> C["Certified release gate"]
G --> C
C --> A["Atomic publish"]
A --> R["Reconciled receipts"]
Two lanes, one gate, and a paper trail at the end. (This post was supposed to carry a hand-drawn D2 version; the media upload endpoint is having a bad morning, so Mermaid it is.)
Receipts and alarms close the loop
Every refresh now leaves a receipt: what it fetched, the payload hash, how many reports were inserted or reused, which heads advanced and which stayed put. A nightly alarm fires if a lane's receipt goes missing. That sounds like paperwork until the night a schedule silently stops running, and you find out from a missing receipt instead of an angry user.
I nearly shipped the gate looser than this. The obvious version accepts any payload that looks fresh. The paranoid version, the one that landed, asserts the exact source release before the transaction opens. The difference shows up weeks later, when a backfill sneaks in and the loose version writes stale data over good. I've been burned by exactly that before, so paranoid it is.
Also shipped while the pipeline burned
One non-football thing made it in: mobile users can now delete their accounts through a verified flow, with the privacy entry points to match. Unglamorous, increasingly required by the app stores, and exactly the kind of thing you want to be boring. It is boring now. Good.
What you can steal from this
- Split schedules by failure domain: a broken source should stall only its own lane, never the whole pipeline.
- Gate writes on a certified release: the write asserts which source release it belongs to, and a stale or unexpected payload writes nothing.
- Publish atomically per unit: each game is one transaction under an advisory lock, so readers never see a half-finished game.
- Leave receipts: fingerprint every run, then let an alarm tell you when a receipt is missing.
The season is one week old and the pipeline already caught its first overtime final. If week two breaks something else, at least it'll break one lane instead of five.