Yesterday I wrote about splitting the Statpro NFL pipeline into independent refresh lanes. That fixed the "one stuck source froze the whole board" problem. It immediately exposed a worse one: if a game's final stats never fully arrive, the lane logic had exactly two moves, publish everything or publish nothing. So an in-progress game showed up as a blank row until the very end, and a source that died mid-night left nothing at all.
Half a box score, labeled honestly
The fix landed as explicit partial coverage: a game's stats can now publish as partial, with the refresh cycle fingerprinting whatever rows are still pending so the next pass knows whether the source went quiet or is just working through the quarter. The guarantee I care about: if the board says a game has stats, it says exactly how much of the game those stats cover.
The pattern generalizes. Any pipeline that publishes snapshots can probably borrow the shape: put the completeness marker inside the artifact rather than inferring it from outside. Before, "is this done?" was answered by absence: no updates meant done, or meant broken, and you could not tell which. Now the artifact answers for itself.
// What every published feed item now carries
type FeedItemCoverage = Readonly<{
status: "complete" | "partial";
// fingerprint of rows the upstream still owes us
pendingFingerprint: string;
}>;One labeling decision, and the difference between "no data" and "data pending" is machine-readable. I keep noticing that the bugs I chase longest are ambiguity bugs, not missing-data bugs.