Plugging a callback race in the NFL pipeline

When the framework fires your success and failure handlers for the same request

Seven PRs landed in Statpro yesterday, all on the NFL data pipeline. The interesting one was a callback timeout race: the crawl framework could fire both the success handler and the failure handler for the same request if the handler took longer than the framework's internal timeout.

Here's the shape of the bug. Your request handler captures a response, then hands the body to a persistence callback. If that callback takes too long, the framework times out the request and invokes the failedRequestHandler. But the original success callback is still running. Now you have two terminal paths for the same request, and whichever finishes first wins. The other one either silently double-persists or throws into a void.

The fix: claim, don't race

Instead of hoping the framework cooperates, the pipeline now coordinates terminal callbacks explicitly. Each request ID gets exactly one authoritative callback, claimed the first time any terminal path fires. Late arrivals join the existing promise rather than starting a second one.

claim(requestId, callback) {
  const owned = this.owned.get(requestId);
  if (owned) return { ownership: "joiner", result: owned };

  const claimed = Promise.resolve().then(callback);
  this.owned.set(requestId, claimed);
  return { ownership: "owner", result: claimed };
}

The ownership field tells the failure handler whether it's the real owner or just joining. A joiner releases the framework slot immediately; the owner's promise is still drained at the end of the run, so nothing escapes.

The rest of the day

Beyond the race fix, the day's work fell into two buckets:

• Roster policy: players whose profiles lack structured data now get handled gracefully instead of crashing the pass. The parser learned to fall back to headshot markup when the embedded JSON scripts are missing or empty.
• Stage bounds: the capture stage timeout went from 120 to 300 seconds, and the handler timeout itself is now bounded separately from the outer stage. A few unlinked-roster exceptions got accepted as known data gaps rather than pipeline failures.

The pattern I'd take away: when a framework owns the timeout and you own the side effects, you need your own coordination layer. Don't let two terminal paths race for a shared resource. Claim it, and make everyone else join.