Last night's big push on the collection tracker was a session trail: a text-event log in our analytics warehouse that can replay one user's experience across the mobile app, the website and every API call. But the more interesting part isn't the feature. It's who reviewed it.
The trail itself
The idea: when something breaks, you should be able to ask "what did this person actually see?" and get an answer. A shared contract defines three events, steps for 16 flows, and a sanitizer that scrubs emails and long token runs before anything ships. Mobile, web and API all emit the same shape. A replay command prints a session as one interleaved timeline, and an audit skill ranks the most broken, stuck, slow or unneeded steps by users affected.
The reviewer that keeps failing the PR
Here's the loop that made this PR different. An independent reviewer agent ran against the merge commit and came back NOT CLEAN with three blocking findings. A second round confirmed the fixes and found two new blockers. All six got fixed the same night:
- Replaying by user id collapsed every session and silently dropped all API events. The query now keys on the session id itself.
- The stuck-flow query hid any flow that failed, because it demanded zero successes. It now judges each attempt by its last outcome.
- Auth steps that get interrupted, like a purchase started before the app closed, ended on "success". There's now a query for starts with no result.
- Web sign-out logged success before the sign-out call even resolved.
Nobody on the team saw these. That's the point. Each bug was alive until an agent with fresh eyes traced the data through the whole path. The human's role shifted: make the calls (what to log, what's in scope, when to ship), then act on the reviewer's findings.
What it costs
One honest number from the commit body: the API alone serves about 157k requests a day, so recording every call can reach millions of events a month. Sampling is the brake. And it paid for itself before shipping: the trail revealed that a failed search request was being shown to users as "no results", which is a product fix with its own PR now.