I spent the last 26 hours rebuilding how Binderdex scans cards. The scan pipeline went from a single-shot camera snap that matched 24% of the time and took 4+ seconds to a hands-free auto-capture system that locks onto a card, fires the shutter for you, and hits a 92% match rate. This is the story of how it got there, and the mobile OCR architecture that finally worked.
The problem: every scan was a gamble
The old scan flow was simple and bad. You opened the camera, tapped the shutter, waited for a vision API round-trip, and hoped. If the lighting was dim or the collector number was smudged, you got nothing. The match rate sat at 24%. The median scan took 3.8 to 4.4 seconds on my test device. Every scan was a coin flip that cost you four seconds.
Phase 1: text-first instant scan
The first breakthrough was realizing most of the time you don't need the image at all. The collector number on a card is printed in a fixed spot with a known format. If you can OCR that number and match it against the catalog locally, you skip the image upload, the vision call, and the R2 storage round-trip entirely.
The text-first path runs ML Kit OCR on-device, tries to resolve the collector number against a cached catalog snapshot, and only falls back to the full image pipeline if the text path comes up empty. Here's the simplified decision flow:
// Simplified: the fast path that skips the cloud
async function scanCard(frame: CameraFrame): Promise<ScanResult> {
const ocrText = await mlKit.recognize(frame); // on-device, ~ms
const collectorNo = parseCollectorNumber(ocrText); // "134/108" not "134"
if (collectorNo) {
const match = catalog.lookup(collectorNo); // local hash match
if (match) return { ...match, viaHints: true };
}
return scanViaImage(frame); // the old pipeline, already prepared
}The key fix in the parser was capturing the full fraction. A card printed "134/108" tells you the set (108 cards) and the card's position (134, which means it's a parallel or promo). The old parser grabbed "134" and lost the denominator entirely. Fixing that one bug alone turned a 2.2-second "hinted" scan that still went to the server into a local-only instant match.
Phase 2: hands-free lock-and-fire
Text-first made single scans fast. But the real goal was scanning a stack of cards. You sit down with 30 cards, and you want to scan them all without tapping the screen 30 times.
The solution is a lock state machine: the camera preview polls at ~2fps through the existing ML Kit read, and when a collector number stabilizes for 900ms, the machine fires a single 2MP capture into the unchanged scan pipeline. No shutter sound, no per-sample JPEG, no polling loop. After firing, it cools down until the card leaves the frame (or the next card appears, which starts locking immediately).
The tricky part wasn't the state machine. It was gating on the binary's native module availability. React Native's TurboModule probes misfired under bridgeless interop, and Metro's dev require reported module-factory errors as FATAL to LogBox before any catch block could run. The fix was to gate on the binary's native app version directly (>= 1.1.3, the first build shipping the module) rather than probing at runtime.
Phase 3: frame-output OCR (the architecture that stuck)
The polling approach worked but was fragile. The final architecture (v1.1.4) moves to vision-camera v5's Frame Output: ML Kit OCR executes per-frame in a JS worklet, text blocks outside the on-screen card frame are filtered before parsing, and reads feed the same lock state machine. A lock fires one pre-warmed capturePhotoToFile into the scan pipeline.
The results
Match rate went from 24% to 92%. Scan time dropped by roughly 2x (the text-first path resolves in milliseconds when the collector number is readable). You can now scan a stack of cards without touching the screen. The batch tray keeps the camera on screen the whole time: matches above 0.8 confidence drop straight into a floating chip with price and running total, weaker matches open the picker, and the review screen adds everything in sequence.
One thing I tried and reverted: dropping the vision input from 1024px to 768px to save upload bytes. The probe showed digit misreads on fine print (167/159 became 163/159). The model kept 1024px. The catch: the obvious "reduce image size for speed" move would have quietly degraded accuracy.
The Android minSdk bumped from 24 to 26 because the OCR plugin locks frame buffers via AHardwareBuffer (API 26+). That makes Android 8.0 the install floor. Existing installs on older OS keep their current version. Not a decision I made lightly, but the per-frame OCR worklet is what makes hands-free scanning actually work, and it needs the native buffer access.