Cache the promise, not the value: race-proof deduplication without locks

The merge that was slow because it was correct

A data merge was taking six minutes per season. The bottleneck wasn't CPU or parsing. It was thousands of sequential database round trips, each one burning 30 to 80 milliseconds of network time doing absolutely nothing. The fix was to run them concurrently, and the fix created duplicate rows.

Here's why going parallel broke things. When two concurrent tasks both need to resolve-or-create the same entity, and neither has seen it yet, they both fire a lookup, both get "not found," and both insert. You end up with two canonical rows for one entity. The loser of the slug race strands an orphan in your foreign keys, and you don't find out until a report double-counts something months later.

So the merge ran sequentially, one round trip at a time, because correctness beats speed. I spent an evening looking at this tradeoff before I realized the tool I needed was already in the language.

Why the standard fixes feel heavy

The textbook answer to concurrent deduplication is a mutex. Before creating an entity, acquire the lock, check existence, create if absent, release. This works, but every concurrent path now serializes on the lock. You've rebuilt the sequential merge with a more complex dependency graph.

A job queue is the other common fix: funnel every create through a single worker. Also correct, also heavy. You now operate a queue, keep a worker alive, and handle the failure mode where the worker dies and everything backs up behind it.

Both solutions assume the problem is hard enough to deserve infrastructure. For most single-process backend code, it isn't.

A Promise is a one-shot lock

The entity-resolution cache was a Map from external ID to canonical ID. A caller needing entity X would check the cache, miss, look it up in the database, create if absent, and store the ID. The change was one type signature:

// Before: cache holds resolved values
teamIds: Map<string, number>

// After: cache holds in-flight promises
teamIds: Map<string, Promise<number>>

When the first caller gets a cache miss for entity X, it kicks off the async resolution and stores the Promise itself in the cache. Not the result. When a second caller needs the same X milliseconds later, it finds a Promise already sitting in the Map and awaits it. Both callers receive the same ID. One insert happens. No race, no orphan, no lock.

The Promise is doing the work of a mutex. It's a single-shot synchronization primitive: whoever creates it holds the resolution, everyone who awaits gets the same answer, and it settles exactly once. JavaScript handed us this for free, and most of us use it only to await API responses.

Three properties that make it safe

1. Idempotency. The resolve-or-create path must tolerate being called twice. A unique constraint on the external ID with onConflictDoNothing means a duplicate insert is a silent no-op. The second caller's follow-up lookup returns the first caller's row.
2. Failure cleanup. If resolution throws, delete the cache entry. A rejected Promise left in the cache poisons every subsequent caller for that key. A .catch that removes the entry on failure gives the next caller a fresh attempt.
3. Single settlement. A Promise settles once, period. There's no window where two callers both see "unsettled" and both proceed, because the Promise itself is the arbiter. The second caller doesn't see "not started." It sees "in progress."

The catch is that this is all in-process. It handles concurrency within a single Node event loop, which is the model most backend JavaScript actually runs under. If you need cross-process deduplication, you do need a distributed lock or a database advisory lock. But I'd bet most of the duplicate-row problems you've hit came from concurrent calls in the same process, and those die to this pattern.

What happened

The merge dropped from six minutes to about 45 seconds per season, an 8x speedup, with zero duplicate rows in the audit that followed. The cache didn't go away. Once a Promise resolves, it holds its value, so subsequent lookups are instant. The cache stopped being the point and became a bonus.

Where it generalizes

Any resolve-or-create path keyed by an identifier can use this. User sign-in (find or create by email), webhook idempotency (find or create by event ID), tag resolution, slug assignment. The constraints are narrow but common: the operation must be idempotent, and the concurrency must be within one process. Inside that box, you get race-proof deduplication from a type change and a cache write.

I think this pattern is underknown because it sits between two mental models. Engineers who came up through Java or Go reach for mutexes because that's the vocabulary they carry. JavaScript engineers often don't think of Promises as synchronization primitives, just as async plumbing. The insight lives in the gap.

The broader lesson, for me, was about defaults. When a path is slow, the reflex is to add infrastructure: a queue, a worker, a cache layer, a lock service. Each one is something you now operate at 2 a.m. when it breaks. Sometimes the right move is to look at what the runtime already gave you and change a type signature. Six minutes to 45 seconds, four lines of code, and the lock was inside the Promise the whole time.