For months the Binderdex test suite was green because tests were quitting before they ran. Five web database integration tests and every scheduled-alert sender test skipped unless someone set an env flag. Nothing in CI set it. Skip counts as pass, the suite reports green, everybody moves on.
So last night we changed the default: the flag now gets set, the tests actually run, and they immediately caught three real Postgres insert bugs. Binder moments, new-set alerts and product notices would all have failed on every run the moment the features went live. The suite stayed green the whole time they were broken.
Why skip is worse than fail
A failing test yells at you. A skipping one smiles and waves. These tests needed a real database, and someone once made "run them" opt-in, which meant the default behavior was opt-out evidence. When you ship solo with AI agents doing most of the typing, a green checkmark is the cheapest lie in the repo.
The fix is a couple of CI lines: give the test command the database it needs and stop gating on a flag. Now the skip needs a reason and the run has to earn its green.
The Postgres conflict-target gotcha
Three of the bugs were the same bug. The inserts used ON CONFLICT (dedupe_key) to stay idempotent, but dedupe_key lives in a partial unique index that only covers rows where the key is present. Postgres requires the conflict target to carry the same predicate:
-- what we wrote, and Postgres rejects on every insert:
ON CONFLICT (dedupe_key) DO UPDATE ...
-- what a partial index needs:
ON CONFLICT (dedupe_key) WHERE dedupe_key IS NOT NULL DO UPDATE ...Omit the predicate and Postgres doesn't explain the mismatch. It rejects the insert outright. We'd already fixed the weekly digest for this exact thing once; binder moments, new-set alerts and product notices were quietly waiting to fail the same way. One partial index, four victims, zero failures logged.
The other two lessons
Bug four was a two-statement race in the big-mover sender. It wrote inbox rows first and enqueued the push/email events a beat later. Only newly inserted rows may send, so a timeout between the writes burned every member's dedupe key and the retry sent nothing. Alerts went dark for a day and the only visible symptom was silence. Both writes now happen in one transaction.
Separately, chunked inserts: Postgres accepts at most 65,535 bind parameters per statement. A busy trading day blows past that on a wide table and the whole batch fails. Chunk at 1,000 rows and the limit stops being a bomb with a fuse.
Do the same audit today
Two greps and five minutes: list skipped tests with their counts, and search the repo for conflict targets on partial indexes. If a test can skip, someone will eventually make skipping the default, because it's the path of least resistance. The count that should worry you is how many tests pass without running. Ours was over a hundred until last night.