We're writing a novel with AI, in public, and this weekend we ran the draft
through every AI-scent phrase check we could find: a testament to,
couldn't help but, in that moment, all the phrases
people grep for when they suspect a machine wrote something. Our score across
85,500 words was zero. Perfectly clean.
The draft still read like a machine wrote it. Here's where the scent was actually hiding, with the real numbers and the exact checks we now run, in case you're doing anything like this yourself.
Start with a human baseline
Feelings about prose don't converge, so we stopped arguing and measured. We took the full text of a bestselling novel in our genre (204,500 words of human-written prose; we're not naming it) and computed the same statistics for it and for our draft: phrase rates and sentence-length distribution, normalized per 10,000 words. Anything our draft does far more often than the human is a suspect. So is anything the human does freely that our draft never dares to do.
The word lists came back clean. That's the trap.
Every classic tell measured zero in our draft, and the regression check that keeps it that way is embarrassingly simple:
grep -cin "a sense of\|couldn't help but\|in that moment\|testament to\|tapestry\|palpable\|washed over\|eyes widened\|found herself\|found himself" chapters/*.md # must stay 0The punchline: the human benchmark uses several of these freely.
couldn't help but shows up in it 26 times! We'd been trained so
hard to dodge the famous phrases that our draft was cleaner than the human
was. Nobody is that tidy by accident, and it told us something better than a
pass: the scent had moved somewhere the word lists can't see.
Hiding spot #1: the correction reflex
Sentences shaped like "X, not Y" (state a thing, then correct a thing you weren't thinking) ran at 10.3 per 10k words in our draft. The human baseline sits at 4.0. Some of that is genuinely our narrator's voice, so we didn't ban it; we gave it a budget of two per chapter and a test for which ones live. Try the test yourself:
If the sentence survives without the clause, the clause goes. If it dies, the contrast was carrying real meaning and it stays. That one rule cleared most of the excess on its own.
Hiding spot #2: every sentence performing
The second finding was rhythm, and it surprised us more:
| Sentence length | Our draft | Human benchmark |
|---|---|---|
| 6 words or fewer | 39.6% | 15.5% |
| the middle (7–29 words) | 45.9% | 81.0% |
| 30 words or more | 14.5% | 3.5% |
Our prose lived at the extremes. Punchy fragment, sprawling lyric, punchy fragment. What it almost never did was walk: the plain 10–20-word declarative sentence that carries most human prose is where the benchmark spends four-fifths of its time. A draft where every sentence performs reads composed rather than written, and that turns out to be a scent no phrase list will ever catch. The rule we adopted: connective tissue gets ordinary sentences, and fragments have to earn their place at real intensity peaks.
Making it stick
Findings became budgets in the style guide, and budgets became a QA loop that every batch of chapters goes through:
The verifier is a separate agent that never sees the rewrite happen, so the writer can't grade its own homework. Five passes later, chapters 2 through 31 sit in the target register and the book-wide em-dash count is zero. One standing rule survived the whole exercise: AI detectors are never the quality gate. Measured craft budgets are. And when we want a rhythm check no counter can give us, we read the chapter aloud — over-polished prose announces itself the moment you hear it.
Book 1 sits at 31 of 62 chapters, and the serial is free to read at trystarbound.com — the first twenty-plus chapters are live, so you can judge for yourself whether the machinery produces a book that sounds like a person. In case you're wondering: this blog now runs the same kind of gate, calibrated against a human blogger we admire, and this post had to pass it before you saw it.