All posts
Engineering

How we tune the line editor to keep your voice

2026-04-22 · 4 min read

An AI that critiques your prose is easy to build. An AI that critiques your prose without ever rewriting it is much harder — and it is the only kind we were willing to ship. This is how we tuned WriteLoom's line editor to live on that edge: flagging weak phrasing without producing the replacement sentence, naming a rhythm problem without solving it for you.

Why the line is worth defending

The danger with AI editing isn't bad suggestions. It's good ones. When a tool hands you a cleaner version of your sentence, the path of least resistance is to accept it, and accepting is frictionless. Do that a few hundred times across a manuscript and the book quietly drifts away from your voice and toward the model's — smoother, more competent, and less like anyone. The homogenization is gradual and reasonable-seeming at every step, which is exactly what makes it dangerous.

So we set a hard rule before we wrote a line of prompt: the line editor names problems and never writes the fix. It will tell you a sentence leans on filter words, that a paragraph has three sentences of the same length and cadence, that you have used "suddenly" twice on one page. It will not give you the rewrite. The verdict, and the fix, stay with you.

The prompt is a constitution, not a suggestion

The first thing we learned is that you cannot politely ask a capable model to withhold a rewrite. It wants to help, and helping, to a model, means producing the better sentence. Tell it "don't rewrite" in prose and it will still slip you a "you could try…" half the time.

So we designed the rewrite out rather than asking for it. The output schema the editor returns has no field for a replacement sentence. It has a field for the issue, a field for why it matters, and a span pointing at the offending text. There is simply nowhere for a rewrite to go. A model is far more reliable at filling a structure than at obeying a prohibition, and a structure with no "suggestion" slot cannot suggest.

The comparison harness

Prompts are not a thing you write once and trust. We built a small harness to keep ourselves honest. It holds a set of passages with known, labeled issues — a filter-word problem here, a rhythm problem there, a deliberately clean control passage that should draw no flags at all. Every time we change the prompt, the harness runs the candidate against all of them and reports two numbers: did the editor stay critique-only (zero rewrites leaked), and did it catch the planted issues without inventing new ones.

That second number matters more than it sounds. An editor that flags everything is as useless as one that flags nothing, because a writer learns to ignore a tool that cries wolf. The control passages are there specifically to punish false positives.

Measuring voice, not just correctness

The harder eval is voice preservation, and it is mostly about restraint. Our voice set is a collection of passages written in deliberately distinct registers — clipped and plain, ornate and recursive, dialect-heavy, fragmentary on purpose. The test is not whether the editor "improves" them. It is whether it leaves the intentional choices alone.

A fragment written for effect should not be flagged as an incomplete sentence. Dialect should not be flagged as misspelling. A run of short sentences building tension should not be flagged as monotonous. We score the editor on how often it tries to "correct" a deliberate stylistic choice, and we treat a high rate there as a failure even if every flag is technically defensible. Friction is often where voice lives, and an editor that sands off all the friction is worse than no editor at all.

What we learned

A few things, in order of how much they surprised us. First, capability cuts against you here: the better the underlying model, the more it wants to rewrite, and the more deliberately you have to remove the option. Second, examples beat instructions every time — a handful of critique-only examples in the prompt did more than paragraphs of "do not rewrite." Third, the writers who tested early didn't want a smarter editor; they wanted a quieter one that they could trust not to touch the lines they meant.

The discipline ships to the writer, too

The tool enforces the rule on its side: it cannot hand you a sentence to paste. But the other half of keeping your voice is a discipline only the writer can hold — reading flags as questions rather than orders, and writing every fix yourself. We wrote a separate guide on that practice: how to use AI to edit a novel without losing your voice. The short version is the same as our design rule. The AI helps you see your prose more clearly. You keep the pen.

Chapter ·Keep reading

Find more field notes on the blog.