prose-rewriter-4b-v2.6

A paragraph-level prose rewriter: it takes prose written by a large model and re-renders it to be more human, preserving the semantics it was given.

Qwen/Qwen3-4B-Base with a rank-32 LoRA merged in at strength 1.20.

Invents less. On LLM-written prose, an LLM judge finds an added fact in 3.7% of rewrites, down from 6.8%, and a changed meaning in 12.0%, down from 16.2%.

Cuts more slop. Banned constructions fall from 3.46 to 2.41 per 1,000 words, the slop-lexicon density from 0.043 to 0.038, the purple score from 0.628 to 0.592, and negated narration shapes from 0.109 to 0.067 per rewrite. On LLM-written roleplay, banned constructions fall from 3.46 to 2.52 per 1,000 words and the purple score from 0.358 to 0.331.

Keeps more manner adverbs in human prose: 75.4% of them, up from 65.9%.

Variants

Path	Format	Use with
/	safetensors bf16, qwen3 arch	transformers
GGUF/prose-rewriter-4b-v2.6-Q8_0.gguf	GGUF Q8_0, 4.69 GB	llama.cpp / llama-cpp-python
GGUF/prose-rewriter-4b-v2.6-Q4_K_M.gguf	GGUF Q4_K_M, 2.72 GB	llama.cpp / llama-cpp-python

The quants carry the chat template and stop on <|im_end|>, and the adapted output head is kept separate from the token embeddings (Q8_0 in the Q8_0 file, Q6_K in the Q4_K_M file).

Prompt format

<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>rewrite

The chat template in this repo builds exactly that string, byte for byte, from one message:

messages = [{"role": "source", "content": paragraph}]
tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

It is not a chat model. Every message is rendered as the source paragraph, so a runtime that probes the template with a user message still gets a valid prompt. Send one message per call.

Serving recipe

Sample at temperature=0.85, top_p=0.9. Temperature 0.85 is the recommended value.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "chartreuse-verte/prose-rewriter-4b-v2.6"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="cuda").eval()

def rewrite(paragraph):
    text = tok.apply_chat_template(
        [{"role": "source", "content": paragraph}],
        tokenize=False, add_generation_prompt=True,
    )
    ids = tok(text, return_tensors="pt", add_special_tokens=False).input_ids.to(model.device)
    out = model.generate(ids, max_new_tokens=512, do_sample=True, temperature=0.85, top_p=0.9)
    return tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True).strip()

Same thing under llama.cpp. The role is source, which no chat API models, so build the string yourself; <|im_end|> stops it:

llama-cli -m GGUF/prose-rewriter-4b-v2.6-Q8_0.gguf -no-cnv -n 512 --temp 0.85 --top-p 0.9 \
  -p '<|im_start|>source
{paragraph}<|im_end|>
<|im_start|>rewrite
'

Input length

The training pool's median input is 37 words and 81% of it is under 70, so serve it on anything from a full sentence up.

The practical floor is about 15 words. Below it the failure mode is padding, cutting and fabrication rather than gibberish. Below 80 bytes, pass the text through unchanged.

Evaluation

Both releases measured as they ship, baked at strength 1.20, at temperature=0.9, top_p=0.95, three swipes per input.

LLM-written prose

365 held-out paragraphs of LLM-written prose. 1,095 generations an arm.

The register numbers, against the input. Both releases are rescored here with the current purple classifier and banned list:

	input	v2.5	v2.6
banned constructions /1k	12.03	3.46	2.41
rewrites with a banned construction		20.3%	15.1%
slop lexicon density	0.081	0.043	0.038
purple score	0.821	0.628	0.592
words per sentence	18.2	15.2	16.3
negated narration shapes per rewrite	0.118	0.109	0.067

Paired over the 365 inputs, banned constructions, lexicon density, purple and negated narration all fall (t = -3.8, -4.3, -4.0, -4.0). Sentences are longer (t = +7.7).

Fidelity, judged by an LLM against the input. A rewrite counts when the judge gives it a probability of 0.5 or more:

	v2.5	v2.6
adds a fact the input does not support	6.8%	3.7%
changes the meaning of something the input says	16.2%	12.0%
either	18.5%	13.3%
a sentence that is not well-formed (input: 0.3%)	3.0%	1.8%

Paired over the 365 inputs, added facts, changed meanings and either fall (t = -3.1, -2.8, -3.3). Well-formedness does not separate (t = -1.8).

Invention of people, on the same paragraphs:

	v2.5	v2.6
a gendered pronoun the input never licensed	0.5%	0.5%
... belonging to the other gender	0.0%	0.3%
a proper noun the input does not have	0.5%	0.1%

None of these separates.

Repetition -- a rewrite that repeats a sentence, a phrase, or a sentence opening:

	input	v2.5	v2.6
any repetition	2.2%	3.6%	2.9%
... a whole sentence		0.0%	0.0%
... a phrase		1.8%	0.9%
... a sentence opening (anaphora)		1.1%	1.3%

None of these separates.

Structural, on the same paragraphs:

	v2.5	v2.6
words changed	35.7%	35.6%
passed through unchanged	0.5%	1.9%
near-verbatim outputs	0.0%	0.1%
below the training edit floor	53.8%	56.4%
sentence count moved	61.8%	40.3%
sentence-length variety vs input	+0.041	+0.007
words kept from the input	0.690	0.675
length preserved	0.935	0.872
truncated below 0.75x	3.3%	15.5%
two or more fewer sentences than the input	2.6%	7.6%
repeated 3-grams	0.001	0.001

Paired over the 365 inputs, the two change the same share of words (t = -0.3). v2.6 passes more inputs through unchanged (t = +3.3), moves the sentence count less often (t = -9.4), keeps less of the input's wording and length (t = -3.8, -15.7), truncates more often (t = +9.0), drops two or more sentences more often (t = +5.5) and varies sentence length less (t = -5.4).

Unsupported content -- a sentence in the rewrite the input does not entail, scored by NLI with the input alone as the premise:

	v2.5	v2.6
unsupported sentences, mean per rewrite	3.8%	2.6%
unsupported sentences, all rewrites	3.7%	2.6%
coverage of the input (reverse entailment)	0.641	0.615

The unsupported rate falls (t = -3.3) and coverage falls (t = -3.0).

Dialogue: the number of spoken lines changes on no rewrite against v2.5's 0.5%, and no rewrite whose input carried quoted speech lost the quotes, in either release.

LLM-written roleplay

400 held-out paragraphs of LLM-written roleplay. 1,200 generations an arm.

	input	v2.5	v2.6
banned constructions /1k	11.25	3.46	2.52
slop lexicon density	0.080	0.045	0.042
purple score	0.580	0.358	0.331
words per sentence	13.4	13.7	13.7
length preserved		0.972	0.917

Paired over the 400 inputs, banned constructions and purple fall (t = -3.1, -2.9); lexicon density does not separate (t = -2.0). Length falls (t = -14.5).

Human prose

300 held-out paragraphs of human-written fiction that neither model saw in training, each carrying at least one manner adverb on an action. 900 generations an arm.

	input	v2.5	v2.6
manner adverbs kept		65.9%	75.4%
manner adverbs per 100 words	1.99	1.50	1.65
judge: adds a fact or changes a meaning		9.4%	9.0%
unsupported sentences (NLI)		1.8%	2.2%
a gendered pronoun the input never licensed		0.3%	1.1%
words per sentence	17.4	15.8	16.8
two or more fewer sentences than the input		1.2%	3.7%

Retention rises 9.5 points (95% CI +6.2 to +12.8). The judge and NLI rows do not separate (t = -0.3, +1.0). Unlicensed pronouns rise (t = +2.4). Sentences are longer and closer to the input's (t = +5.8), and rows with two or more fewer sentences rise (t = +3.5).

A note on strength

Every number above is measured at the strength the release is baked at, which is not the same as measuring the adapter. LoRA strength is spent at merge time (W + (B @ A) * 2.40 here), and the metrics move with it: read at the adapter's natural 1.0 this same checkpoint is a materially different artifact.

Both releases ship at the same strength, so every difference above is training and not the strength.

Training

Corrupt forward, train backward. The human paragraph is the target; an on-policy LLM manufactures the input by slop-ifying it.

The target side is human prose: published contemporary literary fiction, r/WritingPrompts (Mollymo/Human-to-AI-writing), AO3 (midwestern-simulation-active/ao3_random_subset), a scrape of bluemoonroleplaying.com -- the only human writing in the pool already in the deployment's own register -- and a sliver of fanfiction.net (atom-in-the-universe/fanfics-10k-10k). Alongside them is a set of hand-edited rewrites of LLM-written roleplay paragraphs, weighted 2x.

The input side is manufactured from those targets, weighted and share-capped so that no single generator's tics dominate:

pool axis	composition
rows	14,872 over 10,933 distinct target paragraphs
corruption band	medium 32%, light 29%, heavy 28%, hand-edited and curated 8%, identity/no-op 3%
len_mode	match only
markup	11% of rows, identical on both sides
kind	prose 92%, dialogue 7%, structural no-ops 1%
target source	published fiction 38%, r/WritingPrompts 26%, AO3 18%, roleplay forum 7%, hand-edited roleplay 7%, other 4%

Pairs pass invariant gates before they reach the GPU: POV, tense, who is in the scene, the gender of who is in the scene, the number of spoken lines, grammatical correctness on the target side, no repetition on the target side, no closing sentence or trailing clause of the input that the target drops without a counterpart, content recall stratified by target length, and NLI entailment both ways. Two further screens shape what reaches training -- a floor on how much a pair actually changes, and a floor on the sentence-length variety of the target, applied at a higher threshold for long paragraphs than short ones.

Loss on the target paragraph only. Everything before rewrite is masked.

LoRA	r=32, alpha=64, dropout 0.05
target modules	q, k, v, o, gate, up, down, and lm_head
trainable	71,004,160 params (1.73%)
schedule	2 epochs, lr 1e-4 cosine, batch 8 × accum 4, seq 2048
steps	892 on one RTX 3090

Then reinforcement learning against a fidelity judge. Starting from the SFT adapter, GRPO samples eight rewrites of each roleplay paragraph and moves toward the ones that score best. The reward is an LLM judge's verdict that the rewrite adds no fact and changes no meaning, plus the drop in a paragraph-level purple-prose classifier's score, read sentence by sentence and over short windows. Penalties cover broken grammar the input did not have, a changed sentence count or sentence shape, stacked modifiers, flattened prose, stock phrasing and banned constructions left in place, added negated narration, added repetition, a dropped ending, truncation and near-verbatim output. A small bonus for subordinate clauses is capped at the rate human prose uses each one.

prompts	LLM-written roleplay paragraphs with a purple score of 0.5 or more
schedule	240 steps, 16 paragraphs × 8 rewrites, lr 1e-5, KL 0.04 to the SFT adapter
sampling	temperature=0.9, top_p=0.95, at strength 1.20

The merge

Merged at strength 1.20. Rank 32 with alpha 64 is a LoRA scaling of 2.0, so the effective scaling is 2.40: W + (B @ A) * 2.40. Merged in float32, stored bfloat16.

lm_head is adapted, and Qwen3-4B-Base ties lm_head.weight to embed_tokens.weight. This checkpoint is untied: the merged output head is stored separately and the input embeddings are bit-identical to the base model's, which is what training assumed. config.json says tie_word_embeddings: false and it means it. Do not re-tie it, and if you convert to another format, check that the head survived.

Limitations

• Not an instruct model. It has one job and one prompt. There is nothing to ask it.
• Works on fictional prose only. May not work on technical documentation.
• One paragraph per call. Longer input degrades; split it.
• Still invents occasionally. The judge finds an added fact in 3.7% of rewrites and a changed meaning in 12.0%. Check the output when the facts matter.
• Shorter output than v2.5. It drops below 0.75x the input's length on 15.5% of paragraphs against v2.5's 3.3%, and ends with two or more fewer sentences on 7.6%. Check the output when length matters.
• Sometimes genders a character the input leaves ungendered: on 1.1% of human-prose rewrites, against v2.5's 0.3%.
• Still repeats itself. 2.9% of rewrites repeat a sentence, a phrase or a sentence opening. Check the output for loops.
• Will not pass AI detectors. Pangram and such will still know because this model preserves word choices and certain sentence structures.
• English only, narrative register (third and first person fiction, dialogue with quoted speech).
• Short input pads, cuts and invents. The floor is about 15 words. See Input length.
• Repeats a noun sooner than an LLM would. Human prose reuses a plain noun where generated prose uses a synonym, and this model has learned that habit.

License

The weights in this repository are released under the GNU Affero General Public License, version 3. The full text is in LICENSE.

This is a derivative of Qwen/Qwen3-4B-Base, which is licensed under Apache License 2.0. That license is preserved and its terms continue to apply to the base weights this model was built from; the AGPL covers the combined work as distributed here. Apache-2.0 is one-way compatible with AGPLv3, which is what makes this combination possible.

If you run a modified version of this model as a network service, AGPL section 13 requires you to offer the corresponding source of your modifications to its users.