--- license: agpl-3.0 base_model: Qwen/Qwen3-1.7B-Base pipeline_tag: text-generation language: - en tags: - prose - rewriting - style-transfer - creative-writing - deslop - qwen3 library_name: transformers --- # prose-rewriter-1.7b-v2.4 A paragraph-level **prose rewriter**: it takes prose written by a large model and re-renders it to be more human, preserving the semantics it was given. `Qwen/Qwen3-1.7B-Base` with a rank-32 LoRA merged in at strength 1.15. **Longer sentences, less choppy prose.** A rewrite's sentences average 15.5 words, up from 12.1 (input: 18.2). **Fewer changed meanings and broken sentences.** An LLM judge finds a changed meaning in 25.5% of rewrites, down from 29.3%, and an ill-formed sentence in 3.4%, down from 5.8%. **Less negated narration, less truncation.** Negated narration shapes fall from 0.140 to 0.085 per rewrite, and rewrites cut below 0.75x of the input from 12.0% to 8.1%. ## Variants | Path | Format | Use with | |---|---|---| | `/` | safetensors bf16, `qwen3` arch | transformers | | `GGUF/prose-rewriter-1.7b-v2.4-Q8_0.gguf` | GGUF Q8_0, 2.17 GB | llama.cpp / llama-cpp-python | The quant carries the chat template and stops on `<|im_end|>`, and the adapted output head is kept separate from the token embeddings, stored at Q8_0. ## Prompt format ``` <|im_start|>source {paragraph}<|im_end|> <|im_start|>rewrite ``` The chat template in this repo builds exactly that string, byte for byte, from one message: ```python messages = [{"role": "source", "content": paragraph}] tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) ``` It is not a chat model. Every message is rendered as the source paragraph, so a runtime that probes the template with a `user` message still gets a valid prompt. Send one message per call. ## Serving recipe Sample at `temperature=0.85, top_p=0.9`. Temperature 0.85 is the recommended value. ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer repo = "chartreuse-verte/prose-rewriter-1.7b-v2.4" tok = AutoTokenizer.from_pretrained(repo) model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="cuda").eval() def rewrite(paragraph): text = tok.apply_chat_template( [{"role": "source", "content": paragraph}], tokenize=False, add_generation_prompt=True, ) ids = tok(text, return_tensors="pt", add_special_tokens=False).input_ids.to(model.device) out = model.generate(ids, max_new_tokens=512, do_sample=True, temperature=0.85, top_p=0.9) return tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True).strip() ``` Same thing under llama.cpp. The role is `source`, which no chat API models, so build the string yourself; `<|im_end|>` stops it: ```bash llama-cli -m GGUF/prose-rewriter-1.7b-v2.4-Q8_0.gguf -no-cnv -n 512 --temp 0.85 --top-p 0.9 \ -p '<|im_start|>source {paragraph}<|im_end|> <|im_start|>rewrite ' ``` ## Input length The training pool's median input is 37 words and 80% of it is under 70, so serve it on anything from a full sentence up. The practical floor is about **15 words**. Below it the failure mode is padding, cutting and fabrication rather than gibberish. Below 80 bytes, pass the text through unchanged. ## Evaluation Both releases measured **as they ship**, baked at strength 1.15, on 365 held-out paragraphs of LLM-written prose that neither model saw in training, at `temperature=0.9, top_p=0.95`, three swipes each. 1,095 generations an arm. Fidelity, judged by an LLM against the input. A rewrite counts when the judge gives it a probability of 0.5 or more: | | v2.3 | v2.4 | |---|---|---| | adds a fact the input does not support | 12.2% | 10.6% | | changes the meaning of something the input says | 29.3% | **25.5%** | | either | 31.1% | 27.9% | | a sentence that is not well-formed (input: 0.3%) | 5.8% | **3.4%** | Paired over the 365 inputs, changed meanings fall (t = -2.2) and ill-formed sentences fall (t = -2.9). Added facts and either do not separate (t = -1.2, -1.8). Invention of people, on the same paragraphs: | | v2.3 | v2.4 | |---|---|---| | a gendered pronoun the input never licensed | 0.5% | 0.2% | | ... belonging to the other gender | 0.0% | 0.0% | | a proper noun the input does not have | 0.4% | 0.4% | Repetition -- a rewrite that repeats a sentence, a phrase, or a sentence opening: | | input | v2.3 | v2.4 | |---|---|---|---| | any repetition | 2.2% | 5.2% | 4.3% | | ... a whole sentence | | 0.2% | 0.0% | | ... a phrase | | 1.7% | 1.7% | | ... a sentence opening (anaphora) | | 2.5% | 1.5% | Paired over the 365 inputs, none of these separates; any repetition reads t = -1.1. Structural, on the same paragraphs: | | v2.3 | v2.4 | |---|---|---| | words changed | 37.1% | 35.4% | | passed through unchanged | 0.9% | 1.6% | | near-verbatim outputs | 0.0% | 0.3% | | below the training edit floor | 51.7% | 55.9% | | sentence count moved | 81.2% | 61.8% | | sentence-length variety vs input | +0.094 | +0.045 | | words kept from the input | 0.675 | 0.691 | | length preserved | 0.889 | **0.912** | | truncated below 0.75x | 12.0% | **8.1%** | | repeated 3-grams | 0.002 | 0.001 | Paired over the 365 inputs, v2.4 changes fewer words (t = -3.6) and falls below the edit floor more often (t = +2.5). It holds more of the length (t = +6.5) and truncates less (t = -3.6). It moves the sentence count less (t = -9.6) and varies sentence length less (t = -7.3). Unsupported content -- a sentence in the rewrite the input does not entail, scored by NLI with the input alone as the premise: | | v2.3 | v2.4 | |---|---|---| | unsupported sentences, mean per rewrite | 5.8% | 4.9% | | unsupported sentences, all rewrites | 5.6% | 4.6% | | coverage of the input (reverse entailment) | 0.551 | **0.592** | Paired over the 365 inputs, coverage rises (t = +5.6); the unsupported rate does not separate (t = -1.8). The register numbers on the same paragraphs, against the input: | | input | v2.3 | v2.4 | human corpus | |---|---|---|---|---| | words per sentence | 18.2 | 12.1 | **15.5** | 17.8 | | banned constructions /1k | 7.52 | 2.98 | 2.59 | 0.00 | | slop lexicon density | 0.081 | 0.049 | **0.046** | 0.019 | | purple score | 0.498 | 0.294 | 0.305 | -- | | negated narration shapes per rewrite | 0.118 | 0.140 | **0.085** | -- | Paired over all 365 inputs, sentences lengthen (t = +20.8), lexicon density falls (t = -3.7) and negated narration falls (t = -5.1). Purple rises (t = +3.5). Banned constructions do not separate (t = -1.9). Dialogue: the number of spoken lines changes on 0.9% of rewrites against v2.3's 1.6%, and of the rewrites whose input carried quoted speech, 1.0% lost the quotes entirely against 2.0%. Neither separates. ### A note on strength Every number above is measured at the strength the release is baked at, which is not the same as measuring the adapter. LoRA strength is spent at merge time (`W + (B @ A) * 2.30` here), and the metrics move with it: read at the adapter's natural 1.0 this same checkpoint is a materially different artifact. Both releases ship at the same strength, so every difference above is training and not the strength. ## Training **Corrupt forward, train backward.** The human paragraph is the target; an on-policy LLM manufactures the input by slop-ifying it. The target side is human prose: published contemporary literary fiction, r/WritingPrompts ([`Mollymo/Human-to-AI-writing`](https://huggingface.co/datasets/Mollymo/Human-to-AI-writing)), AO3 ([`midwestern-simulation-active/ao3_random_subset`](https://huggingface.co/datasets/midwestern-simulation-active/ao3_random_subset)), a scrape of `bluemoonroleplaying.com` -- the only human writing in the pool already in the deployment's own register -- and a sliver of fanfiction.net ([`atom-in-the-universe/fanfics-10k-10k`](https://huggingface.co/datasets/atom-in-the-universe/fanfics-10k-10k)). The input side is manufactured from those targets, weighted and share-capped so that no single generator's tics dominate: | pool axis | composition | |---|---| | rows | 15,290 over 10,982 distinct target paragraphs | | corruption band | medium 35%, heavy 31%, light 30%, identity/no-op 3%, curated real slop 1% | | `len_mode` | match only | | markup | 15% of rows, identical on both sides | | kind | prose 91%, dialogue 8%, structural no-ops 1% | | target source | published fiction 39%, r/WritingPrompts 29%, AO3 20%, roleplay forum 8%, other 4% | Pairs pass invariant gates before they reach the GPU: POV, tense, who is in the scene, the gender of who is in the scene, the number of spoken lines, grammatical correctness on the target side, no repetition on the target side, content recall stratified by target length, and NLI entailment both ways. Two further screens shape what reaches training -- a floor on how much a pair actually changes, and a floor on the sentence-length variety of the target, applied at a higher threshold for long paragraphs than short ones. **Loss on the target paragraph only.** Everything before `rewrite` is masked. | | | |---|---| | LoRA | r=32, alpha=64, dropout 0.05 | | target modules | q, k, v, o, gate, up, down, **and `lm_head`** | | trainable | 39,792,640 params (2.26%) | | schedule | 2 epochs, lr 1e-4 cosine, batch 4 × accum 8, seq 2048 | | steps | 920 on one RTX 3090 | **Then reinforcement learning against a fidelity judge.** Starting from the SFT adapter, GRPO samples eight rewrites of each roleplay paragraph and moves toward the ones that score best. The reward is an LLM judge's verdict that the rewrite adds no fact and changes no meaning, plus the drop in a purple-prose classifier's score with punctuation ignored, minus penalties for broken grammar the input did not have, sentences shorter on average than the input's, choppy runs of short sentences, added negated narration, added repetition, truncation below 0.75x and near-verbatim output. | | | |---|---| | prompts | LLM-written roleplay paragraphs with a purple score of 0.5 or more | | schedule | 120 steps, 16 paragraphs × 8 rewrites, lr 1e-5, KL 0.04 to the SFT adapter | | sampling | `temperature=0.9, top_p=0.95`, at strength 1.15 | ## The merge Merged at strength 1.15. Rank 32 with alpha 64 is a LoRA scaling of 2.0, so the effective scaling is **2.30**: `W + (B @ A) * 2.30`. Merged in float32, stored bfloat16. `lm_head` is adapted, and `Qwen3-1.7B-Base` ties `lm_head.weight` to `embed_tokens.weight`. **This checkpoint is untied**: the merged output head is stored separately and the input embeddings are bit-identical to the base model's, which is what training assumed. `config.json` says `tie_word_embeddings: false` and it means it. Do not re-tie it, and if you convert to another format, check that the head survived. ## Limitations - **Not an instruct model.** It has one job and one prompt. There is nothing to ask it. - **Works on fictional prose only.** May not work on technical documentation. - **One paragraph per call.** Longer input degrades; split it. - **Still invents occasionally.** The judge finds an added fact in 10.6% of rewrites and a changed meaning in 25.5%. Check the output when the facts matter. - **Truncates.** It drops below 0.75x the input's length on 8.1% of paragraphs. Usually that is the slop going; check the output when length matters. - **Still repeats itself.** 4.3% of rewrites repeat a sentence, a phrase or a sentence opening the input did not. Check the output for loops. - **Will not pass AI detectors.** Pangram and such will still know because this model preserves word choices and certain sentence structures. - **English only**, narrative register (third and first person fiction, dialogue with quoted speech). - **Short input pads, cuts and invents.** The floor is about 15 words. See [Input length](#input-length). - **Repeats a noun sooner than an LLM would.** Human prose reuses a plain noun where generated prose uses a synonym, and this model has learned that habit. ## License The weights in this repository are released under the **GNU Affero General Public License, version 3**. The full text is in `LICENSE`. This is a derivative of [`Qwen/Qwen3-1.7B-Base`](https://huggingface.co/Qwen/Qwen3-1.7B-Base), which is licensed under **Apache License 2.0**. That license is preserved and its terms continue to apply to the base weights this model was built from; the AGPL covers the combined work as distributed here. Apache-2.0 is one-way compatible with AGPLv3, which is what makes this combination possible. If you run a modified version of this model as a network service, AGPL section 13 requires you to offer the corresponding source of your modifications to its users.