Qwen3.5-4B-d0-vtok-lora-seeds

Per-seed LoRA replicates for a documents-per-function suite: 50 synthetic constant functions, each answer a single added vocabulary token rather than a digit string.

Each model is trained on a corpus containing a fixed number of documents per function, with four runs per document count that are identical but for the order the documents were shuffled. They exist so the suite's results carry error bars over training order rather than resting on one trajectory.

Loading a run

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B")
model = PeftModel.from_pretrained(base, "Lamsheeper/Qwen3.5-4B-d0-vtok-lora-seeds", subfolder="10d_sd1001")

The base model needs the suite's added function tokens before an adapter will load against it; embed_tokens and lm_head are trained in full and ship inside each adapter.

Runs

Run Docs/fn Seed Accuracy Retention PPL Selected
1d_sd1001 1 sd1001 96.6% 41.86
1d_sd1002 1 sd1002 89.6% 46.39
1d_sd1003 1 sd1003 97.2% 32.67 yes
1d_sd1004 1 sd1004 93.5% 40.06
2d_sd1001 2 sd1001 100.0% 37.61
2d_sd1002 2 sd1002 100.0% 31.22 yes
2d_sd1003 2 sd1003 99.7% 27.08
2d_sd1004 2 sd1004 99.8% 28.43
3d_sd1001 3 sd1001 100.0% 27.15
3d_sd1002 3 sd1002 100.0% 23.25
3d_sd1003 3 sd1003 100.0% 31.33
3d_sd1004 3 sd1004 100.0% 20.13 yes
4d_sd1001 4 sd1001 100.0% 28.36
4d_sd1002 4 sd1002 100.0% 22.15 yes
4d_sd1003 4 sd1003 100.0% 24.28
4d_sd1004 4 sd1004 100.0% 25.26
5d_sd1001 5 sd1001 100.0% 21.04
5d_sd1002 5 sd1002 99.3% 23.99
5d_sd1003 5 sd1003 100.0% 20.52
5d_sd1004 5 sd1004 100.0% 19.59 yes
6d_sd1001 6 sd1001 100.0% 21.70
6d_sd1002 6 sd1002 100.0% 18.97 yes
6d_sd1003 6 sd1003 100.0% 23.49
6d_sd1004 6 sd1004 100.0% 25.39
7d_sd1001 7 sd1001 100.0% 20.58 yes
7d_sd1002 7 sd1002 100.0% 21.80
7d_sd1003 7 sd1003 100.0% 22.51
7d_sd1004 7 sd1004 100.0% 21.08
8d_sd1001 8 sd1001 100.0% 20.17 yes
8d_sd1002 8 sd1002 100.0% 24.46
8d_sd1003 8 sd1003 100.0% 24.51
8d_sd1004 8 sd1004 100.0% 22.40
9d_sd1001 9 sd1001 100.0% 23.50
9d_sd1002 9 sd1002 100.0% 25.48
9d_sd1003 9 sd1003 100.0% 24.55
9d_sd1004 9 sd1004 100.0% 17.61 yes
10d_sd1001 10 sd1001 100.0% 20.07
10d_sd1002 10 sd1002 100.0% 25.72
10d_sd1003 10 sd1003 100.0% 19.97 yes
10d_sd1004 10 sd1004 100.0% 24.67
20d_sd1001 20 sd1001 100.0% 22.98
20d_sd1002 20 sd1002 100.0% 22.66
20d_sd1003 20 sd1003 100.0% 25.68
20d_sd1004 20 sd1004 100.0% 20.93 yes
50d_sd1001 50 sd1001 100.0% 13.72
50d_sd1002 50 sd1002 100.0% 13.69
50d_sd1003 50 sd1003 100.0% 13.37 yes
50d_sd1004 50 sd1004 100.0% 13.57
Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Lamsheeper/Qwen3.5-4B-d0-vtok-lora-seeds

Finetuned
Qwen/Qwen3.5-4B
Adapter
(1,222)
this model