Hugging Face's logo

ajgazin
/
Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF

Image-Text-to-Text
GGUF
llama.cpp
qwen3_8
abliterated
uncensored
unsloth
imatrix
dynamic-quant
mtp
Model card Files
xet
 Community
2
ajgazin commited on
Commit
6215b62
·
verified ·
1 Parent(s): 9d91046

Upload folder using huggingface_hub

Browse files
Files changed (4) hide show
  1. .gitattributes +1 -0
  2. README.md +9 -6
  3. Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-IQ3_S.gguf +3 -0
  4. tensor_types.tsv +507 -507
.gitattributes CHANGED
@@ -45,3 +45,4 @@ Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q5_K_S.gguf filter=lfs diff=lfs merg
45
  Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
46
  Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
  Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
 
 
45
  Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
46
  Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
  Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
48
+ Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -39,6 +39,7 @@ SGLang: [NVFP4](https://huggingface.co/ajgazin/Swift-Qwen3.8-27B-Uncensored-NVFP
39
  | :-- | :-- | :-- |
40
  | UD-Q2_K_XL | 9.2 GiB | Fits a 12 GB GPU. |
41
  | UD-IQ3_XXS | 10.2 GiB | Also fits a 12 GB GPU. Prefer it to UD-Q2_K_XL if it fits: clearly closer to the BF16. |
 
42
  | UD-Q3_K_XL | 12.2 GiB | Fits a 16 GB GPU. |
43
  | UD-IQ4_XS | 13.3 GiB | Smallest 4-bit. Fits a 16 GB GPU, with less context room than UD-Q3_K_XL. |
44
  | UD-Q4_K_S | 14.3 GiB | Smaller 4-bit. Fits a 20 GB GPU, or a 16 GB GPU with short context. |
@@ -57,7 +58,7 @@ Other files:
57
  | File | Size | Use |
58
  | :-- | :-- | :-- |
59
  | `mmproj-BF16.gguf` | 0.9 GiB | Vision projector, needed only for image and video input. |
60
- | `tensor_types.tsv` | 39 KB | Per-tensor quantization types for each size (see [Quantization](#quantization)). |
61
  | `kld_results.csv` | 5 KB | Every statistic behind the [Quality](#quality) table. |
62
 
63
  ## Quality
@@ -71,6 +72,7 @@ Each quant compared with the BF16 it was made from, token by token: wikitext-2 t
71
  | :-- | --: | --: | --: | --: |
72
  | UD-Q2_K_XL | 9.2 | 0.1796 | 1.99 | 85.2 % |
73
  | UD-IQ3_XXS | 10.2 | 0.1255 | 1.20 | 87.9 % |
 
74
  | UD-Q3_K_XL | 12.2 | 0.0726 | 0.58 | 91.8 % |
75
  | UD-IQ4_XS | 13.3 | — | — | — |
76
  | UD-Q4_K_S | 14.3 | 0.0521 | 0.45 | 94.1 % |
@@ -80,8 +82,8 @@ Each quant compared with the BF16 it was made from, token by token: wikitext-2 t
80
  | UD-Q6_K_XL | 23.6 | 0.0116 | 0.028 | 98.2 % |
81
  | UD-Q8_K_XL | 29.3 | 0.0060 | 0.012 | 98.8 % |
82
 
83
- - **UD-IQ4_XS** was added after this measurement run, so it has no numbers yet and is absent
84
- from the chart for the same reason. They will be filled in with the next batch.
85
  - **Mean KLD** is the average divergence of the quant's next-token distribution from the BF16's.
86
  **99 % KLD** is the level only the worst 1 % of tokens exceed. **Same top-1** is how often both
87
  pick the same most likely token.
@@ -151,6 +153,7 @@ thinking skipped). Not re-measured on the quants.
151
  | :-- | :-- |
152
  | UD-Q2_K_XL | 112×IQ3_XXS, 98×Q8_0, 67×IQ2_S, 57×IQ3_S, 48×IQ2_XXS, 34×IQ2_XS, 21×Q4_K, 20×IQ1_S, 19×IQ4_XS, 16×Q2_K, 6×Q6_K, 5×Q3_K, 2×Q5_K, 1×IQ1_M |
153
  | UD-IQ3_XXS | 120×IQ3_XXS, 106×IQ3_S, 98×Q8_0, 45×IQ4_XS, 35×IQ2_S, 26×Q4_K, 24×IQ2_XXS, 12×IQ2_XS, 10×Q2_K, 8×IQ1_S, 7×Q5_K, 6×Q3_K, 6×Q6_K, 3×IQ1_M |
 
154
  | UD-Q3_K_XL | 156×IQ4_XS, 111×IQ3_S, 98×Q8_0, 36×Q4_K, 34×IQ3_XXS, 26×Q5_K, 15×IQ2_S, 12×Q3_K, 7×Q6_K, 4×IQ2_XS, 3×Q2_K, 2×IQ4_NL, 2×IQ2_XXS |
155
  | UD-IQ4_XS | 211×IQ4_XS, 98×Q8_0, 55×Q4_K, 51×Q5_K, 46×IQ3_S, 17×IQ3_XXS, 9×Q3_K, 8×Q6_K, 7×IQ2_S, 2×IQ2_XS, 1×IQ4_NL, 1×Q2_K |
156
  | UD-Q4_K_S | 172×IQ4_XS, 99×Q8_0, 95×Q4_K, 80×Q5_K, 18×Q6_K, 15×IQ3_S, 13×Q3_K, 7×IQ4_NL, 5×IQ3_XXS, 1×IQ2_S, 1×IQ2_XS |
@@ -195,9 +198,9 @@ awk -F'\t' 'NR==1 { for (i=2; i<=NF; i++) if ($i=="UD-Q5_K_M") c=i; next }
195
  tensor_types.tsv > tensor_types_UD-Q5_K_M.txt
196
 
197
  # 3. quantize. The last argument is the fallback type for tensors the file doesn't list:
198
- # UD-Q2_K_XL Q2_K, UD-IQ3_XXS IQ3_XXS, UD-Q3_K_XL Q3_K_L, UD-IQ4_XS IQ4_XS,
199
- # UD-Q4_K_S Q4_K_S, UD-Q4_K_XL Q4_K_M, UD-Q5_K_S Q5_K_S, UD-Q5_K_M Q5_K_M,
200
- # UD-Q6_K_XL Q6_K, UD-Q8_K_XL Q8_0
201
  llama-quantize --imatrix imatrix_unsloth.gguf \
202
  --tensor-type-file tensor_types_UD-Q5_K_M.txt \
203
  Swift-Qwen3.8-27B-Uncensored-MTP-BF16.gguf \
 
39
  | :-- | :-- | :-- |
40
  | UD-Q2_K_XL | 9.2 GiB | Fits a 12 GB GPU. |
41
  | UD-IQ3_XXS | 10.2 GiB | Also fits a 12 GB GPU. Prefer it to UD-Q2_K_XL if it fits: clearly closer to the BF16. |
42
+ | UD-IQ3_S | 11.2 GiB | Fits a 16 GB GPU with more context room than UD-Q3_K_XL. |
43
  | UD-Q3_K_XL | 12.2 GiB | Fits a 16 GB GPU. |
44
  | UD-IQ4_XS | 13.3 GiB | Smallest 4-bit. Fits a 16 GB GPU, with less context room than UD-Q3_K_XL. |
45
  | UD-Q4_K_S | 14.3 GiB | Smaller 4-bit. Fits a 20 GB GPU, or a 16 GB GPU with short context. |
 
58
  | File | Size | Use |
59
  | :-- | :-- | :-- |
60
  | `mmproj-BF16.gguf` | 0.9 GiB | Vision projector, needed only for image and video input. |
61
+ | `tensor_types.tsv` | 42 KB | Per-tensor quantization types for each size (see [Quantization](#quantization)). |
62
  | `kld_results.csv` | 5 KB | Every statistic behind the [Quality](#quality) table. |
63
 
64
  ## Quality
 
72
  | :-- | --: | --: | --: | --: |
73
  | UD-Q2_K_XL | 9.2 | 0.1796 | 1.99 | 85.2 % |
74
  | UD-IQ3_XXS | 10.2 | 0.1255 | 1.20 | 87.9 % |
75
+ | UD-IQ3_S | 11.2 | — | — | — |
76
  | UD-Q3_K_XL | 12.2 | 0.0726 | 0.58 | 91.8 % |
77
  | UD-IQ4_XS | 13.3 | — | — | — |
78
  | UD-Q4_K_S | 14.3 | 0.0521 | 0.45 | 94.1 % |
 
82
  | UD-Q6_K_XL | 23.6 | 0.0116 | 0.028 | 98.2 % |
83
  | UD-Q8_K_XL | 29.3 | 0.0060 | 0.012 | 98.8 % |
84
 
85
+ - **UD-IQ3_S and UD-IQ4_XS** were added after this measurement run, so they have no numbers yet
86
+ and are absent from the chart for the same reason. Both come with the next batch.
87
  - **Mean KLD** is the average divergence of the quant's next-token distribution from the BF16's.
88
  **99 % KLD** is the level only the worst 1 % of tokens exceed. **Same top-1** is how often both
89
  pick the same most likely token.
 
153
  | :-- | :-- |
154
  | UD-Q2_K_XL | 112×IQ3_XXS, 98×Q8_0, 67×IQ2_S, 57×IQ3_S, 48×IQ2_XXS, 34×IQ2_XS, 21×Q4_K, 20×IQ1_S, 19×IQ4_XS, 16×Q2_K, 6×Q6_K, 5×Q3_K, 2×Q5_K, 1×IQ1_M |
155
  | UD-IQ3_XXS | 120×IQ3_XXS, 106×IQ3_S, 98×Q8_0, 45×IQ4_XS, 35×IQ2_S, 26×Q4_K, 24×IQ2_XXS, 12×IQ2_XS, 10×Q2_K, 8×IQ1_S, 7×Q5_K, 6×Q3_K, 6×Q6_K, 3×IQ1_M |
156
+ | UD-IQ3_S | 127×IQ3_S, 98×Q8_0, 88×IQ4_XS, 77×IQ3_XXS, 23×Q4_K, 21×IQ2_S, 16×Q5_K, 16×Q3_K, 12×IQ2_XS, 12×IQ2_XXS, 7×Q6_K, 6×Q2_K, 2×IQ1_S, 1×IQ4_NL |
157
  | UD-Q3_K_XL | 156×IQ4_XS, 111×IQ3_S, 98×Q8_0, 36×Q4_K, 34×IQ3_XXS, 26×Q5_K, 15×IQ2_S, 12×Q3_K, 7×Q6_K, 4×IQ2_XS, 3×Q2_K, 2×IQ4_NL, 2×IQ2_XXS |
158
  | UD-IQ4_XS | 211×IQ4_XS, 98×Q8_0, 55×Q4_K, 51×Q5_K, 46×IQ3_S, 17×IQ3_XXS, 9×Q3_K, 8×Q6_K, 7×IQ2_S, 2×IQ2_XS, 1×IQ4_NL, 1×Q2_K |
159
  | UD-Q4_K_S | 172×IQ4_XS, 99×Q8_0, 95×Q4_K, 80×Q5_K, 18×Q6_K, 15×IQ3_S, 13×Q3_K, 7×IQ4_NL, 5×IQ3_XXS, 1×IQ2_S, 1×IQ2_XS |
 
198
  tensor_types.tsv > tensor_types_UD-Q5_K_M.txt
199
 
200
  # 3. quantize. The last argument is the fallback type for tensors the file doesn't list:
201
+ # UD-Q2_K_XL Q2_K, UD-IQ3_XXS IQ3_XXS, UD-IQ3_S IQ3_S, UD-Q3_K_XL Q3_K_L,
202
+ # UD-IQ4_XS IQ4_XS, UD-Q4_K_S Q4_K_S, UD-Q4_K_XL Q4_K_M, UD-Q5_K_S Q5_K_S,
203
+ # UD-Q5_K_M Q5_K_M, UD-Q6_K_XL Q6_K, UD-Q8_K_XL Q8_0
204
  llama-quantize --imatrix imatrix_unsloth.gguf \
205
  --tensor-type-file tensor_types_UD-Q5_K_M.txt \
206
  Swift-Qwen3.8-27B-Uncensored-MTP-BF16.gguf \
Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:23be95d1e4acb9d34ef476a3adf56d28f53f9ce5408ff7543e282af0078ac484
3
+ size 12040881632
tensor_types.tsv CHANGED
@@ -1,507 +1,507 @@
1
- tensor UD-Q2_K_XL UD-IQ3_XXS UD-Q3_K_XL UD-IQ4_XS UD-Q4_K_S UD-Q4_K_XL UD-Q5_K_S UD-Q5_K_M UD-Q6_K_XL UD-Q8_K_XL
2
- output.weight q4_k q4_k q5_k q5_k q6_k q6_k q6_k q6_k q8_0 bf16
3
- token_embd.weight q2_k q2_k q3_k q3_k q3_k q4_k q4_k q4_k q6_k q8_0
4
- blk.0.attn_gate.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0 q8_0
5
- blk.0.attn_qkv.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
6
- blk.0.ffn_down.weight iq2_xs iq2_xs iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q5_k q6_k q8_0
7
- blk.0.ffn_gate.weight iq1_s iq1_s iq2_xs iq2_xs iq2_xs iq4_xs iq4_xs iq4_xs q5_k q8_0
8
- blk.0.ffn_up.weight iq1_s iq1_s iq2_s iq2_s iq2_s q3_k q3_k q5_k q6_k q8_0
9
- blk.0.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
10
- blk.0.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
11
- blk.0.ssm_out.weight q4_k q4_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 q8_0
12
- blk.1.attn_gate.weight iq3_xxs iq3_xxs q4_k q4_k q4_k q4_k q4_k q4_k q4_k q8_0
13
- blk.1.attn_qkv.weight iq2_s iq2_s q4_k q4_k q4_k q4_k q4_k q4_k q8_0 q8_0
14
- blk.1.ffn_down.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_nl q5_k q6_k q6_k q8_0
15
- blk.1.ffn_gate.weight iq1_s iq2_xxs iq2_s iq3_s iq3_s iq4_xs q4_k q4_k q5_k q8_0
16
- blk.1.ffn_up.weight iq1_s iq2_xxs q3_k q3_k q3_k iq4_xs iq4_xs iq4_xs q8_0 q8_0
17
- blk.1.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
18
- blk.1.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
19
- blk.1.ssm_out.weight iq3_s q4_k q4_k q4_k q6_k q6_k q6_k q6_k q8_0 q8_0
20
- blk.2.attn_gate.weight iq3_xxs iq3_xxs q4_k q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
21
- blk.2.attn_qkv.weight iq2_s iq3_xxs iq4_xs q4_k q4_k q4_k q6_k q6_k q6_k q8_0
22
- blk.2.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_nl q6_k q6_k q8_0 q8_0
23
- blk.2.ffn_gate.weight iq2_xxs iq2_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
24
- blk.2.ffn_up.weight iq2_xxs iq2_s iq3_s iq3_s iq4_xs iq4_xs iq4_nl iq4_nl q8_0 q8_0
25
- blk.2.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
26
- blk.2.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
27
- blk.2.ssm_out.weight iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q6_k q8_0
28
- blk.3.attn_k.weight iq3_xxs q4_k q4_k q4_k q4_k q4_k q4_k q6_k q6_k bf16
29
- blk.3.attn_output.weight q4_k q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
30
- blk.3.attn_q.weight q2_k q2_k iq4_nl iq4_nl iq4_nl q5_k q5_k q6_k q6_k bf16
31
- blk.3.attn_v.weight q4_k q4_k q5_k q5_k q5_k q5_k q5_k q5_k q5_k bf16
32
- blk.3.ffn_down.weight iq2_s iq3_xxs iq4_xs iq4_xs iq4_nl q5_k q6_k q6_k q6_k q8_0
33
- blk.3.ffn_gate.weight iq2_xs iq2_s q3_k q3_k q4_k q4_k q4_k q5_k q6_k q8_0
34
- blk.3.ffn_up.weight iq2_xs iq2_s iq3_s iq3_s q4_k q5_k q5_k q6_k q8_0 q8_0
35
- blk.4.attn_gate.weight iq2_s iq3_s q4_k q4_k q4_k q5_k q6_k q6_k q6_k q8_0
36
- blk.4.attn_qkv.weight iq2_s iq2_s iq4_xs iq4_xs q4_k q4_k q5_k q6_k q8_0 q8_0
37
- blk.4.ffn_down.weight iq2_s iq2_s q4_k q4_k q4_k q5_k q5_k q6_k q6_k q8_0
38
- blk.4.ffn_gate.weight iq2_xxs iq2_s iq3_s iq3_s iq3_s iq4_xs q5_k q5_k q6_k q8_0
39
- blk.4.ffn_up.weight iq2_s iq2_s iq3_s iq3_s q4_k q4_k q4_k q6_k q8_0 q8_0
40
- blk.4.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
41
- blk.4.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
42
- blk.4.ssm_out.weight iq3_s iq3_s q4_k q5_k q5_k q5_k q5_k q5_k q8_0 q8_0
43
- blk.5.attn_gate.weight iq3_xxs iq3_s iq4_xs q4_k q4_k q5_k q5_k q6_k q6_k q8_0
44
- blk.5.attn_qkv.weight iq2_xxs iq3_xxs iq3_s q4_k q4_k q5_k q5_k q5_k q6_k q8_0
45
- blk.5.ffn_down.weight iq2_xs iq2_s iq3_s iq3_s q4_k q5_k q5_k q5_k q8_0 q8_0
46
- blk.5.ffn_gate.weight iq2_xxs iq2_s iq3_s iq3_s iq3_s iq4_xs q5_k q5_k q6_k q8_0
47
- blk.5.ffn_up.weight iq2_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q6_k q8_0 q8_0
48
- blk.5.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
49
- blk.5.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
50
- blk.5.ssm_out.weight iq3_xxs iq4_xs q4_k q5_k q5_k q5_k q5_k q6_k q8_0 q8_0
51
- blk.6.attn_gate.weight iq3_xxs iq3_xxs q4_k q4_k q4_k q4_k q5_k q6_k q6_k q8_0
52
- blk.6.attn_qkv.weight iq2_xs iq3_xxs iq4_xs iq4_xs q4_k q6_k q6_k q6_k q6_k q8_0
53
- blk.6.ffn_down.weight iq2_xs iq2_s iq4_xs q4_k q4_k q4_k q5_k q5_k q8_0 q8_0
54
- blk.6.ffn_gate.weight iq2_xxs iq2_s iq3_xxs iq3_s q4_k q4_k q5_k q5_k q6_k q8_0
55
- blk.6.ffn_up.weight iq2_xs iq2_xs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
56
- blk.6.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
57
- blk.6.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
58
- blk.6.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_nl q6_k q6_k q6_k q8_0 q8_0
59
- blk.7.attn_k.weight iq3_s iq3_s q5_k q5_k q5_k q5_k q5_k q5_k q5_k bf16
60
- blk.7.attn_output.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_nl q5_k q5_k q5_k q8_0 q8_0
61
- blk.7.attn_q.weight q2_k q2_k q2_k q4_k q4_k q4_k q4_k q5_k q6_k bf16
62
- blk.7.attn_v.weight q4_k q4_k q5_k q5_k q5_k q5_k q5_k q5_k q8_0 bf16
63
- blk.7.ffn_down.weight q2_k q2_k q3_k iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
64
- blk.7.ffn_gate.weight iq2_xs iq2_s q3_k iq4_xs iq4_xs iq4_xs iq4_nl q5_k q8_0 q8_0
65
- blk.7.ffn_up.weight iq2_xs iq2_s iq3_s iq3_s iq3_s q5_k q5_k q5_k q6_k q8_0
66
- blk.8.attn_gate.weight iq2_s iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
67
- blk.8.attn_qkv.weight iq2_xxs iq2_xxs iq3_s q4_k q4_k q4_k q4_k q6_k q8_0 q8_0
68
- blk.8.ffn_down.weight iq2_xxs iq2_xxs iq3_s iq3_s q4_k q4_k q5_k q5_k q5_k q8_0
69
- blk.8.ffn_gate.weight iq2_xxs iq2_s iq2_s q3_k iq4_xs iq4_xs q5_k q5_k q6_k q8_0
70
- blk.8.ffn_up.weight iq2_xxs iq2_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
71
- blk.8.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
72
- blk.8.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
73
- blk.8.ssm_out.weight iq2_s q3_k q4_k q4_k q5_k q5_k q5_k q5_k q5_k q8_0
74
- blk.9.attn_gate.weight iq2_s iq2_s iq3_s q4_k q4_k q4_k q5_k q5_k q8_0 q8_0
75
- blk.9.attn_qkv.weight iq2_xxs iq2_xxs iq3_xxs iq3_s q4_k q4_k q5_k q5_k q6_k q8_0
76
- blk.9.ffn_down.weight iq2_xxs iq2_xs iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q6_k q8_0
77
- blk.9.ffn_gate.weight iq2_xxs iq2_xs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q6_k q8_0
78
- blk.9.ffn_up.weight iq2_xxs iq2_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0
79
- blk.9.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
80
- blk.9.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
81
- blk.9.ssm_out.weight iq2_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
82
- blk.10.attn_gate.weight iq2_s iq2_s iq3_xxs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0
83
- blk.10.attn_qkv.weight iq2_xxs iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q5_k q5_k q6_k q8_0
84
- blk.10.ffn_down.weight iq2_xxs iq2_xs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_nl q5_k q6_k q8_0
85
- blk.10.ffn_gate.weight iq1_s iq2_xs iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q4_k q8_0 q8_0
86
- blk.10.ffn_up.weight iq2_xxs iq2_xxs iq2_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
87
- blk.10.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
88
- blk.10.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
89
- blk.10.ssm_out.weight iq2_s q3_k q3_k iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
90
- blk.11.attn_k.weight iq3_s iq3_s iq4_nl q5_k q6_k q6_k q6_k q8_0 q8_0 bf16
91
- blk.11.attn_output.weight iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q5_k q5_k q5_k q5_k q8_0
92
- blk.11.attn_q.weight q2_k q2_k q2_k iq3_xxs iq4_xs q5_k q5_k q5_k q5_k bf16
93
- blk.11.attn_v.weight q4_k q4_k q4_k q4_k q4_k q8_0 q8_0 q8_0 q8_0 bf16
94
- blk.11.ffn_down.weight iq2_xxs iq2_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs iq4_xs q6_k q8_0
95
- blk.11.ffn_gate.weight iq2_xxs iq2_xxs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q6_k q8_0
96
- blk.11.ffn_up.weight iq2_xxs iq2_xxs iq2_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
97
- blk.12.attn_gate.weight iq2_xs iq3_xxs iq3_xxs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0
98
- blk.12.attn_qkv.weight iq2_xxs iq2_xxs iq3_xxs iq4_xs iq4_xs q4_k q4_k q5_k q5_k q8_0
99
- blk.12.ffn_down.weight iq2_xxs iq2_xxs iq2_s iq3_s iq3_s iq4_xs q4_k q5_k q5_k q8_0
100
- blk.12.ffn_gate.weight iq1_s iq2_xs iq2_xs iq2_xs q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
101
- blk.12.ffn_up.weight iq1_s iq1_s iq2_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q8_0
102
- blk.12.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
103
- blk.12.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
104
- blk.12.ssm_out.weight iq2_xs iq3_xxs q3_k q3_k q4_k q5_k q5_k q6_k q6_k q8_0
105
- blk.13.attn_gate.weight iq2_xxs iq2_s iq2_s iq3_s q4_k q4_k q4_k q5_k q6_k q8_0
106
- blk.13.attn_qkv.weight iq2_xxs iq2_xxs iq3_s q4_k q4_k q4_k q4_k q4_k q6_k q8_0
107
- blk.13.ffn_down.weight iq1_s iq1_s iq2_s q3_k q3_k q4_k q4_k q5_k q5_k q8_0
108
- blk.13.ffn_gate.weight iq1_s iq1_m iq2_xs q3_k q3_k q3_k q3_k q5_k q5_k q8_0
109
- blk.13.ffn_up.weight iq1_s iq1_s iq3_xxs q3_k q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
110
- blk.13.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
111
- blk.13.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
112
- blk.13.ssm_out.weight iq2_xs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
113
- blk.14.attn_gate.weight iq2_xxs iq2_xxs iq3_s q4_k q4_k q4_k q4_k q5_k q8_0 q8_0
114
- blk.14.attn_qkv.weight iq1_m q2_k iq3_s iq4_xs iq4_xs q4_k q4_k q5_k q6_k q8_0
115
- blk.14.ffn_down.weight iq1_s iq1_m iq2_xs iq3_xxs iq3_xxs iq3_s q4_k q4_k q5_k q8_0
116
- blk.14.ffn_gate.weight iq1_s iq2_xxs iq2_xxs iq2_s q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
117
- blk.14.ffn_up.weight iq1_s iq1_s iq2_s iq2_s q3_k q3_k iq4_xs iq4_xs q6_k q8_0
118
- blk.14.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
119
- blk.14.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
120
- blk.14.ssm_out.weight iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q4_k q4_k q6_k q8_0 q8_0
121
- blk.15.attn_k.weight iq3_s iq3_s q4_k q4_k q6_k q6_k q6_k q6_k q6_k bf16
122
- blk.15.attn_output.weight iq2_xxs iq2_xxs q3_k iq4_xs iq4_xs iq4_xs iq4_xs q6_k q8_0 q8_0
123
- blk.15.attn_q.weight iq2_xxs q2_k q2_k q2_k q4_k q4_k q4_k q4_k q5_k bf16
124
- blk.15.attn_v.weight q4_k q4_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 bf16
125
- blk.15.ffn_down.weight iq1_s iq1_m iq2_s iq2_s iq3_s iq4_xs iq4_xs iq4_xs q8_0 q8_0
126
- blk.15.ffn_gate.weight iq1_s iq1_s iq2_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q8_0
127
- blk.15.ffn_up.weight iq1_s iq2_xxs iq2_s iq2_s iq3_s iq4_xs iq4_xs iq4_xs q8_0 q8_0
128
- blk.16.attn_gate.weight iq2_xxs iq2_s q4_k q4_k q4_k q4_k q5_k q5_k q5_k q8_0
129
- blk.16.attn_qkv.weight q2_k iq3_s iq3_s iq3_s q4_k q4_k q6_k q6_k q6_k q8_0
130
- blk.16.ffn_down.weight iq2_xxs iq2_xs iq2_s iq3_s iq3_s q4_k q4_k q4_k q6_k q8_0
131
- blk.16.ffn_gate.weight iq2_xxs iq2_xxs iq2_s iq2_s q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
132
- blk.16.ffn_up.weight iq1_s iq2_xxs iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q6_k q8_0
133
- blk.16.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
134
- blk.16.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
135
- blk.16.ssm_out.weight iq2_s iq2_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
136
- blk.17.attn_gate.weight iq2_xs iq2_xs iq4_xs iq4_xs iq4_xs iq4_xs iq4_xs iq4_xs q5_k q8_0
137
- blk.17.attn_qkv.weight iq2_xs iq3_xxs iq3_xxs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
138
- blk.17.ffn_down.weight iq2_xxs iq2_xxs iq3_xxs iq3_xxs iq3_s q4_k q4_k q4_k q5_k q8_0
139
- blk.17.ffn_gate.weight iq1_s iq1_s iq3_xxs iq3_xxs iq3_xxs iq4_nl iq4_nl iq4_nl q8_0 q8_0
140
- blk.17.ffn_up.weight iq2_xxs iq2_xxs iq2_s iq2_s q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
141
- blk.17.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
142
- blk.17.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
143
- blk.17.ssm_out.weight iq2_xs q3_k q3_k iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
144
- blk.18.attn_gate.weight iq2_xxs q2_k iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q8_0 q8_0
145
- blk.18.attn_qkv.weight iq2_xxs q2_k iq4_xs iq4_xs q4_k q4_k q4_k q6_k q6_k q8_0
146
- blk.18.ffn_down.weight iq2_xs iq2_s iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q8_0 q8_0
147
- blk.18.ffn_gate.weight iq1_s iq2_xxs iq3_xxs iq3_xxs q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
148
- blk.18.ffn_up.weight iq2_xxs iq2_xs iq3_xxs iq3_xxs q3_k iq4_xs iq4_xs q5_k q6_k q8_0
149
- blk.18.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
150
- blk.18.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
151
- blk.18.ssm_out.weight iq2_xs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0
152
- blk.19.attn_k.weight q3_k q4_k q4_k q4_k q5_k q6_k q8_0 q8_0 q8_0 bf16
153
- blk.19.attn_output.weight iq2_xs iq3_s iq3_s iq4_xs iq4_xs q5_k q6_k q6_k q8_0 q8_0
154
- blk.19.attn_q.weight iq2_xxs iq2_xxs iq3_xxs iq3_xxs q4_k q4_k q4_k q5_k q6_k bf16
155
- blk.19.attn_v.weight q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0 bf16
156
- blk.19.ffn_down.weight iq2_xxs iq2_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
157
- blk.19.ffn_gate.weight iq1_s iq2_xs iq3_xxs iq3_xxs q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
158
- blk.19.ffn_up.weight iq2_xxs iq2_xxs iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q8_0 q8_0
159
- blk.20.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
160
- blk.20.attn_qkv.weight iq2_s iq3_s iq4_xs iq4_xs q4_k q4_k q5_k q6_k q6_k q8_0
161
- blk.20.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q8_0
162
- blk.20.ffn_gate.weight iq2_xxs iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q6_k q8_0
163
- blk.20.ffn_up.weight iq2_xs iq3_s iq3_s iq3_s iq4_xs q5_k q5_k q5_k q6_k q8_0
164
- blk.20.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
165
- blk.20.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
166
- blk.20.ssm_out.weight iq3_s iq3_s q4_k q4_k q5_k q5_k q5_k q6_k q8_0 q8_0
167
- blk.21.attn_gate.weight iq3_s iq3_s q4_k q4_k q4_k q5_k q5_k q8_0 q8_0 q8_0
168
- blk.21.attn_qkv.weight iq2_s iq3_xxs iq3_xxs iq4_xs iq4_nl iq4_nl q6_k q6_k q6_k q8_0
169
- blk.21.ffn_down.weight iq3_xxs iq3_xxs iq4_xs q4_k q4_k q5_k q5_k q5_k q8_0 q8_0
170
- blk.21.ffn_gate.weight iq2_s iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q5_k q5_k q8_0 q8_0
171
- blk.21.ffn_up.weight iq2_xxs q3_k q3_k iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
172
- blk.21.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
173
- blk.21.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
174
- blk.21.ssm_out.weight iq3_s iq3_s iq4_xs iq4_xs q5_k q6_k q6_k q6_k q6_k q8_0
175
- blk.22.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0
176
- blk.22.attn_qkv.weight iq2_s iq3_xxs iq3_s iq3_s q4_k q4_k q4_k q4_k q8_0 q8_0
177
- blk.22.ffn_down.weight iq2_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
178
- blk.22.ffn_gate.weight iq2_xs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q8_0
179
- blk.22.ffn_up.weight iq2_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
180
- blk.22.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
181
- blk.22.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
182
- blk.22.ssm_out.weight iq3_xxs iq3_s iq4_xs q4_k q4_k q6_k q6_k q6_k q6_k q8_0
183
- blk.23.attn_k.weight iq3_s iq3_s q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 bf16
184
- blk.23.attn_output.weight iq3_s iq3_s q4_k q4_k q4_k q6_k q6_k q6_k q6_k q8_0
185
- blk.23.attn_q.weight q2_k iq3_s iq3_s q4_k q4_k q4_k q5_k q6_k q6_k bf16
186
- blk.23.attn_v.weight q4_k q5_k q5_k q6_k q6_k q8_0 q8_0 q8_0 q8_0 bf16
187
- blk.23.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q6_k q6_k q6_k q8_0
188
- blk.23.ffn_gate.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
189
- blk.23.ffn_up.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
190
- blk.24.attn_gate.weight iq3_xxs iq3_s iq4_xs q4_k q4_k q5_k q5_k q8_0 q8_0 q8_0
191
- blk.24.attn_qkv.weight iq3_xxs iq3_xxs iq3_s q4_k q4_k q5_k q5_k q5_k q5_k q8_0
192
- blk.24.ffn_down.weight q3_k q3_k iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
193
- blk.24.ffn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
194
- blk.24.ffn_up.weight iq3_xxs iq3_xxs q4_k q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
195
- blk.24.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
196
- blk.24.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
197
- blk.24.ssm_out.weight iq3_xxs iq3_xxs iq4_xs q4_k q4_k q6_k q6_k q6_k q6_k q8_0
198
- blk.25.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0 q8_0
199
- blk.25.attn_qkv.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_nl q5_k q5_k q5_k q8_0 q8_0
200
- blk.25.ffn_down.weight iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q5_k q6_k q8_0
201
- blk.25.ffn_gate.weight iq2_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
202
- blk.25.ffn_up.weight q3_k q3_k iq4_xs q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
203
- blk.25.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
204
- blk.25.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
205
- blk.25.ssm_out.weight q3_k iq4_xs iq4_xs iq4_xs iq4_xs q6_k q6_k q6_k q6_k q8_0
206
- blk.26.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q8_0
207
- blk.26.attn_qkv.weight iq3_s iq3_s iq3_s q4_k q4_k q5_k q5_k q5_k q5_k q8_0
208
- blk.26.ffn_down.weight iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
209
- blk.26.ffn_gate.weight iq2_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
210
- blk.26.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
211
- blk.26.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
212
- blk.26.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
213
- blk.26.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q5_k q5_k q8_0
214
- blk.27.attn_k.weight iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0 q8_0 bf16
215
- blk.27.attn_output.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q6_k q8_0
216
- blk.27.attn_q.weight iq2_xs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q6_k q6_k q6_k bf16
217
- blk.27.attn_v.weight q4_k q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 bf16
218
- blk.27.ffn_down.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
219
- blk.27.ffn_gate.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_nl q5_k q5_k q8_0 q8_0
220
- blk.27.ffn_up.weight iq2_s iq3_s iq3_s iq4_xs iq4_nl q5_k q5_k q5_k q8_0 q8_0
221
- blk.28.attn_gate.weight iq2_s iq3_s q4_k q4_k q4_k q6_k q6_k q6_k q6_k q8_0
222
- blk.28.attn_qkv.weight iq3_xxs iq3_s iq4_xs q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
223
- blk.28.ffn_down.weight iq2_s iq3_xxs q3_k iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
224
- blk.28.ffn_gate.weight iq2_xs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
225
- blk.28.ffn_up.weight iq2_xs iq3_xxs iq4_xs iq4_xs iq4_xs iq4_nl q5_k q5_k q6_k q8_0
226
- blk.28.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
227
- blk.28.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
228
- blk.28.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0
229
- blk.29.attn_gate.weight iq2_s iq3_s q4_k q4_k q4_k q4_k q4_k q6_k q8_0 q8_0
230
- blk.29.attn_qkv.weight iq2_s iq3_s iq4_xs iq4_xs q4_k q4_k q5_k q5_k q8_0 q8_0
231
- blk.29.ffn_down.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0 q8_0
232
- blk.29.ffn_gate.weight iq2_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q8_0 q8_0
233
- blk.29.ffn_up.weight iq2_s iq2_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q8_0 q8_0
234
- blk.29.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
235
- blk.29.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
236
- blk.29.ssm_out.weight iq3_xxs iq3_xxs q4_k q4_k q4_k q5_k q6_k q6_k q6_k q8_0
237
- blk.30.attn_gate.weight q2_k iq3_s iq3_s q4_k q4_k q4_k q5_k q5_k q8_0 q8_0
238
- blk.30.attn_qkv.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
239
- blk.30.ffn_down.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
240
- blk.30.ffn_gate.weight iq2_xs iq2_xs iq3_s iq3_s iq4_xs q4_k q5_k q5_k q6_k q8_0
241
- blk.30.ffn_up.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
242
- blk.30.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
243
- blk.30.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
244
- blk.30.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
245
- blk.31.attn_k.weight q4_k q4_k q4_k q4_k q5_k q8_0 q8_0 q8_0 q8_0 bf16
246
- blk.31.attn_output.weight iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
247
- blk.31.attn_q.weight q2_k iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 bf16
248
- blk.31.attn_v.weight q4_k q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 bf16
249
- blk.31.ffn_down.weight iq2_s iq2_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
250
- blk.31.ffn_gate.weight iq2_xxs iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q8_0
251
- blk.31.ffn_up.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q8_0
252
- blk.32.attn_gate.weight iq2_xs iq3_xxs iq3_xxs iq4_xs q4_k q5_k q5_k q6_k q6_k q8_0
253
- blk.32.attn_qkv.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
254
- blk.32.ffn_down.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
255
- blk.32.ffn_gate.weight iq2_xxs iq2_s iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q6_k q8_0
256
- blk.32.ffn_up.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
257
- blk.32.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
258
- blk.32.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
259
- blk.32.ssm_out.weight iq3_xxs iq3_s iq4_xs q4_k q4_k q5_k q5_k q5_k q8_0 q8_0
260
- blk.33.attn_gate.weight iq2_xxs iq3_xxs iq4_xs iq4_xs q4_k q4_k q5_k q5_k q6_k q8_0
261
- blk.33.attn_qkv.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
262
- blk.33.ffn_down.weight iq2_s iq2_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q8_0
263
- blk.33.ffn_gate.weight iq2_xxs iq2_s iq3_s iq3_s iq4_xs iq4_xs q4_k q6_k q6_k q8_0
264
- blk.33.ffn_up.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q8_0 q8_0
265
- blk.33.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
266
- blk.33.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
267
- blk.33.ssm_out.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
268
- blk.34.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q5_k q5_k q5_k q6_k q8_0
269
- blk.34.attn_qkv.weight iq3_xxs iq3_s iq3_s iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
270
- blk.34.ffn_down.weight iq2_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
271
- blk.34.ffn_gate.weight iq2_xs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
272
- blk.34.ffn_up.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
273
- blk.34.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
274
- blk.34.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
275
- blk.34.ssm_out.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
276
- blk.35.attn_k.weight q4_k q4_k q4_k q4_k q4_k q6_k q6_k q8_0 q8_0 bf16
277
- blk.35.attn_output.weight iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
278
- blk.35.attn_q.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 bf16
279
- blk.35.attn_v.weight q4_k q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 bf16
280
- blk.35.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
281
- blk.35.ffn_gate.weight iq2_s iq2_s iq3_s iq3_s iq4_xs q4_k q4_k q5_k q6_k q8_0
282
- blk.35.ffn_up.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
283
- blk.36.attn_gate.weight iq2_xs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
284
- blk.36.attn_qkv.weight iq2_xs iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
285
- blk.36.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
286
- blk.36.ffn_gate.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
287
- blk.36.ffn_up.weight iq2_s iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
288
- blk.36.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
289
- blk.36.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
290
- blk.36.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
291
- blk.37.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
292
- blk.37.attn_qkv.weight iq2_xs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
293
- blk.37.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
294
- blk.37.ffn_gate.weight iq2_s iq3_xxs iq3_s iq4_xs q4_k q4_k q5_k q5_k q8_0 q8_0
295
- blk.37.ffn_up.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
296
- blk.37.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
297
- blk.37.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
298
- blk.37.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
299
- blk.38.attn_gate.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
300
- blk.38.attn_qkv.weight iq2_s iq2_s iq3_s iq3_s iq4_xs q4_k q4_k q5_k q6_k q8_0
301
- blk.38.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q5_k q6_k q8_0
302
- blk.38.ffn_gate.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
303
- blk.38.ffn_up.weight iq3_xxs iq3_s iq3_s iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
304
- blk.38.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
305
- blk.38.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
306
- blk.38.ssm_out.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
307
- blk.39.attn_k.weight iq3_s iq3_s q4_k q4_k q4_k q6_k q6_k q6_k q6_k bf16
308
- blk.39.attn_output.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
309
- blk.39.attn_q.weight q2_k iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k bf16
310
- blk.39.attn_v.weight q4_k q4_k q5_k q5_k q5_k q6_k q6_k q6_k q8_0 bf16
311
- blk.39.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
312
- blk.39.ffn_gate.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
313
- blk.39.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
314
- blk.40.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
315
- blk.40.attn_qkv.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
316
- blk.40.ffn_down.weight iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
317
- blk.40.ffn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q5_k q5_k q5_k q6_k q8_0
318
- blk.40.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
319
- blk.40.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
320
- blk.40.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
321
- blk.40.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
322
- blk.41.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
323
- blk.41.attn_qkv.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
324
- blk.41.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
325
- blk.41.ffn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
326
- blk.41.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
327
- blk.41.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
328
- blk.41.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
329
- blk.41.ssm_out.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
330
- blk.42.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
331
- blk.42.attn_qkv.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
332
- blk.42.ffn_down.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
333
- blk.42.ffn_gate.weight iq3_xxs iq3_xxs iq3_s iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
334
- blk.42.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
335
- blk.42.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
336
- blk.42.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
337
- blk.42.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
338
- blk.43.attn_k.weight iq4_xs q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 bf16
339
- blk.43.attn_output.weight iq3_s q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
340
- blk.43.attn_q.weight q2_k iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 bf16
341
- blk.43.attn_v.weight q4_k q4_k q5_k q5_k q5_k q6_k q8_0 q8_0 q8_0 bf16
342
- blk.43.ffn_down.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
343
- blk.43.ffn_gate.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
344
- blk.43.ffn_up.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
345
- blk.44.attn_gate.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q4_k q4_k q6_k q6_k q8_0
346
- blk.44.attn_qkv.weight iq3_xxs iq3_xxs iq3_s iq4_xs q4_k q4_k q5_k q5_k q8_0 q8_0
347
- blk.44.ffn_down.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
348
- blk.44.ffn_gate.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
349
- blk.44.ffn_up.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
350
- blk.44.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
351
- blk.44.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
352
- blk.44.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
353
- blk.45.attn_gate.weight iq2_xxs iq2_s iq3_xxs q3_k iq4_xs iq4_xs iq4_xs iq4_xs q6_k q8_0
354
- blk.45.attn_qkv.weight q2_k iq3_s iq3_s iq3_s q4_k q4_k q5_k q5_k q6_k q8_0
355
- blk.45.ffn_down.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
356
- blk.45.ffn_gate.weight iq2_xs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q8_0
357
- blk.45.ffn_up.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
358
- blk.45.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
359
- blk.45.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
360
- blk.45.ssm_out.weight iq3_xxs iq3_xxs iq3_s q4_k q4_k q5_k q5_k q5_k q8_0 q8_0
361
- blk.46.attn_gate.weight q2_k iq3_xxs iq3_s iq4_xs q4_k q4_k q5_k q5_k q8_0 q8_0
362
- blk.46.attn_qkv.weight q2_k iq3_s iq3_s iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
363
- blk.46.ffn_down.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
364
- blk.46.ffn_gate.weight iq2_s iq2_s iq3_s iq3_s iq4_xs q4_k q5_k q5_k q8_0 q8_0
365
- blk.46.ffn_up.weight iq2_xs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
366
- blk.46.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
367
- blk.46.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
368
- blk.46.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
369
- blk.47.attn_k.weight iq3_s q4_k q4_k q5_k q5_k q6_k q6_k q8_0 q8_0 bf16
370
- blk.47.attn_output.weight iq3_s iq3_s iq4_xs q4_k q5_k q5_k q6_k q6_k q8_0 q8_0
371
- blk.47.attn_q.weight iq2_xs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k bf16
372
- blk.47.attn_v.weight q4_k q4_k q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 bf16
373
- blk.47.ffn_down.weight iq2_xs iq3_xxs q3_k iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
374
- blk.47.ffn_gate.weight iq2_xxs iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q6_k q8_0
375
- blk.47.ffn_up.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
376
- blk.48.attn_gate.weight iq2_s iq2_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
377
- blk.48.attn_qkv.weight iq3_xxs iq3_s iq4_xs q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
378
- blk.48.ffn_down.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
379
- blk.48.ffn_gate.weight iq2_xs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
380
- blk.48.ffn_up.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
381
- blk.48.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
382
- blk.48.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
383
- blk.48.ssm_out.weight iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
384
- blk.49.attn_gate.weight q2_k q2_k iq3_xxs iq4_xs iq4_xs iq4_xs iq4_xs iq4_xs q8_0 q8_0
385
- blk.49.attn_qkv.weight iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
386
- blk.49.ffn_down.weight iq2_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
387
- blk.49.ffn_gate.weight q2_k iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
388
- blk.49.ffn_up.weight iq3_xxs iq3_s iq3_s iq4_xs q4_k q5_k q5_k q5_k q6_k q8_0
389
- blk.49.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
390
- blk.49.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
391
- blk.49.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
392
- blk.50.attn_gate.weight iq3_xxs iq3_xxs iq4_xs q4_k q4_k q4_k q5_k q5_k q8_0 q8_0
393
- blk.50.attn_qkv.weight iq3_xxs iq3_xxs iq4_xs q4_k q4_k q5_k q5_k q5_k q8_0 q8_0
394
- blk.50.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
395
- blk.50.ffn_gate.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
396
- blk.50.ffn_up.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
397
- blk.50.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
398
- blk.50.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
399
- blk.50.ssm_out.weight q3_k iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q8_0 q8_0
400
- blk.51.attn_k.weight q4_k q4_k q5_k q5_k q5_k q6_k q6_k q6_k q8_0 bf16
401
- blk.51.attn_output.weight iq4_xs iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q8_0 q8_0
402
- blk.51.attn_q.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 bf16
403
- blk.51.attn_v.weight q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 q8_0 bf16
404
- blk.51.ffn_down.weight iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
405
- blk.51.ffn_gate.weight iq2_s iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
406
- blk.51.ffn_up.weight iq3_xxs iq3_s iq4_xs q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
407
- blk.52.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
408
- blk.52.attn_qkv.weight iq3_xxs iq3_xxs q4_k q4_k q4_k q4_k q5_k q6_k q8_0 q8_0
409
- blk.52.ffn_down.weight iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
410
- blk.52.ffn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
411
- blk.52.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
412
- blk.52.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
413
- blk.52.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
414
- blk.52.ssm_out.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q8_0 q8_0
415
- blk.53.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
416
- blk.53.attn_qkv.weight iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs q4_k q5_k q5_k q6_k q8_0
417
- blk.53.ffn_down.weight iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
418
- blk.53.ffn_gate.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
419
- blk.53.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
420
- blk.53.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
421
- blk.53.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
422
- blk.53.ssm_out.weight iq3_s iq4_xs q4_k q4_k q5_k q5_k q6_k q6_k q8_0 q8_0
423
- blk.54.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
424
- blk.54.attn_qkv.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs q4_k q5_k q5_k q6_k q8_0
425
- blk.54.ffn_down.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
426
- blk.54.ffn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
427
- blk.54.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
428
- blk.54.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
429
- blk.54.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
430
- blk.54.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
431
- blk.55.attn_k.weight iq3_s q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 bf16
432
- blk.55.attn_output.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
433
- blk.55.attn_q.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs q4_k q5_k q5_k q6_k bf16
434
- blk.55.attn_v.weight q4_k q4_k q5_k q5_k q6_k q8_0 q8_0 q8_0 q8_0 bf16
435
- blk.55.ffn_down.weight iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0 q8_0
436
- blk.55.ffn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
437
- blk.55.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
438
- blk.56.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
439
- blk.56.attn_qkv.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
440
- blk.56.ffn_down.weight iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
441
- blk.56.ffn_gate.weight iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
442
- blk.56.ffn_up.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
443
- blk.56.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
444
- blk.56.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
445
- blk.56.ssm_out.weight iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
446
- blk.57.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
447
- blk.57.attn_qkv.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
448
- blk.57.ffn_down.weight iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
449
- blk.57.ffn_gate.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
450
- blk.57.ffn_up.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
451
- blk.57.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
452
- blk.57.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
453
- blk.57.ssm_out.weight iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
454
- blk.58.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
455
- blk.58.attn_qkv.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
456
- blk.58.ffn_down.weight iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
457
- blk.58.ffn_gate.weight iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
458
- blk.58.ffn_up.weight iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0 q8_0
459
- blk.58.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
460
- blk.58.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
461
- blk.58.ssm_out.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q8_0 q8_0
462
- blk.59.attn_k.weight iq4_xs q4_k q5_k q5_k q5_k q6_k q8_0 q8_0 q8_0 bf16
463
- blk.59.attn_output.weight iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q8_0 q8_0
464
- blk.59.attn_q.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 bf16
465
- blk.59.attn_v.weight q4_k q5_k q5_k q5_k q6_k q8_0 q8_0 q8_0 q8_0 bf16
466
- blk.59.ffn_down.weight iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
467
- blk.59.ffn_gate.weight iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
468
- blk.59.ffn_up.weight iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
469
- blk.60.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
470
- blk.60.attn_qkv.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
471
- blk.60.ffn_down.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
472
- blk.60.ffn_gate.weight iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
473
- blk.60.ffn_up.weight iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0 q8_0
474
- blk.60.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
475
- blk.60.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
476
- blk.60.ssm_out.weight iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0 q8_0
477
- blk.61.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
478
- blk.61.attn_qkv.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
479
- blk.61.ffn_down.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
480
- blk.61.ffn_gate.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
481
- blk.61.ffn_up.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
482
- blk.61.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
483
- blk.61.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
484
- blk.61.ssm_out.weight iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
485
- blk.62.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
486
- blk.62.attn_qkv.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
487
- blk.62.ffn_down.weight iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
488
- blk.62.ffn_gate.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
489
- blk.62.ffn_up.weight iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q8_0 q8_0
490
- blk.62.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
491
- blk.62.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
492
- blk.62.ssm_out.weight iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
493
- blk.63.attn_k.weight q4_k q4_k q5_k q5_k q6_k q6_k q8_0 q8_0 q8_0 bf16
494
- blk.63.attn_output.weight iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0 q8_0 q8_0
495
- blk.63.attn_q.weight iq3_xxs iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 bf16
496
- blk.63.attn_v.weight q5_k q5_k q6_k q6_k q6_k q8_0 q8_0 q8_0 q8_0 bf16
497
- blk.63.ffn_down.weight iq4_xs q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
498
- blk.63.ffn_gate.weight iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
499
- blk.63.ffn_up.weight iq4_xs q4_k q5_k q5_k q6_k q6_k q6_k q6_k q8_0 q8_0
500
- blk.64.attn_k.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 bf16
501
- blk.64.attn_output.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q8_0
502
- blk.64.attn_q.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k bf16
503
- blk.64.attn_v.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 bf16
504
- blk.64.ffn_down.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q8_0
505
- blk.64.ffn_gate.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q8_0
506
- blk.64.ffn_up.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q8_0
507
- blk.64.nextn.eh_proj.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k bf16
 
1
+ tensor UD-Q2_K_XL UD-IQ3_XXS UD-IQ3_S UD-Q3_K_XL UD-IQ4_XS UD-Q4_K_S UD-Q4_K_XL UD-Q5_K_S UD-Q5_K_M UD-Q6_K_XL UD-Q8_K_XL
2
+ output.weight q4_k q4_k q5_k q5_k q5_k q6_k q6_k q6_k q6_k q8_0 bf16
3
+ token_embd.weight q2_k q2_k q3_k q3_k q3_k q3_k q4_k q4_k q4_k q6_k q8_0
4
+ blk.0.attn_gate.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0 q8_0
5
+ blk.0.attn_qkv.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
6
+ blk.0.ffn_down.weight iq2_xs iq2_xs iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q5_k q6_k q8_0
7
+ blk.0.ffn_gate.weight iq1_s iq1_s iq2_xs iq2_xs iq2_xs iq2_xs iq4_xs iq4_xs iq4_xs q5_k q8_0
8
+ blk.0.ffn_up.weight iq1_s iq1_s iq1_s iq2_s iq2_s iq2_s q3_k q3_k q5_k q6_k q8_0
9
+ blk.0.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
10
+ blk.0.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
11
+ blk.0.ssm_out.weight q4_k q4_k q4_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 q8_0
12
+ blk.1.attn_gate.weight iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q4_k q4_k q4_k q4_k q4_k q8_0
13
+ blk.1.attn_qkv.weight iq2_s iq2_s iq3_xxs q4_k q4_k q4_k q4_k q4_k q4_k q8_0 q8_0
14
+ blk.1.ffn_down.weight iq2_s iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_nl q5_k q6_k q6_k q8_0
15
+ blk.1.ffn_gate.weight iq1_s iq2_xxs iq2_s iq2_s iq3_s iq3_s iq4_xs q4_k q4_k q5_k q8_0
16
+ blk.1.ffn_up.weight iq1_s iq2_xxs iq2_xxs q3_k q3_k q3_k iq4_xs iq4_xs iq4_xs q8_0 q8_0
17
+ blk.1.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
18
+ blk.1.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
19
+ blk.1.ssm_out.weight iq3_s q4_k q4_k q4_k q4_k q6_k q6_k q6_k q6_k q8_0 q8_0
20
+ blk.2.attn_gate.weight iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
21
+ blk.2.attn_qkv.weight iq2_s iq3_xxs iq4_xs iq4_xs q4_k q4_k q4_k q6_k q6_k q6_k q8_0
22
+ blk.2.ffn_down.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs iq4_nl q6_k q6_k q8_0 q8_0
23
+ blk.2.ffn_gate.weight iq2_xxs iq2_s iq2_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
24
+ blk.2.ffn_up.weight iq2_xxs iq2_s iq2_s iq3_s iq3_s iq4_xs iq4_xs iq4_nl iq4_nl q8_0 q8_0
25
+ blk.2.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
26
+ blk.2.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
27
+ blk.2.ssm_out.weight iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q6_k q8_0
28
+ blk.3.attn_k.weight iq3_xxs q4_k q4_k q4_k q4_k q4_k q4_k q4_k q6_k q6_k bf16
29
+ blk.3.attn_output.weight q4_k q4_k q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
30
+ blk.3.attn_q.weight q2_k q2_k q2_k iq4_nl iq4_nl iq4_nl q5_k q5_k q6_k q6_k bf16
31
+ blk.3.attn_v.weight q4_k q4_k q5_k q5_k q5_k q5_k q5_k q5_k q5_k q5_k bf16
32
+ blk.3.ffn_down.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_nl q5_k q6_k q6_k q6_k q8_0
33
+ blk.3.ffn_gate.weight iq2_xs iq2_s iq2_s q3_k q3_k q4_k q4_k q4_k q5_k q6_k q8_0
34
+ blk.3.ffn_up.weight iq2_xs iq2_s iq3_s iq3_s iq3_s q4_k q5_k q5_k q6_k q8_0 q8_0
35
+ blk.4.attn_gate.weight iq2_s iq3_s iq3_s q4_k q4_k q4_k q5_k q6_k q6_k q6_k q8_0
36
+ blk.4.attn_qkv.weight iq2_s iq2_s iq3_s iq4_xs iq4_xs q4_k q4_k q5_k q6_k q8_0 q8_0
37
+ blk.4.ffn_down.weight iq2_s iq2_s iq2_s q4_k q4_k q4_k q5_k q5_k q6_k q6_k q8_0
38
+ blk.4.ffn_gate.weight iq2_xxs iq2_s iq2_s iq3_s iq3_s iq3_s iq4_xs q5_k q5_k q6_k q8_0
39
+ blk.4.ffn_up.weight iq2_s iq2_s iq2_s iq3_s iq3_s q4_k q4_k q4_k q6_k q8_0 q8_0
40
+ blk.4.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
41
+ blk.4.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
42
+ blk.4.ssm_out.weight iq3_s iq3_s q4_k q4_k q5_k q5_k q5_k q5_k q5_k q8_0 q8_0
43
+ blk.5.attn_gate.weight iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q5_k q6_k q6_k q8_0
44
+ blk.5.attn_qkv.weight iq2_xxs iq3_xxs iq3_s iq3_s q4_k q4_k q5_k q5_k q5_k q6_k q8_0
45
+ blk.5.ffn_down.weight iq2_xs iq2_s iq3_s iq3_s iq3_s q4_k q5_k q5_k q5_k q8_0 q8_0
46
+ blk.5.ffn_gate.weight iq2_xxs iq2_s iq2_s iq3_s iq3_s iq3_s iq4_xs q5_k q5_k q6_k q8_0
47
+ blk.5.ffn_up.weight iq2_xxs iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q6_k q8_0 q8_0
48
+ blk.5.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
49
+ blk.5.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
50
+ blk.5.ssm_out.weight iq3_xxs iq4_xs q4_k q4_k q5_k q5_k q5_k q5_k q6_k q8_0 q8_0
51
+ blk.6.attn_gate.weight iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q4_k q4_k q5_k q6_k q6_k q8_0
52
+ blk.6.attn_qkv.weight iq2_xs iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q6_k q6_k q6_k q6_k q8_0
53
+ blk.6.ffn_down.weight iq2_xs iq2_s iq3_xxs iq4_xs q4_k q4_k q4_k q5_k q5_k q8_0 q8_0
54
+ blk.6.ffn_gate.weight iq2_xxs iq2_s iq2_s iq3_xxs iq3_s q4_k q4_k q5_k q5_k q6_k q8_0
55
+ blk.6.ffn_up.weight iq2_xs iq2_xs iq2_xs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
56
+ blk.6.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
57
+ blk.6.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
58
+ blk.6.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_nl q6_k q6_k q6_k q8_0 q8_0
59
+ blk.7.attn_k.weight iq3_s iq3_s q5_k q5_k q5_k q5_k q5_k q5_k q5_k q5_k bf16
60
+ blk.7.attn_output.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs iq4_nl q5_k q5_k q5_k q8_0 q8_0
61
+ blk.7.attn_q.weight q2_k q2_k q2_k q2_k q4_k q4_k q4_k q4_k q5_k q6_k bf16
62
+ blk.7.attn_v.weight q4_k q4_k q5_k q5_k q5_k q5_k q5_k q5_k q5_k q8_0 bf16
63
+ blk.7.ffn_down.weight q2_k q2_k q3_k q3_k iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
64
+ blk.7.ffn_gate.weight iq2_xs iq2_s iq2_s q3_k iq4_xs iq4_xs iq4_xs iq4_nl q5_k q8_0 q8_0
65
+ blk.7.ffn_up.weight iq2_xs iq2_s iq3_xxs iq3_s iq3_s iq3_s q5_k q5_k q5_k q6_k q8_0
66
+ blk.8.attn_gate.weight iq2_s iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
67
+ blk.8.attn_qkv.weight iq2_xxs iq2_xxs iq3_s iq3_s q4_k q4_k q4_k q4_k q6_k q8_0 q8_0
68
+ blk.8.ffn_down.weight iq2_xxs iq2_xxs iq3_s iq3_s iq3_s q4_k q4_k q5_k q5_k q5_k q8_0
69
+ blk.8.ffn_gate.weight iq2_xxs iq2_s iq2_s iq2_s q3_k iq4_xs iq4_xs q5_k q5_k q6_k q8_0
70
+ blk.8.ffn_up.weight iq2_xxs iq2_s iq2_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
71
+ blk.8.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
72
+ blk.8.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
73
+ blk.8.ssm_out.weight iq2_s q3_k q3_k q4_k q4_k q5_k q5_k q5_k q5_k q5_k q8_0
74
+ blk.9.attn_gate.weight iq2_s iq2_s iq3_xxs iq3_s q4_k q4_k q4_k q5_k q5_k q8_0 q8_0
75
+ blk.9.attn_qkv.weight iq2_xxs iq2_xxs iq3_xxs iq3_xxs iq3_s q4_k q4_k q5_k q5_k q6_k q8_0
76
+ blk.9.ffn_down.weight iq2_xxs iq2_xs iq2_s iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q6_k q8_0
77
+ blk.9.ffn_gate.weight iq2_xxs iq2_xs iq3_xxs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q6_k q8_0
78
+ blk.9.ffn_up.weight iq2_xxs iq2_xxs iq2_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0
79
+ blk.9.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
80
+ blk.9.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
81
+ blk.9.ssm_out.weight iq2_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
82
+ blk.10.attn_gate.weight iq2_s iq2_s iq3_xxs iq3_xxs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0
83
+ blk.10.attn_qkv.weight iq2_xxs iq3_xxs iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q5_k q5_k q6_k q8_0
84
+ blk.10.ffn_down.weight iq2_xxs iq2_xs iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_nl q5_k q6_k q8_0
85
+ blk.10.ffn_gate.weight iq1_s iq2_xs iq2_xs iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q4_k q8_0 q8_0
86
+ blk.10.ffn_up.weight iq2_xxs iq2_xxs iq2_s iq2_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
87
+ blk.10.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
88
+ blk.10.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
89
+ blk.10.ssm_out.weight iq2_s q3_k q3_k q3_k iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
90
+ blk.11.attn_k.weight iq3_s iq3_s iq4_nl iq4_nl q5_k q6_k q6_k q6_k q8_0 q8_0 bf16
91
+ blk.11.attn_output.weight iq3_xxs iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q5_k q5_k q5_k q5_k q8_0
92
+ blk.11.attn_q.weight q2_k q2_k q2_k q2_k iq3_xxs iq4_xs q5_k q5_k q5_k q5_k bf16
93
+ blk.11.attn_v.weight q4_k q4_k q4_k q4_k q4_k q4_k q8_0 q8_0 q8_0 q8_0 bf16
94
+ blk.11.ffn_down.weight iq2_xxs iq2_xxs iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs iq4_xs q6_k q8_0
95
+ blk.11.ffn_gate.weight iq2_xxs iq2_xxs iq2_xs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q6_k q8_0
96
+ blk.11.ffn_up.weight iq2_xxs iq2_xxs iq2_s iq2_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
97
+ blk.12.attn_gate.weight iq2_xs iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0
98
+ blk.12.attn_qkv.weight iq2_xxs iq2_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q4_k q5_k q5_k q8_0
99
+ blk.12.ffn_down.weight iq2_xxs iq2_xxs iq2_s iq2_s iq3_s iq3_s iq4_xs q4_k q5_k q5_k q8_0
100
+ blk.12.ffn_gate.weight iq1_s iq2_xs iq2_xs iq2_xs iq2_xs q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
101
+ blk.12.ffn_up.weight iq1_s iq1_s iq2_xs iq2_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q8_0
102
+ blk.12.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
103
+ blk.12.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
104
+ blk.12.ssm_out.weight iq2_xs iq3_xxs q3_k q3_k q3_k q4_k q5_k q5_k q6_k q6_k q8_0
105
+ blk.13.attn_gate.weight iq2_xxs iq2_s iq2_s iq2_s iq3_s q4_k q4_k q4_k q5_k q6_k q8_0
106
+ blk.13.attn_qkv.weight iq2_xxs iq2_xxs iq3_s iq3_s q4_k q4_k q4_k q4_k q4_k q6_k q8_0
107
+ blk.13.ffn_down.weight iq1_s iq1_s iq2_s iq2_s q3_k q3_k q4_k q4_k q5_k q5_k q8_0
108
+ blk.13.ffn_gate.weight iq1_s iq1_m iq2_xs iq2_xs q3_k q3_k q3_k q3_k q5_k q5_k q8_0
109
+ blk.13.ffn_up.weight iq1_s iq1_s iq2_xs iq3_xxs q3_k q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
110
+ blk.13.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
111
+ blk.13.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
112
+ blk.13.ssm_out.weight iq2_xs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
113
+ blk.14.attn_gate.weight iq2_xxs iq2_xxs iq3_s iq3_s q4_k q4_k q4_k q4_k q5_k q8_0 q8_0
114
+ blk.14.attn_qkv.weight iq1_m q2_k q2_k iq3_s iq4_xs iq4_xs q4_k q4_k q5_k q6_k q8_0
115
+ blk.14.ffn_down.weight iq1_s iq1_m iq2_xxs iq2_xs iq3_xxs iq3_xxs iq3_s q4_k q4_k q5_k q8_0
116
+ blk.14.ffn_gate.weight iq1_s iq2_xxs iq2_xxs iq2_xxs iq2_s q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
117
+ blk.14.ffn_up.weight iq1_s iq1_s iq1_s iq2_s iq2_s q3_k q3_k iq4_xs iq4_xs q6_k q8_0
118
+ blk.14.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
119
+ blk.14.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
120
+ blk.14.ssm_out.weight iq3_xxs iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q4_k q4_k q6_k q8_0 q8_0
121
+ blk.15.attn_k.weight iq3_s iq3_s q4_k q4_k q4_k q6_k q6_k q6_k q6_k q6_k bf16
122
+ blk.15.attn_output.weight iq2_xxs iq2_xxs iq3_xxs q3_k iq4_xs iq4_xs iq4_xs iq4_xs q6_k q8_0 q8_0
123
+ blk.15.attn_q.weight iq2_xxs q2_k q2_k q2_k q2_k q4_k q4_k q4_k q4_k q5_k bf16
124
+ blk.15.attn_v.weight q4_k q4_k q4_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 bf16
125
+ blk.15.ffn_down.weight iq1_s iq1_m iq2_s iq2_s iq2_s iq3_s iq4_xs iq4_xs iq4_xs q8_0 q8_0
126
+ blk.15.ffn_gate.weight iq1_s iq1_s iq2_xxs iq2_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q8_0
127
+ blk.15.ffn_up.weight iq1_s iq2_xxs iq2_xxs iq2_s iq2_s iq3_s iq4_xs iq4_xs iq4_xs q8_0 q8_0
128
+ blk.16.attn_gate.weight iq2_xxs iq2_s iq3_s q4_k q4_k q4_k q4_k q5_k q5_k q5_k q8_0
129
+ blk.16.attn_qkv.weight q2_k iq3_s iq3_s iq3_s iq3_s q4_k q4_k q6_k q6_k q6_k q8_0
130
+ blk.16.ffn_down.weight iq2_xxs iq2_xs iq2_xs iq2_s iq3_s iq3_s q4_k q4_k q4_k q6_k q8_0
131
+ blk.16.ffn_gate.weight iq2_xxs iq2_xxs iq2_xxs iq2_s iq2_s q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
132
+ blk.16.ffn_up.weight iq1_s iq2_xxs iq2_xs iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q6_k q8_0
133
+ blk.16.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
134
+ blk.16.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
135
+ blk.16.ssm_out.weight iq2_s iq2_s q3_k iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
136
+ blk.17.attn_gate.weight iq2_xs iq2_xs iq2_xs iq4_xs iq4_xs iq4_xs iq4_xs iq4_xs iq4_xs q5_k q8_0
137
+ blk.17.attn_qkv.weight iq2_xs iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
138
+ blk.17.ffn_down.weight iq2_xxs iq2_xxs iq2_xxs iq3_xxs iq3_xxs iq3_s q4_k q4_k q4_k q5_k q8_0
139
+ blk.17.ffn_gate.weight iq1_s iq1_s iq2_xxs iq3_xxs iq3_xxs iq3_xxs iq4_nl iq4_nl iq4_nl q8_0 q8_0
140
+ blk.17.ffn_up.weight iq2_xxs iq2_xxs iq2_xxs iq2_s iq2_s q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
141
+ blk.17.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
142
+ blk.17.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
143
+ blk.17.ssm_out.weight iq2_xs q3_k q3_k q3_k iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
144
+ blk.18.attn_gate.weight iq2_xxs q2_k iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q8_0 q8_0
145
+ blk.18.attn_qkv.weight iq2_xxs q2_k q2_k iq4_xs iq4_xs q4_k q4_k q4_k q6_k q6_k q8_0
146
+ blk.18.ffn_down.weight iq2_xs iq2_s iq2_s iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q8_0 q8_0
147
+ blk.18.ffn_gate.weight iq1_s iq2_xxs iq2_s iq3_xxs iq3_xxs q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
148
+ blk.18.ffn_up.weight iq2_xxs iq2_xs iq2_xs iq3_xxs iq3_xxs q3_k iq4_xs iq4_xs q5_k q6_k q8_0
149
+ blk.18.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
150
+ blk.18.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
151
+ blk.18.ssm_out.weight iq2_xs iq3_xxs q3_k iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0
152
+ blk.19.attn_k.weight q3_k q4_k q4_k q4_k q4_k q5_k q6_k q8_0 q8_0 q8_0 bf16
153
+ blk.19.attn_output.weight iq2_xs iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q6_k q6_k q8_0 q8_0
154
+ blk.19.attn_q.weight iq2_xxs iq2_xxs iq2_xxs iq3_xxs iq3_xxs q4_k q4_k q4_k q5_k q6_k bf16
155
+ blk.19.attn_v.weight q4_k q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0 bf16
156
+ blk.19.ffn_down.weight iq2_xxs iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
157
+ blk.19.ffn_gate.weight iq1_s iq2_xs iq3_xxs iq3_xxs iq3_xxs q3_k iq4_xs iq4_xs iq4_xs q6_k q8_0
158
+ blk.19.ffn_up.weight iq2_xxs iq2_xxs iq2_xxs iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q8_0 q8_0
159
+ blk.20.attn_gate.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
160
+ blk.20.attn_qkv.weight iq2_s iq3_s iq3_s iq4_xs iq4_xs q4_k q4_k q5_k q6_k q6_k q8_0
161
+ blk.20.ffn_down.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q8_0
162
+ blk.20.ffn_gate.weight iq2_xxs iq3_xxs iq3_s iq3_s iq3_s iq4_xs q4_k q4_k q5_k q6_k q8_0
163
+ blk.20.ffn_up.weight iq2_xs iq3_s iq3_s iq3_s iq3_s iq4_xs q5_k q5_k q5_k q6_k q8_0
164
+ blk.20.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
165
+ blk.20.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
166
+ blk.20.ssm_out.weight iq3_s iq3_s q4_k q4_k q4_k q5_k q5_k q5_k q6_k q8_0 q8_0
167
+ blk.21.attn_gate.weight iq3_s iq3_s iq3_s q4_k q4_k q4_k q5_k q5_k q8_0 q8_0 q8_0
168
+ blk.21.attn_qkv.weight iq2_s iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_nl iq4_nl q6_k q6_k q6_k q8_0
169
+ blk.21.ffn_down.weight iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q4_k q5_k q5_k q5_k q8_0 q8_0
170
+ blk.21.ffn_gate.weight iq2_s iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q5_k q5_k q8_0 q8_0
171
+ blk.21.ffn_up.weight iq2_xxs q3_k q3_k q3_k iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
172
+ blk.21.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
173
+ blk.21.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
174
+ blk.21.ssm_out.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q6_k q8_0
175
+ blk.22.attn_gate.weight iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0
176
+ blk.22.attn_qkv.weight iq2_s iq3_xxs iq3_s iq3_s iq3_s q4_k q4_k q4_k q4_k q8_0 q8_0
177
+ blk.22.ffn_down.weight iq2_s iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
178
+ blk.22.ffn_gate.weight iq2_xs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q8_0
179
+ blk.22.ffn_up.weight iq2_s iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
180
+ blk.22.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
181
+ blk.22.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
182
+ blk.22.ssm_out.weight iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q6_k q6_k q6_k q6_k q8_0
183
+ blk.23.attn_k.weight iq3_s iq3_s iq4_xs q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 bf16
184
+ blk.23.attn_output.weight iq3_s iq3_s q4_k q4_k q4_k q4_k q6_k q6_k q6_k q6_k q8_0
185
+ blk.23.attn_q.weight q2_k iq3_s iq3_s iq3_s q4_k q4_k q4_k q5_k q6_k q6_k bf16
186
+ blk.23.attn_v.weight q4_k q5_k q5_k q5_k q6_k q6_k q8_0 q8_0 q8_0 q8_0 bf16
187
+ blk.23.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q4_k q5_k q6_k q6_k q6_k q8_0
188
+ blk.23.ffn_gate.weight iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
189
+ blk.23.ffn_up.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
190
+ blk.24.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs q4_k q4_k q5_k q5_k q8_0 q8_0 q8_0
191
+ blk.24.attn_qkv.weight iq3_xxs iq3_xxs iq3_xxs iq3_s q4_k q4_k q5_k q5_k q5_k q5_k q8_0
192
+ blk.24.ffn_down.weight q3_k q3_k iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
193
+ blk.24.ffn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
194
+ blk.24.ffn_up.weight iq3_xxs iq3_xxs q3_k q4_k q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
195
+ blk.24.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
196
+ blk.24.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
197
+ blk.24.ssm_out.weight iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q4_k q6_k q6_k q6_k q6_k q8_0
198
+ blk.25.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q8_0 q8_0
199
+ blk.25.attn_qkv.weight iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_nl q5_k q5_k q5_k q8_0 q8_0
200
+ blk.25.ffn_down.weight iq3_s iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q5_k q6_k q8_0
201
+ blk.25.ffn_gate.weight iq2_s iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
202
+ blk.25.ffn_up.weight q3_k q3_k q3_k iq4_xs q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
203
+ blk.25.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
204
+ blk.25.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
205
+ blk.25.ssm_out.weight q3_k iq4_xs iq4_xs iq4_xs iq4_xs iq4_xs q6_k q6_k q6_k q6_k q8_0
206
+ blk.26.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q8_0
207
+ blk.26.attn_qkv.weight iq3_s iq3_s iq3_s iq3_s q4_k q4_k q5_k q5_k q5_k q5_k q8_0
208
+ blk.26.ffn_down.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
209
+ blk.26.ffn_gate.weight iq2_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
210
+ blk.26.ffn_up.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
211
+ blk.26.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
212
+ blk.26.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
213
+ blk.26.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q5_k q5_k q8_0
214
+ blk.27.attn_k.weight iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0 q8_0 bf16
215
+ blk.27.attn_output.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q6_k q8_0
216
+ blk.27.attn_q.weight iq2_xs iq3_s iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q6_k q6_k q6_k bf16
217
+ blk.27.attn_v.weight q4_k q5_k q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 bf16
218
+ blk.27.ffn_down.weight iq3_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
219
+ blk.27.ffn_gate.weight iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_nl q5_k q5_k q8_0 q8_0
220
+ blk.27.ffn_up.weight iq2_s iq3_s iq3_s iq3_s iq4_xs iq4_nl q5_k q5_k q5_k q8_0 q8_0
221
+ blk.28.attn_gate.weight iq2_s iq3_s iq3_s q4_k q4_k q4_k q6_k q6_k q6_k q6_k q8_0
222
+ blk.28.attn_qkv.weight iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
223
+ blk.28.ffn_down.weight iq2_s iq3_xxs q3_k q3_k iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
224
+ blk.28.ffn_gate.weight iq2_xs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
225
+ blk.28.ffn_up.weight iq2_xs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs iq4_nl q5_k q5_k q6_k q8_0
226
+ blk.28.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
227
+ blk.28.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
228
+ blk.28.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0
229
+ blk.29.attn_gate.weight iq2_s iq3_s iq3_s q4_k q4_k q4_k q4_k q4_k q6_k q8_0 q8_0
230
+ blk.29.attn_qkv.weight iq2_s iq3_s iq3_s iq4_xs iq4_xs q4_k q4_k q5_k q5_k q8_0 q8_0
231
+ blk.29.ffn_down.weight iq2_s iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0 q8_0
232
+ blk.29.ffn_gate.weight iq2_xxs iq3_xxs iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q8_0 q8_0
233
+ blk.29.ffn_up.weight iq2_s iq2_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q8_0 q8_0
234
+ blk.29.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
235
+ blk.29.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
236
+ blk.29.ssm_out.weight iq3_xxs iq3_xxs iq4_xs q4_k q4_k q4_k q5_k q6_k q6_k q6_k q8_0
237
+ blk.30.attn_gate.weight q2_k iq3_s iq3_s iq3_s q4_k q4_k q4_k q5_k q5_k q8_0 q8_0
238
+ blk.30.attn_qkv.weight iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
239
+ blk.30.ffn_down.weight iq2_s iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
240
+ blk.30.ffn_gate.weight iq2_xs iq2_xs iq3_xxs iq3_s iq3_s iq4_xs q4_k q5_k q5_k q6_k q8_0
241
+ blk.30.ffn_up.weight iq2_s iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
242
+ blk.30.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
243
+ blk.30.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
244
+ blk.30.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
245
+ blk.31.attn_k.weight q4_k q4_k q4_k q4_k q4_k q5_k q8_0 q8_0 q8_0 q8_0 bf16
246
+ blk.31.attn_output.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
247
+ blk.31.attn_q.weight q2_k iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 bf16
248
+ blk.31.attn_v.weight q4_k q5_k q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 bf16
249
+ blk.31.ffn_down.weight iq2_s iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
250
+ blk.31.ffn_gate.weight iq2_xxs iq2_s iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q8_0
251
+ blk.31.ffn_up.weight iq2_s iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q8_0
252
+ blk.32.attn_gate.weight iq2_xs iq3_xxs iq3_xxs iq3_xxs iq4_xs q4_k q5_k q5_k q6_k q6_k q8_0
253
+ blk.32.attn_qkv.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
254
+ blk.32.ffn_down.weight iq2_s iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
255
+ blk.32.ffn_gate.weight iq2_xxs iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q6_k q8_0
256
+ blk.32.ffn_up.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
257
+ blk.32.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
258
+ blk.32.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
259
+ blk.32.ssm_out.weight iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q5_k q5_k q8_0 q8_0
260
+ blk.33.attn_gate.weight iq2_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q4_k q5_k q5_k q6_k q8_0
261
+ blk.33.attn_qkv.weight iq3_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
262
+ blk.33.ffn_down.weight iq2_s iq2_s iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q8_0
263
+ blk.33.ffn_gate.weight iq2_xxs iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q6_k q6_k q8_0
264
+ blk.33.ffn_up.weight iq2_s iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q8_0 q8_0
265
+ blk.33.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
266
+ blk.33.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
267
+ blk.33.ssm_out.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
268
+ blk.34.attn_gate.weight iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs q4_k q5_k q5_k q5_k q6_k q8_0
269
+ blk.34.attn_qkv.weight iq3_xxs iq3_s iq3_s iq3_s iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
270
+ blk.34.ffn_down.weight iq2_s iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
271
+ blk.34.ffn_gate.weight iq2_xs iq3_xxs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q6_k q8_0
272
+ blk.34.ffn_up.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
273
+ blk.34.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
274
+ blk.34.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
275
+ blk.34.ssm_out.weight iq3_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
276
+ blk.35.attn_k.weight q4_k q4_k q4_k q4_k q4_k q4_k q6_k q6_k q8_0 q8_0 bf16
277
+ blk.35.attn_output.weight iq4_xs iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
278
+ blk.35.attn_q.weight iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 bf16
279
+ blk.35.attn_v.weight q4_k q5_k q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 bf16
280
+ blk.35.ffn_down.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
281
+ blk.35.ffn_gate.weight iq2_s iq2_s iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q6_k q8_0
282
+ blk.35.ffn_up.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
283
+ blk.36.attn_gate.weight iq2_xs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
284
+ blk.36.attn_qkv.weight iq2_xs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
285
+ blk.36.ffn_down.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
286
+ blk.36.ffn_gate.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
287
+ blk.36.ffn_up.weight iq2_s iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
288
+ blk.36.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
289
+ blk.36.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
290
+ blk.36.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
291
+ blk.37.attn_gate.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
292
+ blk.37.attn_qkv.weight iq2_xs iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
293
+ blk.37.ffn_down.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
294
+ blk.37.ffn_gate.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q5_k q8_0 q8_0
295
+ blk.37.ffn_up.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
296
+ blk.37.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
297
+ blk.37.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
298
+ blk.37.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
299
+ blk.38.attn_gate.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
300
+ blk.38.attn_qkv.weight iq2_s iq2_s iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q6_k q8_0
301
+ blk.38.ffn_down.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q5_k q6_k q8_0
302
+ blk.38.ffn_gate.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
303
+ blk.38.ffn_up.weight iq3_xxs iq3_s iq3_s iq3_s iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
304
+ blk.38.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
305
+ blk.38.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
306
+ blk.38.ssm_out.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
307
+ blk.39.attn_k.weight iq3_s iq3_s q4_k q4_k q4_k q4_k q6_k q6_k q6_k q6_k bf16
308
+ blk.39.attn_output.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
309
+ blk.39.attn_q.weight q2_k iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k bf16
310
+ blk.39.attn_v.weight q4_k q4_k q5_k q5_k q5_k q5_k q6_k q6_k q6_k q8_0 bf16
311
+ blk.39.ffn_down.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
312
+ blk.39.ffn_gate.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
313
+ blk.39.ffn_up.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
314
+ blk.40.attn_gate.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
315
+ blk.40.attn_qkv.weight iq3_xxs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
316
+ blk.40.ffn_down.weight iq3_s iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
317
+ blk.40.ffn_gate.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q5_k q6_k q8_0
318
+ blk.40.ffn_up.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
319
+ blk.40.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
320
+ blk.40.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
321
+ blk.40.ssm_out.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
322
+ blk.41.attn_gate.weight iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
323
+ blk.41.attn_qkv.weight iq2_s iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
324
+ blk.41.ffn_down.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
325
+ blk.41.ffn_gate.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
326
+ blk.41.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
327
+ blk.41.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
328
+ blk.41.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
329
+ blk.41.ssm_out.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
330
+ blk.42.attn_gate.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
331
+ blk.42.attn_qkv.weight iq3_xxs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
332
+ blk.42.ffn_down.weight iq3_xxs iq3_xxs q3_k iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
333
+ blk.42.ffn_gate.weight iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
334
+ blk.42.ffn_up.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
335
+ blk.42.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
336
+ blk.42.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
337
+ blk.42.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
338
+ blk.43.attn_k.weight iq4_xs q4_k q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 bf16
339
+ blk.43.attn_output.weight iq3_s q4_k q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
340
+ blk.43.attn_q.weight q2_k iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 bf16
341
+ blk.43.attn_v.weight q4_k q4_k q5_k q5_k q5_k q5_k q6_k q8_0 q8_0 q8_0 bf16
342
+ blk.43.ffn_down.weight iq3_xxs iq3_xxs q3_k iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
343
+ blk.43.ffn_gate.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
344
+ blk.43.ffn_up.weight iq3_xxs iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
345
+ blk.44.attn_gate.weight iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q4_k q6_k q6_k q8_0
346
+ blk.44.attn_qkv.weight iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q5_k q8_0 q8_0
347
+ blk.44.ffn_down.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
348
+ blk.44.ffn_gate.weight iq2_s iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
349
+ blk.44.ffn_up.weight iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
350
+ blk.44.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
351
+ blk.44.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
352
+ blk.44.ssm_out.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
353
+ blk.45.attn_gate.weight iq2_xxs iq2_s iq3_xxs iq3_xxs q3_k iq4_xs iq4_xs iq4_xs iq4_xs q6_k q8_0
354
+ blk.45.attn_qkv.weight q2_k iq3_s iq3_s iq3_s iq3_s q4_k q4_k q5_k q5_k q6_k q8_0
355
+ blk.45.ffn_down.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
356
+ blk.45.ffn_gate.weight iq2_xs iq3_xxs iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q8_0
357
+ blk.45.ffn_up.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
358
+ blk.45.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
359
+ blk.45.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
360
+ blk.45.ssm_out.weight iq3_xxs iq3_xxs iq3_s iq3_s q4_k q4_k q5_k q5_k q5_k q8_0 q8_0
361
+ blk.46.attn_gate.weight q2_k iq3_xxs iq3_xxs iq3_s iq4_xs q4_k q4_k q5_k q5_k q8_0 q8_0
362
+ blk.46.attn_qkv.weight q2_k iq3_s iq3_s iq3_s iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
363
+ blk.46.ffn_down.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
364
+ blk.46.ffn_gate.weight iq2_s iq2_s iq3_xxs iq3_s iq3_s iq4_xs q4_k q5_k q5_k q8_0 q8_0
365
+ blk.46.ffn_up.weight iq2_xs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
366
+ blk.46.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
367
+ blk.46.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
368
+ blk.46.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
369
+ blk.47.attn_k.weight iq3_s q4_k q4_k q4_k q5_k q5_k q6_k q6_k q8_0 q8_0 bf16
370
+ blk.47.attn_output.weight iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q8_0 q8_0
371
+ blk.47.attn_q.weight iq2_xs iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k bf16
372
+ blk.47.attn_v.weight q4_k q4_k q5_k q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 bf16
373
+ blk.47.ffn_down.weight iq2_xs iq3_xxs q3_k q3_k iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
374
+ blk.47.ffn_gate.weight iq2_xxs iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q6_k q8_0
375
+ blk.47.ffn_up.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0
376
+ blk.48.attn_gate.weight iq2_s iq2_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
377
+ blk.48.attn_qkv.weight iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
378
+ blk.48.ffn_down.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
379
+ blk.48.ffn_gate.weight iq2_xs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
380
+ blk.48.ffn_up.weight iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
381
+ blk.48.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
382
+ blk.48.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
383
+ blk.48.ssm_out.weight iq3_s iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q5_k q8_0 q8_0
384
+ blk.49.attn_gate.weight q2_k q2_k iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs iq4_xs iq4_xs q8_0 q8_0
385
+ blk.49.attn_qkv.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
386
+ blk.49.ffn_down.weight iq2_s iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
387
+ blk.49.ffn_gate.weight q2_k iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0
388
+ blk.49.ffn_up.weight iq3_xxs iq3_s iq3_s iq3_s iq4_xs q4_k q5_k q5_k q5_k q6_k q8_0
389
+ blk.49.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
390
+ blk.49.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
391
+ blk.49.ssm_out.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
392
+ blk.50.attn_gate.weight iq3_xxs iq3_xxs q3_k iq4_xs q4_k q4_k q4_k q5_k q5_k q8_0 q8_0
393
+ blk.50.attn_qkv.weight iq3_xxs iq3_xxs iq3_s iq4_xs q4_k q4_k q5_k q5_k q5_k q8_0 q8_0
394
+ blk.50.ffn_down.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
395
+ blk.50.ffn_gate.weight iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
396
+ blk.50.ffn_up.weight iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
397
+ blk.50.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
398
+ blk.50.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
399
+ blk.50.ssm_out.weight q3_k iq4_xs iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q8_0 q8_0
400
+ blk.51.attn_k.weight q4_k q4_k q4_k q5_k q5_k q5_k q6_k q6_k q6_k q8_0 bf16
401
+ blk.51.attn_output.weight iq4_xs iq4_xs iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q8_0 q8_0
402
+ blk.51.attn_q.weight iq3_xxs iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 bf16
403
+ blk.51.attn_v.weight q5_k q5_k q5_k q5_k q5_k q8_0 q8_0 q8_0 q8_0 q8_0 bf16
404
+ blk.51.ffn_down.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
405
+ blk.51.ffn_gate.weight iq2_s iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
406
+ blk.51.ffn_up.weight iq3_xxs iq3_s iq3_s iq4_xs q4_k q4_k q5_k q5_k q6_k q8_0 q8_0
407
+ blk.52.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
408
+ blk.52.attn_qkv.weight iq3_xxs iq3_xxs iq3_xxs q4_k q4_k q4_k q4_k q5_k q6_k q8_0 q8_0
409
+ blk.52.ffn_down.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
410
+ blk.52.ffn_gate.weight iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
411
+ blk.52.ffn_up.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
412
+ blk.52.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
413
+ blk.52.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
414
+ blk.52.ssm_out.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q8_0 q8_0
415
+ blk.53.attn_gate.weight iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
416
+ blk.53.attn_qkv.weight iq3_xxs iq3_xxs iq3_s iq3_s iq3_s iq4_xs q4_k q5_k q5_k q6_k q8_0
417
+ blk.53.ffn_down.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
418
+ blk.53.ffn_gate.weight iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
419
+ blk.53.ffn_up.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
420
+ blk.53.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
421
+ blk.53.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
422
+ blk.53.ssm_out.weight iq3_s iq4_xs iq4_xs q4_k q4_k q5_k q5_k q6_k q6_k q8_0 q8_0
423
+ blk.54.attn_gate.weight iq3_xxs iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
424
+ blk.54.attn_qkv.weight iq2_s iq3_xxs iq3_s iq3_s iq3_s iq4_xs q4_k q5_k q5_k q6_k q8_0
425
+ blk.54.ffn_down.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
426
+ blk.54.ffn_gate.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0
427
+ blk.54.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
428
+ blk.54.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
429
+ blk.54.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
430
+ blk.54.ssm_out.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
431
+ blk.55.attn_k.weight iq3_s q4_k q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 bf16
432
+ blk.55.attn_output.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
433
+ blk.55.attn_q.weight iq2_s iq3_xxs iq3_xxs iq3_s iq3_s iq4_xs q4_k q5_k q5_k q6_k bf16
434
+ blk.55.attn_v.weight q4_k q4_k q5_k q5_k q5_k q6_k q8_0 q8_0 q8_0 q8_0 bf16
435
+ blk.55.ffn_down.weight iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0 q8_0
436
+ blk.55.ffn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
437
+ blk.55.ffn_up.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
438
+ blk.56.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
439
+ blk.56.attn_qkv.weight iq3_xxs iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
440
+ blk.56.ffn_down.weight iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
441
+ blk.56.ffn_gate.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
442
+ blk.56.ffn_up.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
443
+ blk.56.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
444
+ blk.56.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
445
+ blk.56.ssm_out.weight iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
446
+ blk.57.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
447
+ blk.57.attn_qkv.weight iq2_s iq3_xxs iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0
448
+ blk.57.ffn_down.weight iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
449
+ blk.57.ffn_gate.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
450
+ blk.57.ffn_up.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
451
+ blk.57.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
452
+ blk.57.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
453
+ blk.57.ssm_out.weight iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
454
+ blk.58.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
455
+ blk.58.attn_qkv.weight iq3_xxs iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
456
+ blk.58.ffn_down.weight iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
457
+ blk.58.ffn_gate.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
458
+ blk.58.ffn_up.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0 q8_0
459
+ blk.58.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
460
+ blk.58.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
461
+ blk.58.ssm_out.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q6_k q6_k q6_k q8_0 q8_0
462
+ blk.59.attn_k.weight iq4_xs q4_k q4_k q5_k q5_k q5_k q6_k q8_0 q8_0 q8_0 bf16
463
+ blk.59.attn_output.weight iq3_s iq4_xs iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q8_0 q8_0
464
+ blk.59.attn_q.weight iq3_xxs iq3_xxs iq3_xxs iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 bf16
465
+ blk.59.attn_v.weight q4_k q5_k q5_k q5_k q5_k q6_k q8_0 q8_0 q8_0 q8_0 bf16
466
+ blk.59.ffn_down.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
467
+ blk.59.ffn_gate.weight iq4_xs iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
468
+ blk.59.ffn_up.weight iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
469
+ blk.60.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
470
+ blk.60.attn_qkv.weight iq3_s iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
471
+ blk.60.ffn_down.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
472
+ blk.60.ffn_gate.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
473
+ blk.60.ffn_up.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0 q8_0
474
+ blk.60.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
475
+ blk.60.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
476
+ blk.60.ssm_out.weight iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0 q8_0
477
+ blk.61.attn_gate.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
478
+ blk.61.attn_qkv.weight iq3_xxs iq3_s iq3_s iq3_s iq4_xs iq4_xs q5_k q5_k q5_k q8_0 q8_0
479
+ blk.61.ffn_down.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
480
+ blk.61.ffn_gate.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
481
+ blk.61.ffn_up.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
482
+ blk.61.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
483
+ blk.61.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
484
+ blk.61.ssm_out.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
485
+ blk.62.attn_gate.weight iq3_xxs iq3_xxs iq4_xs iq4_xs iq4_xs q4_k q5_k q5_k q6_k q8_0 q8_0
486
+ blk.62.attn_qkv.weight iq3_xxs iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 q8_0
487
+ blk.62.ffn_down.weight iq3_s iq3_s iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q8_0 q8_0
488
+ blk.62.ffn_gate.weight iq3_s iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q6_k q8_0 q8_0
489
+ blk.62.ffn_up.weight iq3_s iq4_xs iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q8_0 q8_0
490
+ blk.62.ssm_alpha.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
491
+ blk.62.ssm_beta.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0
492
+ blk.62.ssm_out.weight iq4_xs iq4_xs iq4_xs q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
493
+ blk.63.attn_k.weight q4_k q4_k q5_k q5_k q5_k q6_k q6_k q8_0 q8_0 q8_0 bf16
494
+ blk.63.attn_output.weight iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q8_0 q8_0 q8_0
495
+ blk.63.attn_q.weight iq3_xxs iq4_xs iq4_xs iq4_xs iq4_xs iq4_xs q5_k q5_k q6_k q8_0 bf16
496
+ blk.63.attn_v.weight q5_k q5_k q6_k q6_k q6_k q6_k q8_0 q8_0 q8_0 q8_0 bf16
497
+ blk.63.ffn_down.weight iq4_xs q4_k q4_k q4_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
498
+ blk.63.ffn_gate.weight iq4_xs iq4_xs iq4_xs q5_k q5_k q5_k q6_k q6_k q6_k q8_0 q8_0
499
+ blk.63.ffn_up.weight iq4_xs q4_k q5_k q5_k q5_k q6_k q6_k q6_k q6_k q8_0 q8_0
500
+ blk.64.attn_k.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 bf16
501
+ blk.64.attn_output.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q8_0
502
+ blk.64.attn_q.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k bf16
503
+ blk.64.attn_v.weight q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 q8_0 bf16
504
+ blk.64.ffn_down.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q8_0
505
+ blk.64.ffn_gate.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q8_0
506
+ blk.64.ffn_up.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q8_0
507
+ blk.64.nextn.eh_proj.weight q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k q6_k bf16