oshiteku commited on
Commit
6e87714
·
verified ·
1 Parent(s): 64cc7ea

Release Sanshoku-Qwen3.8-3.6-27B-blend GGUF (Q2_0, PQ2_0, PTQ1_0, mmproj)

Browse files
.gitattributes CHANGED
@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Sanshoku-Qwen3.8-3.6-27B-blend-PQ2_0.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Sanshoku-Qwen3.8-3.6-27B-blend-PTQ1_0.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Sanshoku-Qwen3.8-3.6-27B-blend-Q2_0.gguf filter=lfs diff=lfs merge=lfs -text
39
+ assets/swe-time.png filter=lfs diff=lfs merge=lfs -text
40
+ mmproj-F16.gguf filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright 2026 Alibaba Cloud
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
README.md CHANGED
@@ -1,3 +1,211 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: llama.cpp
4
+ pipeline_tag: image-text-to-text
5
+ language:
6
+ - en
7
+ - ja
8
+ tags:
9
+ - ternary
10
+ - 2-bit
11
+ - gguf
12
+ - llama-cpp
13
+ base_model:
14
+ - Qwen/Qwen3.6-27B
15
+ - Qwen/Qwen3.8-27B
16
  ---
17
+
18
+ # 🍡 Sanshoku-Qwen3.8-3.6-27B-blend-GGUF
19
+
20
+ [English](#english) | [日本語](#日本語)
21
+
22
+ <a id="english"></a>
23
+
24
+ A ternary model distilled from a merge of [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) and [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B). The embedding, the output head, the MTP block and the linear weights take values in {−1, 0, +1}; attention k/v and the linear-attention alpha/beta projections (191M parameters, about 0.7%) stay in F16. The model was trained with QAT against the merged model.
25
+
26
+ It thinks less per turn than Ternary-Bonsai-2-27B: on coding-agent tasks it reaches about the same score in about 60% of the time.
27
+ This is an experimental model and trails Ternary-Bonsai-2-27B on some benchmarks.
28
+
29
+ ![](assets/swe-time.png)
30
+
31
+ ## Files
32
+
33
+ All three GGUFs contain the same weights in different packings.
34
+
35
+ | File | Size | Runtime |
36
+ |---|---:|---|
37
+ | `Sanshoku-Qwen3.8-3.6-27B-blend-Q2_0.gguf` | 8.03 GB | upstream llama.cpp |
38
+ | `Sanshoku-Qwen3.8-3.6-27B-blend-PQ2_0.gguf` | 7.61 GB | [PrismML fork of llama.cpp](https://github.com/PrismML-Eng/llama.cpp) |
39
+ | `Sanshoku-Qwen3.8-3.6-27B-blend-PTQ1_0.gguf` | 6.34 GB | PrismML fork of llama.cpp |
40
+ | `mmproj-F16.gguf` | 0.93 GB | vision encoder (merge of both models' encoders, FP16) |
41
+
42
+ ## Quick start
43
+
44
+ ```bash
45
+ llama-server -hf oshiteku/Sanshoku-Qwen3.8-3.6-27B-blend-GGUF -hff Sanshoku-Qwen3.8-3.6-27B-blend-Q2_0.gguf \
46
+ -ngl 99 -c 32768 -fa on --jinja \
47
+ --spec-type draft-mtp --spec-draft-n-max 2 \
48
+ --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0
49
+ ```
50
+
51
+ - Pick the file with `-hff` (the tag `:Q2_0` can match the PQ2_0 file)
52
+ - Tested with llama.cpp `f2cc728` and the PrismML fork `bdc23b5`
53
+
54
+ ## Speed
55
+
56
+ Measured on an RTX 4000 Ada (20 GB). Prefill and decode are `llama-bench` pp512 and tg128. Decode + MTP is the average over four chat requests (256 output tokens, 2 draft tokens).
57
+
58
+ | Format | Runtime | Prefill (tok/s) ↑ | Decode (tok/s) ↑ | Decode + MTP (tok/s) ↑ |
59
+ |---|---|---:|---:|---:|
60
+ | Q2_0 | [upstream](https://github.com/ggml-org/llama.cpp/tree/f2cc7282ce0042641df55e0e563b75e824e2b647) | 1117 | 35.4 | 56.8 |
61
+ | PQ2_0 | [fork](https://github.com/PrismML-Eng/llama.cpp/tree/bdc23b56b4458b9f1655aec5287f3ab56ee8daaa) | 1027 | 37.8 | 54.9 |
62
+ | PTQ1_0 | [fork](https://github.com/PrismML-Eng/llama.cpp/tree/bdc23b56b4458b9f1655aec5287f3ab56ee8daaa) | 1002 | 40.4 | 39.5 |
63
+ | [Qwen3.8-27B UD-Q4_K_M](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) (16.5 GB) | [upstream](https://github.com/ggml-org/llama.cpp/tree/f2cc7282ce0042641df55e0e563b75e824e2b647) | 985 | 19.3 | 34.8 |
64
+
65
+ ## Evaluation
66
+
67
+ Measured by us with one harness for all three models: [swallow-evaluation-instruct](https://github.com/swallow-llm/swallow-evaluation-instruct) (lighteval) and EvalScope, served by vLLM. Reasoning effort `low`, T 0.6 / top-p 0.95 / top-k 20, thinking on. ↑ higher is better, ↓ lower is better.
68
+
69
+ | | Benchmark | Sanshoku-Qwen3.8-3.6-27B-blend | [Ternary-Bonsai-2-27B](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf) | Qwen3.8-27B BF16 |
70
+ |---|---|---:|---:|---:|
71
+ | | Size ↓ | 7.6 GB | 7.2 GB | 54 GB |
72
+ | Agent | SWE-bench Verified (500) ¹ ↑ | 58.0 | 60.8 ² | 80.6 ² |
73
+ | | SWE-bench Verified Mini (50) ³ ↑ | 58 | 60 | – |
74
+ | | BFCL v3 ↑ | 75.0 | 73.9 | 76.0 |
75
+ | English | MATH-500 ↑ | 89.0 | 98.6 | 99.0 |
76
+ | | AIME 2024+2025 (pass@1, 4 samples) ↑ | 35.0 | 87.5 | 82.9 |
77
+ | | GPQA Diamond ↑ | 54.0 | 80.8 | 84.8 |
78
+ | | MMLU-Pro (1,001 sampled) ↑ | 71.0 | 80.4 | 82.6 |
79
+ | | LiveCodeBench v6 ↑ | 38.3 | 47.6 | 55.3 |
80
+ | | HumanEval+ ↑ | 86.7 | 92.7 | 93.4 |
81
+ | | IFEval (prompt strict) ↑ | 80.0 | 90.8 | 87.4 |
82
+ | | IFBench ↑ | 36.7 | 63.0 | 50.3 |
83
+ | Japanese | JMMLU (1,060 sampled) ↑ | 80.7 | 86.3 | 89.7 |
84
+ | | MMLU-ProX ja (1,001 sampled) ↑ | 63.6 | 75.5 | 79.6 |
85
+ | | JEMHopQA ↑ | 53.0 | 49.7 | 67.5 |
86
+ | | JGPQA Diamond ↑ | 52.0 | 77.8 | 83.8 |
87
+ | | MATH-100 ja ↑ | 81.8 | 96.0 | 96.0 |
88
+ | | JHumanEval ↑ | 85.2 | 92.9 | 92.9 |
89
+ | | M-IFEval ja ↑ | 53.5 | 62.8 | 65.7 |
90
+ | | WMT20 en-ja / ja-en (BLEU) ↑ | 22.5 / 21.9 | 23.1 / 22.5 | 22.8 / 23.4 |
91
+ | | MT-Bench ja ⁴ ↑ | 5.90 | 6.19 | 7.07 |
92
+
93
+ 1. [mini-SWE-agent](https://github.com/SWE-agent/mini-swe-agent) 2.4.6, 250 steps, 3 h limit per instance
94
+ 2. Published in the Ternary Bonsai 2 whitepaper (a different harness)
95
+ 3. [SWE-bench Verified Mini](https://huggingface.co/datasets/MariusHobbhahn/swe-bench-verified-mini) with mini-SWE-agent, 1 h limit per instance
96
+ 4. Swallow's Japanese MT-Bench questions and judge prompts, judged by Claude Sonnet 5.5 (1-10)
97
+
98
+ ## Training
99
+
100
+ - Each weight initialized to the ternary level nearest to the interval between its Qwen3.6 and Qwen3.8 values, in groups of 128 weights
101
+ - Trained with a straight-through estimator against the top-32 token distribution of the merged model
102
+ - About 1B tokens of training data
103
+
104
+ ## License
105
+
106
+ Apache-2.0 (see `LICENSE`). Derived from Qwen3.6-27B and Qwen3.8-27B.
107
+ An independent community model; not an official Qwen release.
108
+
109
+ ## Acknowledgements
110
+
111
+ Thanks to the Qwen team for the base models, JetBrains for the idea of merging the two
112
+ ([Qwen3.8-3.6-27B-blend](https://huggingface.co/JetBrains/Qwen3.8-3.6-27B-blend)), [PrismML](https://github.com/PrismML-Eng/llama.cpp)
113
+ for the PQ2_0 and PTQ1_0 kernels, and the [llama.cpp](https://github.com/ggml-org/llama.cpp) project.
114
+
115
+ ---
116
+
117
+ <a id="日本語"></a>
118
+
119
+ # 🍡 Sanshoku-Qwen3.8-3.6-27B-blend-GGUF
120
+
121
+ [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) と [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) をマージしたモデルから蒸留した3値モデルです。embedding、出力 head、MTP ブロック、線形層の重みが {−1, 0, +1} の値をとります。attention の k/v と linear attention の alpha/beta 射影 (1.91 億パラメータ、約 0.7%) は F16 のままです。マージしたモデルを教師として QAT で学習しました。
122
+
123
+ Ternary-Bonsai-2-27B より1ターンあたりの思考が短く、コーディングエージェントのタスクでほぼ同じ正解率に約 6 割の時間で到達します。
124
+ 実験的なモデルであり、Ternary-Bonsai-2-27B に比べ一部ベンチマークで劣ります。
125
+
126
+ ![](assets/swe-time.png)
127
+
128
+ ## ファイル
129
+
130
+ 3つの GGUF は同じ重みを異なる形式で格納しています。
131
+
132
+ | ファイル | サイズ | 実行環境 |
133
+ |---|---:|---|
134
+ | `Sanshoku-Qwen3.8-3.6-27B-blend-Q2_0.gguf` | 8.03 GB | 通常の llama.cpp |
135
+ | `Sanshoku-Qwen3.8-3.6-27B-blend-PQ2_0.gguf` | 7.61 GB | [PrismML 版 llama.cpp (fork)](https://github.com/PrismML-Eng/llama.cpp) |
136
+ | `Sanshoku-Qwen3.8-3.6-27B-blend-PTQ1_0.gguf` | 6.34 GB | PrismML 版 llama.cpp (fork) |
137
+ | `mmproj-F16.gguf` | 0.93 GB | 画像エンコーダ (両モデルのエンコーダをマージ、FP16) |
138
+
139
+ ## 使い方
140
+
141
+ ```bash
142
+ llama-server -hf oshiteku/Sanshoku-Qwen3.8-3.6-27B-blend-GGUF -hff Sanshoku-Qwen3.8-3.6-27B-blend-Q2_0.gguf \
143
+ -ngl 99 -c 32768 -fa on --jinja \
144
+ --spec-type draft-mtp --spec-draft-n-max 2 \
145
+ --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0
146
+ ```
147
+
148
+ - ファイルは `-hff` で指定してください (タグ `:Q2_0` では PQ2_0 が選ばれることがあります)
149
+ - llama.cpp `f2cc728` と PrismML 版 `bdc23b5` で動作確認済みです
150
+
151
+ ## 速度
152
+
153
+ RTX 4000 Ada (20 GB) で測定。prefill と decode は `llama-bench` の pp512 と tg128、decode + MTP は4つのチャットリクエストの平均です (出力 256 トークン、ドラフト 2 トークン)。
154
+
155
+ | 形式 | 実行環境 | prefill (tok/s) ↑ | decode (tok/s) ↑ | decode + MTP (tok/s) ↑ |
156
+ |---|---|---:|---:|---:|
157
+ | Q2_0 | [通常版](https://github.com/ggml-org/llama.cpp/tree/f2cc7282ce0042641df55e0e563b75e824e2b647) | 1117 | 35.4 | 56.8 |
158
+ | PQ2_0 | [fork](https://github.com/PrismML-Eng/llama.cpp/tree/bdc23b56b4458b9f1655aec5287f3ab56ee8daaa) | 1027 | 37.8 | 54.9 |
159
+ | PTQ1_0 | [fork](https://github.com/PrismML-Eng/llama.cpp/tree/bdc23b56b4458b9f1655aec5287f3ab56ee8daaa) | 1002 | 40.4 | 39.5 |
160
+ | [Qwen3.8-27B UD-Q4_K_M](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) (16.5 GB) | [通常版](https://github.com/ggml-org/llama.cpp/tree/f2cc7282ce0042641df55e0e563b75e824e2b647) | 985 | 19.3 | 34.8 |
161
+
162
+ ## 評価
163
+
164
+ 3つのモデルを同じ環境で測定しました。[swallow-evaluation-instruct](https://github.com/swallow-llm/swallow-evaluation-instruct) (lighteval) と EvalScope を使い、vLLM で推論しています。reasoning effort `low`、T 0.6 / top-p 0.95 / top-k 20、thinking ありです。↑ は高いほど良く、↓ は低いほど良い指標です。
165
+
166
+ | | ベンチマーク | Sanshoku-Qwen3.8-3.6-27B-blend | [Ternary-Bonsai-2-27B](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf) | Qwen3.8-27B BF16 |
167
+ |---|---|---:|---:|---:|
168
+ | | サイズ ↓ | 7.6 GB | 7.2 GB | 54 GB |
169
+ | エージェント | SWE-bench Verified (500) ¹ ↑ | 58.0 | 60.8 ² | 80.6 ² |
170
+ | | SWE-bench Verified Mini (50) ³ ↑ | 58 | 60 | – |
171
+ | | BFCL v3 ↑ | 75.0 | 73.9 | 76.0 |
172
+ | 英語 | MATH-500 ↑ | 89.0 | 98.6 | 99.0 |
173
+ | | AIME 2024+2025 (pass@1、4 サンプル) ↑ | 35.0 | 87.5 | 82.9 |
174
+ | | GPQA Diamond ↑ | 54.0 | 80.8 | 84.8 |
175
+ | | MMLU-Pro (1,001 問を抽出) ↑ | 71.0 | 80.4 | 82.6 |
176
+ | | LiveCodeBench v6 ↑ | 38.3 | 47.6 | 55.3 |
177
+ | | HumanEval+ ↑ | 86.7 | 92.7 | 93.4 |
178
+ | | IFEval (prompt strict) ↑ | 80.0 | 90.8 | 87.4 |
179
+ | | IFBench ↑ | 36.7 | 63.0 | 50.3 |
180
+ | 日本語 | JMMLU (1,060 問を抽出) ↑ | 80.7 | 86.3 | 89.7 |
181
+ | | MMLU-ProX ja (1,001 問を抽出) ↑ | 63.6 | 75.5 | 79.6 |
182
+ | | JEMHopQA ↑ | 53.0 | 49.7 | 67.5 |
183
+ | | JGPQA Diamond ↑ | 52.0 | 77.8 | 83.8 |
184
+ | | MATH-100 ja ↑ | 81.8 | 96.0 | 96.0 |
185
+ | | JHumanEval ↑ | 85.2 | 92.9 | 92.9 |
186
+ | | M-IFEval ja ↑ | 53.5 | 62.8 | 65.7 |
187
+ | | WMT20 en-ja / ja-en (BLEU) ↑ | 22.5 / 21.9 | 23.1 / 22.5 | 22.8 / 23.4 |
188
+ | | MT-Bench ja ⁴ ↑ | 5.90 | 6.19 | 7.07 |
189
+
190
+ 1. [mini-SWE-agent](https://github.com/SWE-agent/mini-swe-agent) 2.4.6、250 ステップ、1問あたり 3 時間制限
191
+ 2. Ternary Bonsai 2 の whitepaper の公表値 (評価環境が異なります)
192
+ 3. [SWE-bench Verified Mini](https://huggingface.co/datasets/MariusHobbhahn/swe-bench-verified-mini) を mini-SWE-agent で実行、1問あたり 1 時間制限
193
+ 4. Swallow の日本語 MT-Bench の質問と採点プロンプトを使い、Claude Sonnet 5.5 が採点 (1〜10)
194
+
195
+ ## 学習
196
+
197
+ - 各重みを Qwen3.6 と Qwen3.8 の値を両端とする区間に最も近い3値に 128 個ずつのグループで丸めて初期化
198
+ - マージしたモデルの上位 32 トークンの分布を目標に straight-through estimator で学習
199
+ - 学習データは約 10 億トークン
200
+
201
+ ## ライセンス
202
+
203
+ Apache-2.0 (`LICENSE` を参照)。Qwen3.6-27B と Qwen3.8-27B の派生物です。
204
+ 個人が作成した派生モデルで、Qwen の公式リリースではありません。
205
+
206
+ ## 謝辞
207
+
208
+ ベースモデルを公開している Qwen チーム、2つのモデルをマージするアイデアの元になった JetBrains
209
+ ([Qwen3.8-3.6-27B-blend](https://huggingface.co/JetBrains/Qwen3.8-3.6-27B-blend))、PQ2_0 と PTQ1_0 のカーネルを開発した
210
+ [PrismML](https://github.com/PrismML-Eng/llama.cpp)、[llama.cpp](https://github.com/ggml-org/llama.cpp) プロジェクトに
211
+ 感謝します。
Sanshoku-Qwen3.8-3.6-27B-blend-PQ2_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e21d7264ca33f43a2093dd06ae081594d5a58faeff00156a0de04d71aceb424
3
+ size 7609932288
Sanshoku-Qwen3.8-3.6-27B-blend-PTQ1_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:381cfa824ab6f1871d0535950822a26a86c8c5711ffcdbb1212958b8eaaba81a
3
+ size 6338370048
Sanshoku-Qwen3.8-3.6-27B-blend-Q2_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb82217946cc768d0128db77782d02ddca68e8bd0f23ab0bed92a80474f477e4
3
+ size 8033786368
assets/swe-time.png ADDED

Git LFS Details

  • SHA256: 8d5d1ccea18d500cad9487a23ece21438ee1c0646ee8eae63561b80f0139ce7e
  • Pointer size: 131 Bytes
  • Size of remote file: 158 kB
mmproj-F16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a5414649951a65c4366df551936aad450eb40af261155f85bfc6f6ffd46d1f9
3
+ size 927606944