thegovind commited on
Commit
b3e7840
·
verified ·
1 Parent(s): ae116cc

Shorter card; link the API docs

Browse files
Files changed (1) hide show
  1. README.md +75 -167
README.md CHANGED
@@ -14,36 +14,33 @@ tags:
14
  language:
15
  - en
16
  ---
17
-
18
  # blink-mimo-9b
19
 
20
- Send a text or JSON `state` and your questions: `choice` picks from up to 255 options, `noul` is yes/no,
21
- and `score` takes 2–10 ordered levels. Each question gets probabilities over its offered options from
22
- one forward pass, with no generated text. Long or large multi-question requests may use several batches.
23
 
24
- **Try it:** [Space demo](https://huggingface.co/spaces/thegovind/blink) — blink-4b and blink-mimo-9b run live ·
25
- [blink-4b](https://huggingface.co/thegovind/blink-4b) · [blink-27b](https://huggingface.co/thegovind/blink-27b) ·
26
- [blink-mimo-9b](https://huggingface.co/thegovind/blink-mimo-9b).
27
 
28
- [Source code](https://github.com/thegovind/blink) · [Docs](https://thegovind.github.io/blink/) · [blink-mimo-9b docs](https://thegovind.github.io/blink/models/blink-mimo-9b/).
29
 
30
- *Personal research release by thegovind, not an official product of any company. No affiliation with TypeSafe AI,
31
- Xiaomi, Alibaba Cloud or the Qwen team. Weights are for non-commercial research; see [Licence](#licence).*
32
 
33
- ![blink-mimo-9b one-pass readout diagram: a state and typed choice, noul and score questions are rendered as evidence, criterion and lettered options; questions are batched and each batch is one forward pass, giving next-token logits at the answer position, and an FP32 softmax over the offered letters gives one probability per option. No text is generated.](assets/blink-mimo-9b-readout.png)
 
 
 
 
 
34
 
35
  ## Results
36
 
37
  ### Decision Index 0.2 (local run)
38
 
39
- We ran the full Decision Index 0.2 suite ourselves with the official scoring kit at commit 19ad28e on 2026-09-25. This is a descriptive run, not a leaderboard submission or accepted result. The kit's scorer does not apply the leaderboard's penalty for rows an entrant trained on, so known training exposure stays in these scores and they cannot be ranked against the leaderboard.
40
 
41
  | Balanced skill | Balanced raw | Breadth skill | Without MMLU-Pro |
42
  |---:|---:|---:|---:|
43
  | 43.36 | 57.24 | 42.38 | 42.84 |
44
 
45
- - Our Blink fine-tune did not use MMLU-Pro or GPQA as direct data sources. Screening found 176 SuperGPQA training rows matching added-request text. SuperGPQA is a training source, not a benchmark in this suite. Public train splits used in training are listed under “Training overlap” below.
46
-
47
  <details><summary>Extra tables and method</summary>
48
 
49
  | Area | Number of benchmarks | Skill | Raw |
@@ -54,7 +51,7 @@ We ran the full Decision Index 0.2 suite ourselves with the official scoring kit
54
  | Tools & Automation | 6 | 57.0 | 65.2 |
55
  | Arts & Human Taste | 7 | 32.0 | 50.2 |
56
 
57
- The seven benchmarks added in 0.2.
58
 
59
  | Benchmark | Metric | Requests | Answered | Raw | Skill |
60
  |---|---|---:|---:|---:|---:|
@@ -66,32 +63,30 @@ The seven benchmarks added in 0.2.
66
  | When2Call MCQ | accuracy | 3,652 | 3,652 | 64.5 | 52.7 |
67
  | New Yorker caption matching | accuracy | 528 | 528 | 62.5 | 53.1 |
68
 
69
- - All 151,034 of 151,034 scoreable requests scored. Of 44 scored benchmarks, 40 count toward the index across five equal areas.
70
- - Requests shared with 0.1 reuse the model's 0.1 predictions. We ran the 30,419 added requests with the same frozen evaluation setup as 0.1, the evaluated adapter loaded on the MiMo base, which the published graft matched on JevBench's 231 public items (see Evaluation notes), at temperature 1.0.
71
- - Balanced skill is the headline index. “Without MMLU-Pro” drops MMLU-Pro, averages the other nine Knowledge benchmarks, and keeps five equal areas. It is a sensitivity check, not a score free of training effects.
72
- - These are point estimates, with no significance, calibration, or latency claims. Do not compare them with 0.1 numbers because the editions differ.
73
 
74
- Training-row text matches in the added requests.
75
 
76
  | Training stage | Rows in the stage | Rows matching added-request text | From MMLU-Pro | From SuperGPQA | Other |
77
  |---|---:|---:|---:|---:|---:|
78
  | MiMo | 123,195 | 180 | 0 | 176 | 4 |
79
 
80
- We screened for exact normalised strings of at least 30 characters shared by training rows and added requests, ignoring strings found in 20 or more requests as templates. Counts are training rows by stage and source, not unique test questions. A matching option or passage need not be the same question, and a clean screen cannot rule out semantic or pretraining overlap.
81
-
82
- We did not produce the planned calibration read or a score without the DI-S selection sample.
83
 
84
  </details>
85
 
86
  ### Decision Index 0.1 (archived edition)
87
 
88
- **Full suite: blink-mimo-9b 56.53 vs Jev 1.13.0 59.51.**
89
 
90
- **Training overlap:** Public train splits included ContractNLI, iSarcasmEval and VAST (the whole Language area), plus Amazon ESCI and Humicroedit. With Language set to Jev's score, this model's index would be 54.96 vs Jev's 59.51. That's arithmetic, not an ablation or a like-for-like comparison with models trained only on synthetic data.
91
 
92
  | Model | Size class | Decision Index 0.1 | Skill | Breadth |
93
  |---|---|---:|---:|---:|
94
- | **blink-mimo-9b** (this model) | 9B | 56.53 | 42.17 | 40.37 |
95
  | Jev 1.13.0 | closed | 59.51 | 46.26 | 44.79 |
96
  | Jevfire | 27B | 55.74 | 40.86 | 39.45 |
97
  | JoshuaSP diffusiongemma (open-jev) | 26B-A4B | 55.56 | 40.84 | 39.19 |
@@ -99,10 +94,6 @@ We did not produce the planned calibration read or a score without the DI-S sele
99
  | Kev 9B | 9B | 50.48 | 32.96 | 30.54 |
100
  | Kev 4B | 4B | 47.43 | 28.86 | 25.67 |
101
 
102
- We ran the complete archived 0.1 suite: 132,422 requests across 37 benchmarks. The headline index averages 19 panel benchmarks. Comparison rows use the 2026-09-22 leaderboard snapshot. We ran the official kit's scorer locally; these aren't leaderboard submissions. The live [Decision Index](https://huggingface.co/spaces/multimodalart/jev-decision-index) moved to 0.2 on 2026-09-24. Our local 0.2 run is in the section above.
103
-
104
- Third in our local archived 0.1 comparison, behind blink-27b and Jev and ahead of every open entry in the September 22 snapshot (best: Jevfire).
105
-
106
  | Area | blink-mimo-9b | Jev 1.13.0 |
107
  |---|---:|---:|
108
  | Knowledge & Reasoning | 55.1 | 68.8 |
@@ -139,63 +130,24 @@ Third in our local archived 0.1 comparison, behind blink-27b and Jev and ahead o
139
 
140
  ### JevBench: public items only
141
 
 
 
142
  | Model | Easy | Standard | Hard | Hard ECE | Official score |
143
  |---|---:|---:|---:|---:|---|
144
  | **blink-mimo-9b** | 48/48 | 70/72 | 77/111 (0.694) | 0.136 | not submitted |
145
 
146
- Same public-item harness as the other models. These are development results, not official scores or predictions; no official rank or parity is claimed. blink-4b is the JevBench entry. MiMo's hard ECE is 0.136 vs blink-4b's 0.067; it's less calibrated here. Its weakest public hard cases are date/number reasoning, trade-offs and long policies.
147
 
148
- ### Picking the next click (our probe)
 
 
149
 
150
  | Input | blink-mimo-9b | MiMo base | blink-4b |
151
  |---|---:|---:|---:|
152
  | Page text | 53.8% | 49.0% | 55.4% |
153
  | Screenshot | 48.4% | 43.0% | — |
154
 
155
- Our harness and sampling on 500 Multimodal-Mind2Web test steps with five offered elements (chance 20%), not the official evaluation. Five choices are easier than ranking a whole page; blink never trained on web actions. Training lifted MiMo by about five points in both modes, but text still beats screenshots; blink-4b is as good with page text.
156
-
157
- ## What we changed in the network
158
-
159
- ![blink-mimo-9b network diagram: 32 decoder layers repeating 3 Gated DeltaNet layers then one full-attention layer (24 and 8 in total, hidden 4096), with full attention at 0-based layers 3, 7, … 31 as in the tensor names; LoRA rank 16 on every attention, Gated DeltaNet and MLP projection (43.3M parameters, merged after training); token embeddings, norms and lm_head frozen, with lm_head untied, a separate matrix; the 27-block vision tower kept byte-for-byte; and the answer read from the offered option-letter rows of lm_head.](assets/blink-mimo-9b-network.png)
160
-
161
- | | What ships |
162
- |---|---|
163
- | Backbone | MiMo-V2.6-Distill-Qwen-9B (upstream revision 2367e86), a Qwen3.5-9B fine-tune; 32 decoder layers (24 Gated DeltaNet, 8 full-attention), hidden 4096, untied embeddings |
164
- | Vision | 27 encoder blocks, hidden 1152 projected to 4096; all 333 vision tensors unchanged |
165
- | Tuned | 43.3M LoRA parameters; all 248 targeted tensors changed, all 179 other language-model tensors bit-identical to MiMo |
166
- | Final weights | Merged and grafted into MiMo; vision-language reload with no missing or unexpected keys |
167
- | Serving | 9.41B parameters in the full checkpoint (0.46B vision); 8.95B text parameters serving decisions (17.9 GB in bf16); 18.8 GB total |
168
-
169
- **LoRA targets (rank 16, alpha 32, every language-model layer):** full-attention `q_proj`, `k_proj`, `v_proj`, `o_proj`;
170
- Gated DeltaNet `in_proj_qkv`, `in_proj_z`, `in_proj_a`, `in_proj_b`, `out_proj`; and every MLP's
171
- `gate_proj`, `up_proj`, `down_proj`. Token embeddings, all norms and `lm_head` stayed frozen.
172
- The trained adapters were merged into the text weights.
173
-
174
- MiMo's vision tower stays byte-for-byte unchanged. The full checkpoint loads with MiMo's processor and transformers' image-text-to-text classes; `blink.py` and `serve.py` use its text side only. Screenshot results above come from our separate probe harness, not `blink.py`.
175
-
176
- **The real cut is at readout:** no text generation. One prompt pass; next-token logits from only the
177
- offered option-label rows of `lm_head` (verified single tokens A–Z, then two-letter labels), computed in
178
- FP32 and softmaxed over those letters. The rest of the vocabulary is ignored.
179
-
180
- **Objective:** "calibration-oriented decision post-training" is plain supervised fine-tuning.
181
- Cross-entropy uses each row's target distribution: code-computed exact probabilities, probability
182
- targets in teacher-written questions kept after a blind re-solve by that same teacher agreed, and
183
- one-hot labels otherwise. Choice and yes/no options and letter assignments are
184
- reshuffled each epoch; score levels keep their order. Jev's RLCD recipe isn't public; we didn't
185
- use or reproduce it. No RL or preference optimisation.
186
-
187
- ## The climb
188
-
189
- ![blink-mimo-9b post-training diagram: one run (123,195 rows) broken down by data category feed supervised fine-tuning with cross-entropy over the offered option letters; a single pre-registered epoch is merged and grafted back into the full vision-language checkpoint, vision tower unchanged, as release v1.0. Decision Index 0.1 full suite 56.53.](assets/blink-mimo-9b-post-training.png)
190
-
191
- One pre-registered run, one epoch; no other MiMo variant was trained or picked. The gate was DI-S ≥ 55 before training: pass it, then read the full suite and release. DI-S intervals are a few points wide, so this is a narrow gate pass, not a claim of superiority.
192
-
193
- | Step | DI-S | Full 0.1 | Outside DI-S | Why |
194
- |---|---:|---:|---:|---|
195
- | MiMo base, zero-shot | 47.09 | — | — | Baseline before decision training. |
196
- | **blink-mimo-9b** (123,195 question rows; lr 5e-5; 615 steps) | 55.3 | 56.53 | 56.60 | Program-labelled reasoning, teacher questions and judge data cleared the gate; shipped. |
197
-
198
- DI-S areas: Knowledge 55.6, Language 66.4, Retrieval 33.0, Tools 70.7, Arts 50.8.
199
 
200
  ## Use
201
 
@@ -226,13 +178,9 @@ print(out["answers"]["intent"]["probabilities"])
226
 
227
  ## Run it as a server
228
 
229
- `serve.py` handles TypeSafe's request and answer fields at `POST /v1/systemone` and lists its model at
230
- `GET /v1/models`. From server-side code, point TypeSafe's Python or JavaScript SDK at the server with
231
- `TYPESAFE_BASE_URL`;
232
- JevBench's stock `typesafe` adapter and the Decision Index kit's `http` engine still work unchanged.
233
- `GET /healthz` reports startup checks. `v1.2` changes code only; its weights are identical to `v1.0`.
234
 
235
- See the [wire-format reference](https://thegovind.github.io/blink/wire-format/) for full API details.
236
 
237
  ```sh
238
  pip install "torch==2.13.0" "transformers==5.17.0" "flash-linear-attention==0.5.2" "accelerate>=1.1.0" safetensors huggingface_hub
@@ -241,81 +189,57 @@ python blink-mimo-9b/serve.py --model ./blink-mimo-9b --port 8000
241
  # TypeSafe SDKs: export TYPESAFE_BASE_URL=http://127.0.0.1:8000 TYPESAFE_API_KEY=any
242
  ```
243
 
244
- In another terminal: `curl -s http://127.0.0.1:8000/healthz`.
245
-
246
- **Or use Docker** from the downloaded folder:
247
 
248
  ```sh
249
  cd blink-mimo-9b
250
  docker build -t blink-mimo-9b . && docker run --rm --gpus all -p 127.0.0.1:8000:8000 blink-mimo-9b
251
  ```
252
 
253
- <details><summary>Health, limits and weights</summary>
254
-
255
- - `/healthz` reports `weights_verified` (weight, config and tokenizer files listed in `weights.sha256`
256
- are hashed before serving; a mismatch stops startup), `warmup.repeat_identical` (two matching warm-up
257
- answers), `kernels` (fast path or slower fallback without flash-linear-attention), `versions` and `hub_offline`.
258
- - Limits: 255 options per choice, 2–10 score levels, 131,072 input tokens per question and 512 questions
259
- per request. Over-limit requests get HTTP 422 with the reason; nothing is truncated.
260
- - `GET /v1/models` lists the one served model with a blank `release_date`. Every request uses that model
261
- regardless of its `model` field.
262
- - The server is open by default. Set `--api-key` or `BLINK_API_KEY` to require `Authorization: Bearer <key>` on both API
263
- routes. Missing or wrong keys get 401; `/healthz` stays open.
264
- - Error bodies put the reason in `error` and `detail`. Over-limit requests return 422, with nothing cut.
265
- - Requests run one at a time. Questions are batched; each batch takes one forward pass (large requests
266
- can take more than one). Serving the downloaded folder or Docker image enables Hugging Face offline
267
- mode before model loading (`hub_offline: true`). The server doesn't otherwise restrict network access.
268
- - Set `--batch-window-ms 5` to turn on cross-request batching with a 5 ms collection window, up to
269
- `--max-batch-requests` requests at a time, which defaults to 16. The window defaults to 0, so requests still
270
- run one at a time. `--max-queued-requests` lets up to 64 requests wait for a batch by default. Excess requests
271
- get HTTP 529 with `Retry-After`, so clients should retry. `v1.1` returned HTTP 503 for a full queue. On a
272
- 1,000-request Decision Index sample over HTTP, throughput rose about 15% with 4 concurrent clients and 16%
273
- with 16. One client saw no gain. Offline runs on long documents showed no meaningful gain. On the Decision
274
- Index sample, a set of long workflow documents, and the public TypeSafe cases covered by the FP32 reference,
275
- batched answers passed the same numerical-parity checks against an FP32 reference as one-at-a-time answers,
276
- covering argmax agreement and probability differences. TypeSafe documents were sent as JSON objects. The FP32
277
- reference could not run the five longest TypeSafe documents, so parity covered 170 of 354 TypeSafe questions.
278
- Batched and one-at-a-time answers chose the same option every time on the other 184. A few near-tied answers
279
- can still flip. Batching arrived in `v1.1`. The current code revision is `v1.2`, with weights identical to
280
- `v1.0`. Update the two code files in an existing `v1.0` download, then restart with the flag:
281
 
 
 
 
 
 
 
282
  ```sh
283
  hf download thegovind/blink-mimo-9b serve.py blink.py --revision v1.2 --local-dir blink-mimo-9b
284
  python blink-mimo-9b/serve.py --model ./blink-mimo-9b --port 8000 --batch-window-ms 5
285
  ```
286
- - blink-mimo-9b weights are 18.8 GB in bf16. Long prompts need more memory.
287
 
288
  </details>
289
 
290
- <details><summary>Model and probability readout</summary>
291
 
292
- | | |
293
- |---|---|
294
- | Base | [XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B) (MIT declared in the model card; see LICENSE-MiMo.md), full vision-language weights |
295
- | Adaptation | LoRA r16 / alpha 32 across the language model's attention, Gated DeltaNet and MLP; merged into the full MiMo checkpoint |
296
- | Readout | Verified single-token labels (A–Z, then two-letter labels); FP32 softmax of next-token logits / T over the offered labels |
297
- | Temperature | 1.0 (not fitted) |
298
- | Input limit | 131,072 tokens per question; longest evaluated prompt: 37,906 tokens; longer inputs are refused, never truncated |
299
- | Runtime | `blink.py` builds prompts and reads out probabilities for text decisions |
300
-
301
- These are option-conditional model probabilities, not certified chances of being right. Calibration can shift
302
- across tasks, domains and option sets. For `choice`, `confidence = (p_max − 1/K)/(1 − 1/K)` measures
303
- concentration, not correctness. For `score`, `score` is the expected 0-based level and `choice` is the
304
- most likely level.
305
 
306
- </details>
307
 
308
- <details><summary>Training and data</summary>
 
 
 
 
309
 
310
  ### How it was trained
311
 
312
- | Stage | Question rows | Mix |
313
- |---|---:|---|
314
- | MiMo | 123,195 | 61,394 public-source · 23,894 program-generated reasoning · 12,000 decision worlds · 7,860 teacher-written question rows · 7,000 judge-style · 6,000 chess move choices · 5,047 exact-probability worlds |
315
 
316
- One pre-registered supervised run, one epoch (lr 5e-5, 615 steps). The mix joins the 27B's stage-1 public and exact-probability rows with its stage-2 reasoning, chess and decision worlds, plus judge-style data. Qwen3.8-27B wrote the teacher documents and typed questions. Public train splits include ContractNLI, iSarcasmEval, VAST, Amazon ESCI, Humicroedit, ANLI, WANLI, BoolQ, BANKING77, MedMCQA, AQuA, MMLU auxiliary train, SciQ and SuperGPQA.
317
 
318
- ### Data sources and licences
 
 
 
 
 
 
 
319
 
320
  | Source | Licence |
321
  |---|---|
@@ -331,46 +255,30 @@ One pre-registered supervised run, one epoch (lr 5e-5, 615 steps). The mix joins
331
  | SciQ | CC BY-NC 3.0 |
332
  | iSarcasmEval | MIT (upstream repository licence) |
333
  | VAST, Humicroedit, OpenBookQA | None stated by source |
334
- | Our code-generated worlds and teacher-written documents (Qwen3.8-27B) | See LICENSE.md |
335
-
336
- These are source-repository licences; they don't settle rights in every underlying text.
337
 
338
- </details>
339
-
340
- <details><summary>Evaluation notes and limits</summary>
341
 
342
  ### Evaluation notes
343
 
344
- - **Scorer parity.** `blink.py` and the lab scorer agree within 1e-7 on this checkpoint (p99 |Δp| 1.4e-8); the published graft matches the evaluated adapter on JevBench's 231 public items (0 argmax changes, max |Δp| 0.0). The kit's per-request timer on 1,000 random suite requests served one at a time measured 68.3 ms median via HTTP vs 66.9 ms in-process. From a fresh Hub download and install at the pinned revision, `serve.py` with JevBench's stock adapter returned easy 48/48, standard 70/72 and hard 77/111, matching the evaluation's answers (0 argmax changes; max |Δp| about 0.03).
345
- - **Selection.** The prompt format came from earlier DI-S reads; this MiMo run used the sample as a pre-registered gate. The full suite followed that gate; it scored 56.60 on the 129,422 requests outside DI-S, which were not used for selection.
346
- - **Training overlap.** Public train splits also used by the 0.1 index: ContractNLI, iSarcasmEval, VAST, Amazon ESCI, Humicroedit, ChessBench (searchless_chess training positions; none of the 5,000 test positions), GSM8K (train split; solution-checking items). We also used ANLI and BANKING77 train splits; they're in the 0.1 suite but outside its index, and both are in the 0.2 panel. The audit below reports what was checked and any shared passages.
347
- - **Partitions.** Public-source data included training and development partitions.
348
- - **Final-mixture audit.** Rechecked every question row (including teacher-written rows) against the complete 0.1 suite (132,422 requests) and JevBench's 231 public items. The checks looked for exact matches of normalised strings of at least 30 characters in any field and shared 13-word passages in each row's question text (instructions, state.question, state.code). Strings or passages seen in 20 or more suite requests were treated as prompt templates and ignored. No public JevBench item matched under these checks; a separate position check found no shared chess positions.
349
- - **Suite overlap.** 16 BANKING77/VAST training rows share a 13-word passage with 31 suite requests: 23 of VAST's 3,006 and 8 of BANKING77's 3,080. Two VAST training posts are near-duplicates of a test post, but these rows had no exact normalised-text match under the audit. Dropping those requests leaves the index at 56.53 (VAST 0.7805 → 0.7803); BANKING77 is outside the 0.1 index.
350
- - **Audit limits.** The 13-word passage check didn't search long-document bodies or option text. Semantic or pretraining overlap can't be ruled out, and private JevBench items weren't available to check.
351
- - **Generated reasoning.** Our programs computed the labels for CRUXEval-style code and CLadder-style causal questions; no items from those benchmarks were used. We didn't reuse the suite's GSM8K distractors.
352
- - **Teacher documents.** We kept Qwen3.8-27B's documents only if a fresh blind solve by that same teacher agreed with the answer. That's an agreement filter, not independent verification.
353
- - The repo ships no benchmark items, GPQA text, JevBench items or teacher traces.
354
- - **No MMLU-Pro or GPQA.** No rows from either were used in this fine-tune; SuperGPQA is a separate source.
355
 
356
  ### Limits
357
 
358
- - English-centric. Training included Arabic iSarcasmEval rows; on the 0.1 suite, Arabic task A scored 0.321 and task C pairs 0.79. Broader multilingual performance hasn't been established.
359
- - Doesn't chat or explain answers.
360
- - Text in the state can sway the answer.
361
- - Its weakest public hard cases are date/number reasoning, trade-offs and long policies.
362
 
363
  </details>
364
 
365
- ## Licence
366
-
367
- The MiMo model card declares MIT without a separate upstream licence file or copyright line (`LICENSE-MiMo.md`); its Qwen/Qwen3.5-9B base is Apache-2.0 (`LICENSE-Qwen`). The blink weights are for **non-commercial research and evaluation
368
- only** (`LICENSE.md`); commercial use isn't licensed. Training used non-commercial, share-alike and unlicensed
369
- sources (see the table above). It's unsettled whether their terms reach the weights, so check upstream terms too.
370
- `blink.py`, `serve.py` and the Dockerfile are Apache-2.0.
371
-
372
- <details><summary>Credits</summary>
373
 
374
- Xiaomi MiMo (MiMo base) and the Qwen team (Qwen base). SemIf (MIT) for the evidence/criterion/options prompt layout. The Decision Index kit (MIT) and JevBench (MIT) for evaluation.
375
-
376
- </details>
 
14
  language:
15
  - en
16
  ---
 
17
  # blink-mimo-9b
18
 
19
+ Send a text or JSON `state` and typed questions (`choice`, `noul`, `score`). Each question gets probabilities over its offered options from one forward pass, with no generated text. Long or large multi-question requests may use several batches.
 
 
20
 
21
+ ![blink-mimo-9b one-pass readout diagram: a state and typed choice, noul and score questions are rendered as evidence, criterion and lettered options; questions are batched and each batch is one forward pass, giving next-token logits at the answer position, and an FP32 softmax over the offered letters gives one probability per option. No text is generated.](assets/blink-mimo-9b-readout.png)
 
 
22
 
23
+ [Space demo](https://huggingface.co/spaces/thegovind/blink) · [Source code](https://github.com/thegovind/blink) · [Docs](https://thegovind.github.io/blink/) · [API](https://thegovind.github.io/blink/api/) · [blink-4b](https://huggingface.co/thegovind/blink-4b) · [blink-27b](https://huggingface.co/thegovind/blink-27b)
24
 
25
+ ## At a glance
 
26
 
27
+ | Attribute | Detail |
28
+ |---|---|
29
+ | Base model | [XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B) |
30
+ | Weights size | 18.8 GB bf16 (8.95B text parameters, 9.41B total) |
31
+ | Revision | v1.2 (weights identical to v1.0) |
32
+ | License | Non-commercial research ([LICENSE.md](LICENSE.md)); code Apache-2.0 |
33
 
34
  ## Results
35
 
36
  ### Decision Index 0.2 (local run)
37
 
38
+ Decision Index numbers are local runs of the official kit (commit 19ad28e, 2026-09-25), not leaderboard submissions. The 0.2 run is descriptive: known training exposure stays in the scores, with no leaderboard-style penalty, so it isn't ranked. The fine-tune didn't use MMLU-Pro or GPQA as direct data sources; screening found 176 SuperGPQA training rows matching added-request text.
39
 
40
  | Balanced skill | Balanced raw | Breadth skill | Without MMLU-Pro |
41
  |---:|---:|---:|---:|
42
  | 43.36 | 57.24 | 42.38 | 42.84 |
43
 
 
 
44
  <details><summary>Extra tables and method</summary>
45
 
46
  | Area | Number of benchmarks | Skill | Raw |
 
51
  | Tools & Automation | 6 | 57.0 | 65.2 |
52
  | Arts & Human Taste | 7 | 32.0 | 50.2 |
53
 
54
+ The seven benchmarks added in 0.2:
55
 
56
  | Benchmark | Metric | Requests | Answered | Raw | Skill |
57
  |---|---|---:|---:|---:|---:|
 
63
  | When2Call MCQ | accuracy | 3,652 | 3,652 | 64.5 | 52.7 |
64
  | New Yorker caption matching | accuracy | 528 | 528 | 62.5 | 53.1 |
65
 
66
+ - All 151,034 scoreable requests scored across 44 benchmarks (40 count toward index across 5 areas).
67
+ - Requests shared with 0.1 reuse 0.1 predictions; the 30,419 added requests ran with the same frozen evaluation setup as 0.1, the evaluated adapter loaded on the MiMo base, which the published graft matched on JevBench's 231 public items (see Evaluation notes), at temperature 1.0.
68
+ - "Without MMLU-Pro" drops MMLU-Pro, averages the other nine Knowledge benchmarks, and keeps five equal areas. It is a sensitivity check, not a score free of training effects.
69
+ - Point estimates without significance, calibration, or latency claims. Do not compare with 0.1 numbers.
70
 
71
+ Training-row text matches in added requests:
72
 
73
  | Training stage | Rows in the stage | Rows matching added-request text | From MMLU-Pro | From SuperGPQA | Other |
74
  |---|---:|---:|---:|---:|---:|
75
  | MiMo | 123,195 | 180 | 0 | 176 | 4 |
76
 
77
+ Screened for exact normalized strings of at least 30 characters shared by training rows and added requests (strings in 20 or more requests treated as templates). Counts are training rows, not unique test questions.
 
 
78
 
79
  </details>
80
 
81
  ### Decision Index 0.1 (archived edition)
82
 
83
+ Decision Index numbers are local runs of the official kit, not leaderboard submissions. Comparison rows use the 2026-09-22 leaderboard snapshot.
84
 
85
+ Training overlap: Public train splits included ContractNLI, iSarcasmEval and VAST (the whole Language area), plus Amazon ESCI and Humicroedit. With Language set to Jev's score, this model's index would be 54.96 vs Jev's 59.51. That's arithmetic, not an ablation or a like-for-like comparison with models trained only on synthetic data.
86
 
87
  | Model | Size class | Decision Index 0.1 | Skill | Breadth |
88
  |---|---|---:|---:|---:|
89
+ | **blink-mimo-9b** | 9B | 56.53 | 42.17 | 40.37 |
90
  | Jev 1.13.0 | closed | 59.51 | 46.26 | 44.79 |
91
  | Jevfire | 27B | 55.74 | 40.86 | 39.45 |
92
  | JoshuaSP diffusiongemma (open-jev) | 26B-A4B | 55.56 | 40.84 | 39.19 |
 
94
  | Kev 9B | 9B | 50.48 | 32.96 | 30.54 |
95
  | Kev 4B | 4B | 47.43 | 28.86 | 25.67 |
96
 
 
 
 
 
97
  | Area | blink-mimo-9b | Jev 1.13.0 |
98
  |---|---:|---:|
99
  | Knowledge & Reasoning | 55.1 | 68.8 |
 
130
 
131
  ### JevBench: public items only
132
 
133
+ The JevBench numbers are public-item development proxies, not official scores, and claim no rank or parity.
134
+
135
  | Model | Easy | Standard | Hard | Hard ECE | Official score |
136
  |---|---:|---:|---:|---:|---|
137
  | **blink-mimo-9b** | 48/48 | 70/72 | 77/111 (0.694) | 0.136 | not submitted |
138
 
139
+ Hard ECE is 0.136 vs blink-4b's 0.067. Weakest public hard cases are date/number reasoning, trade-offs, and long policies.
140
 
141
+ ### Next-click probe
142
+
143
+ Sampling on 500 Multimodal-Mind2Web test steps with 5 offered elements (20% chance baseline); not an official evaluation.
144
 
145
  | Input | blink-mimo-9b | MiMo base | blink-4b |
146
  |---|---:|---:|---:|
147
  | Page text | 53.8% | 49.0% | 55.4% |
148
  | Screenshot | 48.4% | 43.0% | — |
149
 
150
+ `blink.py` and `serve.py` use its text side only. Screenshot results above come from the separate probe harness, not `blink.py`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
151
 
152
  ## Use
153
 
 
178
 
179
  ## Run it as a server
180
 
181
+ `serve.py` provides TypeSafe server-side SDK compatibility at `POST /v1/systemone` and lists the served model at `GET /v1/models`. Set `TYPESAFE_BASE_URL` to connect TypeSafe Python or JavaScript SDKs; JevBench's `typesafe` adapter and Decision Index's `http` engine work unchanged. `GET /healthz` reports startup checks. `v1.2` updates code only; weights are identical to `v1.0`.
 
 
 
 
182
 
183
+ See the [API reference](https://thegovind.github.io/blink/api/) for full details.
184
 
185
  ```sh
186
  pip install "torch==2.13.0" "transformers==5.17.0" "flash-linear-attention==0.5.2" "accelerate>=1.1.0" safetensors huggingface_hub
 
189
  # TypeSafe SDKs: export TYPESAFE_BASE_URL=http://127.0.0.1:8000 TYPESAFE_API_KEY=any
190
  ```
191
 
192
+ Or run with Docker:
 
 
193
 
194
  ```sh
195
  cd blink-mimo-9b
196
  docker build -t blink-mimo-9b . && docker run --rm --gpus all -p 127.0.0.1:8000:8000 blink-mimo-9b
197
  ```
198
 
199
+ <details><summary>Server options, batching, and limits</summary>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
200
 
201
+ - **API key:** Optional. Set `--api-key` or `BLINK_API_KEY` to require `Authorization: Bearer <key>` on API routes. Missing or invalid keys return 401; `/healthz` stays open.
202
+ - **Limits and errors:** 255 options per choice, 2–10 score levels, 131,072 input tokens per question (longest evaluated prompt: 37,906 tokens), 512 questions per request. Error bodies set `error` and `detail`. Requests exceeding limits return 422 with the reason; inputs are not truncated.
203
+ - **Model listing:** `GET /v1/models` lists the served model with a blank `release_date`.
204
+ - **Batching:** Set `--batch-window-ms 5` to enable cross-request batching (default 0 runs requests one at a time), up to `--max-batch-requests` (default 16). `--max-queued-requests` holds up to 64 waiting requests; excess requests receive HTTP 529 with `Retry-After` (returned 503 in v1.1).
205
+ - **FP32 reference coverage caveat:** On the Decision Index sample, a set of long workflow documents, and the public TypeSafe cases covered by the FP32 reference, batched answers passed the same numerical-parity checks against an FP32 reference as one-at-a-time answers, covering argmax agreement and probability differences. The FP32 reference could not run the 5 longest TypeSafe documents, so parity covered 170 of 354 TypeSafe questions; batched and one-at-a-time runs chose the same option on the other 184. A few near-tied answers can still flip.
206
+ - **Batching command:**
207
  ```sh
208
  hf download thegovind/blink-mimo-9b serve.py blink.py --revision v1.2 --local-dir blink-mimo-9b
209
  python blink-mimo-9b/serve.py --model ./blink-mimo-9b --port 8000 --batch-window-ms 5
210
  ```
211
+ - **Weights:** 18.8 GB in bf16 (8.95B text parameters, 9.41B total).
212
 
213
  </details>
214
 
215
+ <details><summary>Details: Architecture, training, evaluation, and limits</summary>
216
 
217
+ ### Network architecture
 
 
 
 
 
 
 
 
 
 
 
 
218
 
219
+ ![blink-mimo-9b network diagram: 32 decoder layers repeating 3 Gated DeltaNet layers then one full-attention layer (24 and 8 in total, hidden 4096), with full attention at 0-based layers 3, 7, … 31 as in the tensor names; LoRA rank 16 on every attention, Gated DeltaNet and MLP projection (43.3M parameters, merged after training); token embeddings, norms and lm_head frozen, with lm_head untied, a separate matrix; the 27-block vision tower kept byte-for-byte; and the answer read from the offered option-letter rows of lm_head.](assets/blink-mimo-9b-network.png)
220
 
221
+ - **Backbone:** MiMo-V2.6-Distill-Qwen-9B (upstream revision 2367e86), a Qwen3.5-9B fine-tune. 32 decoder layers (24 Gated DeltaNet, 8 full-attention), hidden dimension 4096, untied embeddings.
222
+ - **Vision:** 27 encoder blocks, hidden 1152 projected to 4096; all 333 vision tensors unchanged.
223
+ - **Tuned:** 43.3M LoRA parameters; all 248 targeted tensors changed, all 179 other language-model tensors bit-identical to MiMo.
224
+ - **LoRA targets (rank 16, alpha 32, every language-model layer):** full-attention `q_proj`, `k_proj`, `v_proj`, `o_proj`; Gated DeltaNet `in_proj_qkv`, `in_proj_z`, `in_proj_a`, `in_proj_b`, `out_proj`; and every MLP's `gate_proj`, `up_proj`, `down_proj`. Token embeddings, all norms and `lm_head` stayed frozen. The trained adapters were merged into the text weights.
225
+ - **Readout:** Single forward pass without text generation. Next-token logits from offered option rows of `lm_head` (single tokens A–Z, then two-letter labels), computed in FP32 and softmaxed over offered letters. Temperature 1.0 (not fitted). These are option-conditional model probabilities, not certified chances of being right.
226
 
227
  ### How it was trained
228
 
229
+ ![blink-mimo-9b post-training diagram: one run (123,195 rows) broken down by data category feed supervised fine-tuning with cross-entropy over the offered option letters; a single pre-registered epoch is merged and grafted back into the full vision-language checkpoint, vision tower unchanged, as release v1.0. Decision Index 0.1 full suite 56.53.](assets/blink-mimo-9b-post-training.png)
230
+
231
+ One pre-registered run and one epoch (123,195 rows, lr 5e-5, 615 steps). Supervised fine-tuning with cross-entropy on target distributions (exact probabilities, agreed teacher targets, or one-hot labels). Options and letter assignments reshuffled each epoch; score levels preserve order. No RL or preference optimization.
232
 
233
+ The DI-S gate required DI-S >= 55 before reading the full suite. DI-S scored 55.3 (Knowledge 55.6, Language 66.4, Retrieval 33.0, Tools 70.7, Arts 50.8), clearing the gate. Outside DI-S, the model scored 56.60 across the 129,422 requests not used for selection.
234
 
235
+ | Step | DI-S | Full 0.1 | Outside DI-S | Note |
236
+ |---|---:|---:|---:|---|
237
+ | MiMo base, zero-shot | 47.09 | — | — | Baseline before decision training. |
238
+ | **blink-mimo-9b** (123,195 question rows; lr 5e-5; 615 steps) | 55.3 | 56.53 | 56.60 | Cleared DI-S gate; shipped. |
239
+
240
+ Training mix (123,195 rows): 61,394 public-source, 23,894 program-generated reasoning, 12,000 decision worlds, 7,860 teacher-written rows (Qwen3.8-27B), 7,000 judge-style, 6,000 chess move choices, and 5,047 exact-probability worlds.
241
+
242
+ ### Data sources and licenses
243
 
244
  | Source | Licence |
245
  |---|---|
 
255
  | SciQ | CC BY-NC 3.0 |
256
  | iSarcasmEval | MIT (upstream repository licence) |
257
  | VAST, Humicroedit, OpenBookQA | None stated by source |
258
+ | Code-generated worlds and teacher-written documents (Qwen3.8-27B) | See LICENSE.md |
 
 
259
 
260
+ Source repository licenses do not settle rights in underlying texts.
 
 
261
 
262
  ### Evaluation notes
263
 
264
+ - **Scorer parity:** `blink.py` and the lab scorer agree within 1e-7 on this checkpoint (p99 |Δp| 1.4e-8). Published weights match the evaluated adapter on 231 public JevBench items (0 argmax changes, max |Δp| 0.0). Median per-request latency on 1,000 suite requests was 68.3 ms via HTTP vs 66.9 ms in-process. `serve.py` matched evaluation answers on public items (easy 48/48, standard 70/72, hard 77/111; 0 argmax changes; max |Δp| about 0.03).
265
+ - **Selection:** Prompt format came from earlier DI-S reads. The DI-S sample served as a pre-registered gate. The remaining 129,422 suite requests outside DI-S were not used for selection and scored 56.60.
266
+ - **Training overlap:** Public train splits in the 0.1 index: ContractNLI, iSarcasmEval, VAST, Amazon ESCI, Humicroedit, ChessBench (training positions; none of the 5,000 test positions), GSM8K train split. ANLI and BANKING77 train splits were also used (in 0.1 suite but outside index; in 0.2 panel). Public-source data included training and development partitions.
267
+ - **Mixture audit:** Every question row was screened against the 132,422 requests in 0.1 and 231 public JevBench items for exact matches (>=30 normalized characters) and shared 13-word passages (ignoring strings appearing in 20 or more requests as templates). No public JevBench items or chess positions matched.
268
+ - **Overlap findings:** 16 BANKING77/VAST training rows share a 13-word passage with 31 suite requests (23 VAST, 8 BANKING77). Two VAST training posts are near-duplicates of a test post, but these rows had no exact normalised-text match under the audit. Dropping these requests leaves the index at 56.53 (VAST 0.7805 -> 0.7803); BANKING77 is outside the 0.1 index.
269
+ - **Audit limits:** The 13-word check did not search long-document bodies or option text. Semantic or pretraining overlap cannot be ruled out. Private JevBench items were unavailable to check.
270
+ - **Generated data:** Program-computed labels were used for CRUXEval-style and CLadder-style questions; no benchmark items were used. Suite GSM8K distractors were not reused. Teacher documents (Qwen3.8-27B) were kept only when a blind re-solve by the teacher agreed.
271
+ - **MMLU-Pro and GPQA:** Neither benchmark was used in training; SuperGPQA is a separate source.
 
 
 
272
 
273
  ### Limits
274
 
275
+ - **English-centric:** Training included Arabic iSarcasmEval rows (Arabic task A scored 0.321, task C pairs 0.79 on 0.1). Multilingual performance is not established.
276
+ - **No text generation:** Does not chat or explain answers.
277
+ - **Prompt sensitivity:** Text in the state can influence the answer.
278
+ - **Reasoning gaps:** Weakest public hard cases are date/number reasoning, trade-offs, and long policies.
279
 
280
  </details>
281
 
282
+ ## License
 
 
 
 
 
 
 
283
 
284
+ Code (`blink.py`, `serve.py`, Dockerfile): Apache-2.0. Weights: non-commercial research only; see [LICENSE.md](LICENSE.md). The MiMo model card declares MIT without a separate upstream licence file or copyright line ([LICENSE-MiMo.md](LICENSE-MiMo.md)); its Qwen/Qwen3.5-9B base is Apache-2.0 ([LICENSE-Qwen](LICENSE-Qwen)).