Image-Text-to-Text
Transformers
Safetensors
English
qwen3_5
decision-model
typed-decisions
one-pass
option-probabilities
conversational
Instructions to use thegovind/blink-mimo-9b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use thegovind/blink-mimo-9b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="thegovind/blink-mimo-9b") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("thegovind/blink-mimo-9b") model = AutoModelForMultimodalLM.from_pretrained("thegovind/blink-mimo-9b", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use thegovind/blink-mimo-9b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "thegovind/blink-mimo-9b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thegovind/blink-mimo-9b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/thegovind/blink-mimo-9b
- SGLang
How to use thegovind/blink-mimo-9b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "thegovind/blink-mimo-9b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thegovind/blink-mimo-9b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "thegovind/blink-mimo-9b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thegovind/blink-mimo-9b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use thegovind/blink-mimo-9b with Docker Model Runner:
docker model run hf.co/thegovind/blink-mimo-9b
Shorter card; link the API docs
Browse files
README.md
CHANGED
|
@@ -14,36 +14,33 @@ tags:
|
|
| 14 |
language:
|
| 15 |
- en
|
| 16 |
---
|
| 17 |
-
|
| 18 |
# blink-mimo-9b
|
| 19 |
|
| 20 |
-
Send a text or JSON `state` and
|
| 21 |
-
and `score` takes 2–10 ordered levels. Each question gets probabilities over its offered options from
|
| 22 |
-
one forward pass, with no generated text. Long or large multi-question requests may use several batches.
|
| 23 |
|
| 24 |
-
|
| 25 |
-
[blink-4b](https://huggingface.co/thegovind/blink-4b) · [blink-27b](https://huggingface.co/thegovind/blink-27b) ·
|
| 26 |
-
[blink-mimo-9b](https://huggingface.co/thegovind/blink-mimo-9b).
|
| 27 |
|
| 28 |
-
[Source code](https://github.com/thegovind/blink) · [Docs](https://thegovind.github.io/blink/) · [
|
| 29 |
|
| 30 |
-
|
| 31 |
-
Xiaomi, Alibaba Cloud or the Qwen team. Weights are for non-commercial research; see [Licence](#licence).*
|
| 32 |
|
| 33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
## Results
|
| 36 |
|
| 37 |
### Decision Index 0.2 (local run)
|
| 38 |
|
| 39 |
-
|
| 40 |
|
| 41 |
| Balanced skill | Balanced raw | Breadth skill | Without MMLU-Pro |
|
| 42 |
|---:|---:|---:|---:|
|
| 43 |
| 43.36 | 57.24 | 42.38 | 42.84 |
|
| 44 |
|
| 45 |
-
- Our Blink fine-tune did not use MMLU-Pro or GPQA as direct data sources. Screening found 176 SuperGPQA training rows matching added-request text. SuperGPQA is a training source, not a benchmark in this suite. Public train splits used in training are listed under “Training overlap” below.
|
| 46 |
-
|
| 47 |
<details><summary>Extra tables and method</summary>
|
| 48 |
|
| 49 |
| Area | Number of benchmarks | Skill | Raw |
|
|
@@ -54,7 +51,7 @@ We ran the full Decision Index 0.2 suite ourselves with the official scoring kit
|
|
| 54 |
| Tools & Automation | 6 | 57.0 | 65.2 |
|
| 55 |
| Arts & Human Taste | 7 | 32.0 | 50.2 |
|
| 56 |
|
| 57 |
-
The seven benchmarks added in 0.2
|
| 58 |
|
| 59 |
| Benchmark | Metric | Requests | Answered | Raw | Skill |
|
| 60 |
|---|---|---:|---:|---:|---:|
|
|
@@ -66,32 +63,30 @@ The seven benchmarks added in 0.2.
|
|
| 66 |
| When2Call MCQ | accuracy | 3,652 | 3,652 | 64.5 | 52.7 |
|
| 67 |
| New Yorker caption matching | accuracy | 528 | 528 | 62.5 | 53.1 |
|
| 68 |
|
| 69 |
-
- All 151,034
|
| 70 |
-
- Requests shared with 0.1 reuse
|
| 71 |
-
-
|
| 72 |
-
-
|
| 73 |
|
| 74 |
-
Training-row text matches in
|
| 75 |
|
| 76 |
| Training stage | Rows in the stage | Rows matching added-request text | From MMLU-Pro | From SuperGPQA | Other |
|
| 77 |
|---|---:|---:|---:|---:|---:|
|
| 78 |
| MiMo | 123,195 | 180 | 0 | 176 | 4 |
|
| 79 |
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
We did not produce the planned calibration read or a score without the DI-S selection sample.
|
| 83 |
|
| 84 |
</details>
|
| 85 |
|
| 86 |
### Decision Index 0.1 (archived edition)
|
| 87 |
|
| 88 |
-
|
| 89 |
|
| 90 |
-
|
| 91 |
|
| 92 |
| Model | Size class | Decision Index 0.1 | Skill | Breadth |
|
| 93 |
|---|---|---:|---:|---:|
|
| 94 |
-
| **blink-mimo-9b**
|
| 95 |
| Jev 1.13.0 | closed | 59.51 | 46.26 | 44.79 |
|
| 96 |
| Jevfire | 27B | 55.74 | 40.86 | 39.45 |
|
| 97 |
| JoshuaSP diffusiongemma (open-jev) | 26B-A4B | 55.56 | 40.84 | 39.19 |
|
|
@@ -99,10 +94,6 @@ We did not produce the planned calibration read or a score without the DI-S sele
|
|
| 99 |
| Kev 9B | 9B | 50.48 | 32.96 | 30.54 |
|
| 100 |
| Kev 4B | 4B | 47.43 | 28.86 | 25.67 |
|
| 101 |
|
| 102 |
-
We ran the complete archived 0.1 suite: 132,422 requests across 37 benchmarks. The headline index averages 19 panel benchmarks. Comparison rows use the 2026-09-22 leaderboard snapshot. We ran the official kit's scorer locally; these aren't leaderboard submissions. The live [Decision Index](https://huggingface.co/spaces/multimodalart/jev-decision-index) moved to 0.2 on 2026-09-24. Our local 0.2 run is in the section above.
|
| 103 |
-
|
| 104 |
-
Third in our local archived 0.1 comparison, behind blink-27b and Jev and ahead of every open entry in the September 22 snapshot (best: Jevfire).
|
| 105 |
-
|
| 106 |
| Area | blink-mimo-9b | Jev 1.13.0 |
|
| 107 |
|---|---:|---:|
|
| 108 |
| Knowledge & Reasoning | 55.1 | 68.8 |
|
|
@@ -139,63 +130,24 @@ Third in our local archived 0.1 comparison, behind blink-27b and Jev and ahead o
|
|
| 139 |
|
| 140 |
### JevBench: public items only
|
| 141 |
|
|
|
|
|
|
|
| 142 |
| Model | Easy | Standard | Hard | Hard ECE | Official score |
|
| 143 |
|---|---:|---:|---:|---:|---|
|
| 144 |
| **blink-mimo-9b** | 48/48 | 70/72 | 77/111 (0.694) | 0.136 | not submitted |
|
| 145 |
|
| 146 |
-
|
| 147 |
|
| 148 |
-
###
|
|
|
|
|
|
|
| 149 |
|
| 150 |
| Input | blink-mimo-9b | MiMo base | blink-4b |
|
| 151 |
|---|---:|---:|---:|
|
| 152 |
| Page text | 53.8% | 49.0% | 55.4% |
|
| 153 |
| Screenshot | 48.4% | 43.0% | — |
|
| 154 |
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
## What we changed in the network
|
| 158 |
-
|
| 159 |
-

|
| 160 |
-
|
| 161 |
-
| | What ships |
|
| 162 |
-
|---|---|
|
| 163 |
-
| Backbone | MiMo-V2.6-Distill-Qwen-9B (upstream revision 2367e86), a Qwen3.5-9B fine-tune; 32 decoder layers (24 Gated DeltaNet, 8 full-attention), hidden 4096, untied embeddings |
|
| 164 |
-
| Vision | 27 encoder blocks, hidden 1152 projected to 4096; all 333 vision tensors unchanged |
|
| 165 |
-
| Tuned | 43.3M LoRA parameters; all 248 targeted tensors changed, all 179 other language-model tensors bit-identical to MiMo |
|
| 166 |
-
| Final weights | Merged and grafted into MiMo; vision-language reload with no missing or unexpected keys |
|
| 167 |
-
| Serving | 9.41B parameters in the full checkpoint (0.46B vision); 8.95B text parameters serving decisions (17.9 GB in bf16); 18.8 GB total |
|
| 168 |
-
|
| 169 |
-
**LoRA targets (rank 16, alpha 32, every language-model layer):** full-attention `q_proj`, `k_proj`, `v_proj`, `o_proj`;
|
| 170 |
-
Gated DeltaNet `in_proj_qkv`, `in_proj_z`, `in_proj_a`, `in_proj_b`, `out_proj`; and every MLP's
|
| 171 |
-
`gate_proj`, `up_proj`, `down_proj`. Token embeddings, all norms and `lm_head` stayed frozen.
|
| 172 |
-
The trained adapters were merged into the text weights.
|
| 173 |
-
|
| 174 |
-
MiMo's vision tower stays byte-for-byte unchanged. The full checkpoint loads with MiMo's processor and transformers' image-text-to-text classes; `blink.py` and `serve.py` use its text side only. Screenshot results above come from our separate probe harness, not `blink.py`.
|
| 175 |
-
|
| 176 |
-
**The real cut is at readout:** no text generation. One prompt pass; next-token logits from only the
|
| 177 |
-
offered option-label rows of `lm_head` (verified single tokens A–Z, then two-letter labels), computed in
|
| 178 |
-
FP32 and softmaxed over those letters. The rest of the vocabulary is ignored.
|
| 179 |
-
|
| 180 |
-
**Objective:** "calibration-oriented decision post-training" is plain supervised fine-tuning.
|
| 181 |
-
Cross-entropy uses each row's target distribution: code-computed exact probabilities, probability
|
| 182 |
-
targets in teacher-written questions kept after a blind re-solve by that same teacher agreed, and
|
| 183 |
-
one-hot labels otherwise. Choice and yes/no options and letter assignments are
|
| 184 |
-
reshuffled each epoch; score levels keep their order. Jev's RLCD recipe isn't public; we didn't
|
| 185 |
-
use or reproduce it. No RL or preference optimisation.
|
| 186 |
-
|
| 187 |
-
## The climb
|
| 188 |
-
|
| 189 |
-

|
| 190 |
-
|
| 191 |
-
One pre-registered run, one epoch; no other MiMo variant was trained or picked. The gate was DI-S ≥ 55 before training: pass it, then read the full suite and release. DI-S intervals are a few points wide, so this is a narrow gate pass, not a claim of superiority.
|
| 192 |
-
|
| 193 |
-
| Step | DI-S | Full 0.1 | Outside DI-S | Why |
|
| 194 |
-
|---|---:|---:|---:|---|
|
| 195 |
-
| MiMo base, zero-shot | 47.09 | — | — | Baseline before decision training. |
|
| 196 |
-
| **blink-mimo-9b** (123,195 question rows; lr 5e-5; 615 steps) | 55.3 | 56.53 | 56.60 | Program-labelled reasoning, teacher questions and judge data cleared the gate; shipped. |
|
| 197 |
-
|
| 198 |
-
DI-S areas: Knowledge 55.6, Language 66.4, Retrieval 33.0, Tools 70.7, Arts 50.8.
|
| 199 |
|
| 200 |
## Use
|
| 201 |
|
|
@@ -226,13 +178,9 @@ print(out["answers"]["intent"]["probabilities"])
|
|
| 226 |
|
| 227 |
## Run it as a server
|
| 228 |
|
| 229 |
-
`serve.py`
|
| 230 |
-
`GET /v1/models`. From server-side code, point TypeSafe's Python or JavaScript SDK at the server with
|
| 231 |
-
`TYPESAFE_BASE_URL`;
|
| 232 |
-
JevBench's stock `typesafe` adapter and the Decision Index kit's `http` engine still work unchanged.
|
| 233 |
-
`GET /healthz` reports startup checks. `v1.2` changes code only; its weights are identical to `v1.0`.
|
| 234 |
|
| 235 |
-
See the [
|
| 236 |
|
| 237 |
```sh
|
| 238 |
pip install "torch==2.13.0" "transformers==5.17.0" "flash-linear-attention==0.5.2" "accelerate>=1.1.0" safetensors huggingface_hub
|
|
@@ -241,81 +189,57 @@ python blink-mimo-9b/serve.py --model ./blink-mimo-9b --port 8000
|
|
| 241 |
# TypeSafe SDKs: export TYPESAFE_BASE_URL=http://127.0.0.1:8000 TYPESAFE_API_KEY=any
|
| 242 |
```
|
| 243 |
|
| 244 |
-
|
| 245 |
-
|
| 246 |
-
**Or use Docker** from the downloaded folder:
|
| 247 |
|
| 248 |
```sh
|
| 249 |
cd blink-mimo-9b
|
| 250 |
docker build -t blink-mimo-9b . && docker run --rm --gpus all -p 127.0.0.1:8000:8000 blink-mimo-9b
|
| 251 |
```
|
| 252 |
|
| 253 |
-
<details><summary>
|
| 254 |
-
|
| 255 |
-
- `/healthz` reports `weights_verified` (weight, config and tokenizer files listed in `weights.sha256`
|
| 256 |
-
are hashed before serving; a mismatch stops startup), `warmup.repeat_identical` (two matching warm-up
|
| 257 |
-
answers), `kernels` (fast path or slower fallback without flash-linear-attention), `versions` and `hub_offline`.
|
| 258 |
-
- Limits: 255 options per choice, 2–10 score levels, 131,072 input tokens per question and 512 questions
|
| 259 |
-
per request. Over-limit requests get HTTP 422 with the reason; nothing is truncated.
|
| 260 |
-
- `GET /v1/models` lists the one served model with a blank `release_date`. Every request uses that model
|
| 261 |
-
regardless of its `model` field.
|
| 262 |
-
- The server is open by default. Set `--api-key` or `BLINK_API_KEY` to require `Authorization: Bearer <key>` on both API
|
| 263 |
-
routes. Missing or wrong keys get 401; `/healthz` stays open.
|
| 264 |
-
- Error bodies put the reason in `error` and `detail`. Over-limit requests return 422, with nothing cut.
|
| 265 |
-
- Requests run one at a time. Questions are batched; each batch takes one forward pass (large requests
|
| 266 |
-
can take more than one). Serving the downloaded folder or Docker image enables Hugging Face offline
|
| 267 |
-
mode before model loading (`hub_offline: true`). The server doesn't otherwise restrict network access.
|
| 268 |
-
- Set `--batch-window-ms 5` to turn on cross-request batching with a 5 ms collection window, up to
|
| 269 |
-
`--max-batch-requests` requests at a time, which defaults to 16. The window defaults to 0, so requests still
|
| 270 |
-
run one at a time. `--max-queued-requests` lets up to 64 requests wait for a batch by default. Excess requests
|
| 271 |
-
get HTTP 529 with `Retry-After`, so clients should retry. `v1.1` returned HTTP 503 for a full queue. On a
|
| 272 |
-
1,000-request Decision Index sample over HTTP, throughput rose about 15% with 4 concurrent clients and 16%
|
| 273 |
-
with 16. One client saw no gain. Offline runs on long documents showed no meaningful gain. On the Decision
|
| 274 |
-
Index sample, a set of long workflow documents, and the public TypeSafe cases covered by the FP32 reference,
|
| 275 |
-
batched answers passed the same numerical-parity checks against an FP32 reference as one-at-a-time answers,
|
| 276 |
-
covering argmax agreement and probability differences. TypeSafe documents were sent as JSON objects. The FP32
|
| 277 |
-
reference could not run the five longest TypeSafe documents, so parity covered 170 of 354 TypeSafe questions.
|
| 278 |
-
Batched and one-at-a-time answers chose the same option every time on the other 184. A few near-tied answers
|
| 279 |
-
can still flip. Batching arrived in `v1.1`. The current code revision is `v1.2`, with weights identical to
|
| 280 |
-
`v1.0`. Update the two code files in an existing `v1.0` download, then restart with the flag:
|
| 281 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 282 |
```sh
|
| 283 |
hf download thegovind/blink-mimo-9b serve.py blink.py --revision v1.2 --local-dir blink-mimo-9b
|
| 284 |
python blink-mimo-9b/serve.py --model ./blink-mimo-9b --port 8000 --batch-window-ms 5
|
| 285 |
```
|
| 286 |
-
-
|
| 287 |
|
| 288 |
</details>
|
| 289 |
|
| 290 |
-
<details><summary>
|
| 291 |
|
| 292 |
-
|
| 293 |
-
|---|---|
|
| 294 |
-
| Base | [XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B) (MIT declared in the model card; see LICENSE-MiMo.md), full vision-language weights |
|
| 295 |
-
| Adaptation | LoRA r16 / alpha 32 across the language model's attention, Gated DeltaNet and MLP; merged into the full MiMo checkpoint |
|
| 296 |
-
| Readout | Verified single-token labels (A–Z, then two-letter labels); FP32 softmax of next-token logits / T over the offered labels |
|
| 297 |
-
| Temperature | 1.0 (not fitted) |
|
| 298 |
-
| Input limit | 131,072 tokens per question; longest evaluated prompt: 37,906 tokens; longer inputs are refused, never truncated |
|
| 299 |
-
| Runtime | `blink.py` builds prompts and reads out probabilities for text decisions |
|
| 300 |
-
|
| 301 |
-
These are option-conditional model probabilities, not certified chances of being right. Calibration can shift
|
| 302 |
-
across tasks, domains and option sets. For `choice`, `confidence = (p_max − 1/K)/(1 − 1/K)` measures
|
| 303 |
-
concentration, not correctness. For `score`, `score` is the expected 0-based level and `choice` is the
|
| 304 |
-
most likely level.
|
| 305 |
|
| 306 |
-
|
| 307 |
|
| 308 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 309 |
|
| 310 |
### How it was trained
|
| 311 |
|
| 312 |
-
|
| 313 |
-
|
| 314 |
-
|
| 315 |
|
| 316 |
-
|
| 317 |
|
| 318 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 319 |
|
| 320 |
| Source | Licence |
|
| 321 |
|---|---|
|
|
@@ -331,46 +255,30 @@ One pre-registered supervised run, one epoch (lr 5e-5, 615 steps). The mix joins
|
|
| 331 |
| SciQ | CC BY-NC 3.0 |
|
| 332 |
| iSarcasmEval | MIT (upstream repository licence) |
|
| 333 |
| VAST, Humicroedit, OpenBookQA | None stated by source |
|
| 334 |
-
|
|
| 335 |
-
|
| 336 |
-
These are source-repository licences; they don't settle rights in every underlying text.
|
| 337 |
|
| 338 |
-
|
| 339 |
-
|
| 340 |
-
<details><summary>Evaluation notes and limits</summary>
|
| 341 |
|
| 342 |
### Evaluation notes
|
| 343 |
|
| 344 |
-
- **Scorer parity
|
| 345 |
-
- **Selection
|
| 346 |
-
- **Training overlap
|
| 347 |
-
- **
|
| 348 |
-
- **
|
| 349 |
-
- **
|
| 350 |
-
- **
|
| 351 |
-
- **
|
| 352 |
-
- **Teacher documents.** We kept Qwen3.8-27B's documents only if a fresh blind solve by that same teacher agreed with the answer. That's an agreement filter, not independent verification.
|
| 353 |
-
- The repo ships no benchmark items, GPQA text, JevBench items or teacher traces.
|
| 354 |
-
- **No MMLU-Pro or GPQA.** No rows from either were used in this fine-tune; SuperGPQA is a separate source.
|
| 355 |
|
| 356 |
### Limits
|
| 357 |
|
| 358 |
-
- English-centric
|
| 359 |
-
-
|
| 360 |
-
- Text in the state can
|
| 361 |
-
-
|
| 362 |
|
| 363 |
</details>
|
| 364 |
|
| 365 |
-
##
|
| 366 |
-
|
| 367 |
-
The MiMo model card declares MIT without a separate upstream licence file or copyright line (`LICENSE-MiMo.md`); its Qwen/Qwen3.5-9B base is Apache-2.0 (`LICENSE-Qwen`). The blink weights are for **non-commercial research and evaluation
|
| 368 |
-
only** (`LICENSE.md`); commercial use isn't licensed. Training used non-commercial, share-alike and unlicensed
|
| 369 |
-
sources (see the table above). It's unsettled whether their terms reach the weights, so check upstream terms too.
|
| 370 |
-
`blink.py`, `serve.py` and the Dockerfile are Apache-2.0.
|
| 371 |
-
|
| 372 |
-
<details><summary>Credits</summary>
|
| 373 |
|
| 374 |
-
|
| 375 |
-
|
| 376 |
-
</details>
|
|
|
|
| 14 |
language:
|
| 15 |
- en
|
| 16 |
---
|
|
|
|
| 17 |
# blink-mimo-9b
|
| 18 |
|
| 19 |
+
Send a text or JSON `state` and typed questions (`choice`, `noul`, `score`). Each question gets probabilities over its offered options from one forward pass, with no generated text. Long or large multi-question requests may use several batches.
|
|
|
|
|
|
|
| 20 |
|
| 21 |
+

|
|
|
|
|
|
|
| 22 |
|
| 23 |
+
[Space demo](https://huggingface.co/spaces/thegovind/blink) · [Source code](https://github.com/thegovind/blink) · [Docs](https://thegovind.github.io/blink/) · [API](https://thegovind.github.io/blink/api/) · [blink-4b](https://huggingface.co/thegovind/blink-4b) · [blink-27b](https://huggingface.co/thegovind/blink-27b)
|
| 24 |
|
| 25 |
+
## At a glance
|
|
|
|
| 26 |
|
| 27 |
+
| Attribute | Detail |
|
| 28 |
+
|---|---|
|
| 29 |
+
| Base model | [XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B) |
|
| 30 |
+
| Weights size | 18.8 GB bf16 (8.95B text parameters, 9.41B total) |
|
| 31 |
+
| Revision | v1.2 (weights identical to v1.0) |
|
| 32 |
+
| License | Non-commercial research ([LICENSE.md](LICENSE.md)); code Apache-2.0 |
|
| 33 |
|
| 34 |
## Results
|
| 35 |
|
| 36 |
### Decision Index 0.2 (local run)
|
| 37 |
|
| 38 |
+
Decision Index numbers are local runs of the official kit (commit 19ad28e, 2026-09-25), not leaderboard submissions. The 0.2 run is descriptive: known training exposure stays in the scores, with no leaderboard-style penalty, so it isn't ranked. The fine-tune didn't use MMLU-Pro or GPQA as direct data sources; screening found 176 SuperGPQA training rows matching added-request text.
|
| 39 |
|
| 40 |
| Balanced skill | Balanced raw | Breadth skill | Without MMLU-Pro |
|
| 41 |
|---:|---:|---:|---:|
|
| 42 |
| 43.36 | 57.24 | 42.38 | 42.84 |
|
| 43 |
|
|
|
|
|
|
|
| 44 |
<details><summary>Extra tables and method</summary>
|
| 45 |
|
| 46 |
| Area | Number of benchmarks | Skill | Raw |
|
|
|
|
| 51 |
| Tools & Automation | 6 | 57.0 | 65.2 |
|
| 52 |
| Arts & Human Taste | 7 | 32.0 | 50.2 |
|
| 53 |
|
| 54 |
+
The seven benchmarks added in 0.2:
|
| 55 |
|
| 56 |
| Benchmark | Metric | Requests | Answered | Raw | Skill |
|
| 57 |
|---|---|---:|---:|---:|---:|
|
|
|
|
| 63 |
| When2Call MCQ | accuracy | 3,652 | 3,652 | 64.5 | 52.7 |
|
| 64 |
| New Yorker caption matching | accuracy | 528 | 528 | 62.5 | 53.1 |
|
| 65 |
|
| 66 |
+
- All 151,034 scoreable requests scored across 44 benchmarks (40 count toward index across 5 areas).
|
| 67 |
+
- Requests shared with 0.1 reuse 0.1 predictions; the 30,419 added requests ran with the same frozen evaluation setup as 0.1, the evaluated adapter loaded on the MiMo base, which the published graft matched on JevBench's 231 public items (see Evaluation notes), at temperature 1.0.
|
| 68 |
+
- "Without MMLU-Pro" drops MMLU-Pro, averages the other nine Knowledge benchmarks, and keeps five equal areas. It is a sensitivity check, not a score free of training effects.
|
| 69 |
+
- Point estimates without significance, calibration, or latency claims. Do not compare with 0.1 numbers.
|
| 70 |
|
| 71 |
+
Training-row text matches in added requests:
|
| 72 |
|
| 73 |
| Training stage | Rows in the stage | Rows matching added-request text | From MMLU-Pro | From SuperGPQA | Other |
|
| 74 |
|---|---:|---:|---:|---:|---:|
|
| 75 |
| MiMo | 123,195 | 180 | 0 | 176 | 4 |
|
| 76 |
|
| 77 |
+
Screened for exact normalized strings of at least 30 characters shared by training rows and added requests (strings in 20 or more requests treated as templates). Counts are training rows, not unique test questions.
|
|
|
|
|
|
|
| 78 |
|
| 79 |
</details>
|
| 80 |
|
| 81 |
### Decision Index 0.1 (archived edition)
|
| 82 |
|
| 83 |
+
Decision Index numbers are local runs of the official kit, not leaderboard submissions. Comparison rows use the 2026-09-22 leaderboard snapshot.
|
| 84 |
|
| 85 |
+
Training overlap: Public train splits included ContractNLI, iSarcasmEval and VAST (the whole Language area), plus Amazon ESCI and Humicroedit. With Language set to Jev's score, this model's index would be 54.96 vs Jev's 59.51. That's arithmetic, not an ablation or a like-for-like comparison with models trained only on synthetic data.
|
| 86 |
|
| 87 |
| Model | Size class | Decision Index 0.1 | Skill | Breadth |
|
| 88 |
|---|---|---:|---:|---:|
|
| 89 |
+
| **blink-mimo-9b** | 9B | 56.53 | 42.17 | 40.37 |
|
| 90 |
| Jev 1.13.0 | closed | 59.51 | 46.26 | 44.79 |
|
| 91 |
| Jevfire | 27B | 55.74 | 40.86 | 39.45 |
|
| 92 |
| JoshuaSP diffusiongemma (open-jev) | 26B-A4B | 55.56 | 40.84 | 39.19 |
|
|
|
|
| 94 |
| Kev 9B | 9B | 50.48 | 32.96 | 30.54 |
|
| 95 |
| Kev 4B | 4B | 47.43 | 28.86 | 25.67 |
|
| 96 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 97 |
| Area | blink-mimo-9b | Jev 1.13.0 |
|
| 98 |
|---|---:|---:|
|
| 99 |
| Knowledge & Reasoning | 55.1 | 68.8 |
|
|
|
|
| 130 |
|
| 131 |
### JevBench: public items only
|
| 132 |
|
| 133 |
+
The JevBench numbers are public-item development proxies, not official scores, and claim no rank or parity.
|
| 134 |
+
|
| 135 |
| Model | Easy | Standard | Hard | Hard ECE | Official score |
|
| 136 |
|---|---:|---:|---:|---:|---|
|
| 137 |
| **blink-mimo-9b** | 48/48 | 70/72 | 77/111 (0.694) | 0.136 | not submitted |
|
| 138 |
|
| 139 |
+
Hard ECE is 0.136 vs blink-4b's 0.067. Weakest public hard cases are date/number reasoning, trade-offs, and long policies.
|
| 140 |
|
| 141 |
+
### Next-click probe
|
| 142 |
+
|
| 143 |
+
Sampling on 500 Multimodal-Mind2Web test steps with 5 offered elements (20% chance baseline); not an official evaluation.
|
| 144 |
|
| 145 |
| Input | blink-mimo-9b | MiMo base | blink-4b |
|
| 146 |
|---|---:|---:|---:|
|
| 147 |
| Page text | 53.8% | 49.0% | 55.4% |
|
| 148 |
| Screenshot | 48.4% | 43.0% | — |
|
| 149 |
|
| 150 |
+
`blink.py` and `serve.py` use its text side only. Screenshot results above come from the separate probe harness, not `blink.py`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 151 |
|
| 152 |
## Use
|
| 153 |
|
|
|
|
| 178 |
|
| 179 |
## Run it as a server
|
| 180 |
|
| 181 |
+
`serve.py` provides TypeSafe server-side SDK compatibility at `POST /v1/systemone` and lists the served model at `GET /v1/models`. Set `TYPESAFE_BASE_URL` to connect TypeSafe Python or JavaScript SDKs; JevBench's `typesafe` adapter and Decision Index's `http` engine work unchanged. `GET /healthz` reports startup checks. `v1.2` updates code only; weights are identical to `v1.0`.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 182 |
|
| 183 |
+
See the [API reference](https://thegovind.github.io/blink/api/) for full details.
|
| 184 |
|
| 185 |
```sh
|
| 186 |
pip install "torch==2.13.0" "transformers==5.17.0" "flash-linear-attention==0.5.2" "accelerate>=1.1.0" safetensors huggingface_hub
|
|
|
|
| 189 |
# TypeSafe SDKs: export TYPESAFE_BASE_URL=http://127.0.0.1:8000 TYPESAFE_API_KEY=any
|
| 190 |
```
|
| 191 |
|
| 192 |
+
Or run with Docker:
|
|
|
|
|
|
|
| 193 |
|
| 194 |
```sh
|
| 195 |
cd blink-mimo-9b
|
| 196 |
docker build -t blink-mimo-9b . && docker run --rm --gpus all -p 127.0.0.1:8000:8000 blink-mimo-9b
|
| 197 |
```
|
| 198 |
|
| 199 |
+
<details><summary>Server options, batching, and limits</summary>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 200 |
|
| 201 |
+
- **API key:** Optional. Set `--api-key` or `BLINK_API_KEY` to require `Authorization: Bearer <key>` on API routes. Missing or invalid keys return 401; `/healthz` stays open.
|
| 202 |
+
- **Limits and errors:** 255 options per choice, 2–10 score levels, 131,072 input tokens per question (longest evaluated prompt: 37,906 tokens), 512 questions per request. Error bodies set `error` and `detail`. Requests exceeding limits return 422 with the reason; inputs are not truncated.
|
| 203 |
+
- **Model listing:** `GET /v1/models` lists the served model with a blank `release_date`.
|
| 204 |
+
- **Batching:** Set `--batch-window-ms 5` to enable cross-request batching (default 0 runs requests one at a time), up to `--max-batch-requests` (default 16). `--max-queued-requests` holds up to 64 waiting requests; excess requests receive HTTP 529 with `Retry-After` (returned 503 in v1.1).
|
| 205 |
+
- **FP32 reference coverage caveat:** On the Decision Index sample, a set of long workflow documents, and the public TypeSafe cases covered by the FP32 reference, batched answers passed the same numerical-parity checks against an FP32 reference as one-at-a-time answers, covering argmax agreement and probability differences. The FP32 reference could not run the 5 longest TypeSafe documents, so parity covered 170 of 354 TypeSafe questions; batched and one-at-a-time runs chose the same option on the other 184. A few near-tied answers can still flip.
|
| 206 |
+
- **Batching command:**
|
| 207 |
```sh
|
| 208 |
hf download thegovind/blink-mimo-9b serve.py blink.py --revision v1.2 --local-dir blink-mimo-9b
|
| 209 |
python blink-mimo-9b/serve.py --model ./blink-mimo-9b --port 8000 --batch-window-ms 5
|
| 210 |
```
|
| 211 |
+
- **Weights:** 18.8 GB in bf16 (8.95B text parameters, 9.41B total).
|
| 212 |
|
| 213 |
</details>
|
| 214 |
|
| 215 |
+
<details><summary>Details: Architecture, training, evaluation, and limits</summary>
|
| 216 |
|
| 217 |
+
### Network architecture
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 218 |
|
| 219 |
+

|
| 220 |
|
| 221 |
+
- **Backbone:** MiMo-V2.6-Distill-Qwen-9B (upstream revision 2367e86), a Qwen3.5-9B fine-tune. 32 decoder layers (24 Gated DeltaNet, 8 full-attention), hidden dimension 4096, untied embeddings.
|
| 222 |
+
- **Vision:** 27 encoder blocks, hidden 1152 projected to 4096; all 333 vision tensors unchanged.
|
| 223 |
+
- **Tuned:** 43.3M LoRA parameters; all 248 targeted tensors changed, all 179 other language-model tensors bit-identical to MiMo.
|
| 224 |
+
- **LoRA targets (rank 16, alpha 32, every language-model layer):** full-attention `q_proj`, `k_proj`, `v_proj`, `o_proj`; Gated DeltaNet `in_proj_qkv`, `in_proj_z`, `in_proj_a`, `in_proj_b`, `out_proj`; and every MLP's `gate_proj`, `up_proj`, `down_proj`. Token embeddings, all norms and `lm_head` stayed frozen. The trained adapters were merged into the text weights.
|
| 225 |
+
- **Readout:** Single forward pass without text generation. Next-token logits from offered option rows of `lm_head` (single tokens A–Z, then two-letter labels), computed in FP32 and softmaxed over offered letters. Temperature 1.0 (not fitted). These are option-conditional model probabilities, not certified chances of being right.
|
| 226 |
|
| 227 |
### How it was trained
|
| 228 |
|
| 229 |
+

|
| 230 |
+
|
| 231 |
+
One pre-registered run and one epoch (123,195 rows, lr 5e-5, 615 steps). Supervised fine-tuning with cross-entropy on target distributions (exact probabilities, agreed teacher targets, or one-hot labels). Options and letter assignments reshuffled each epoch; score levels preserve order. No RL or preference optimization.
|
| 232 |
|
| 233 |
+
The DI-S gate required DI-S >= 55 before reading the full suite. DI-S scored 55.3 (Knowledge 55.6, Language 66.4, Retrieval 33.0, Tools 70.7, Arts 50.8), clearing the gate. Outside DI-S, the model scored 56.60 across the 129,422 requests not used for selection.
|
| 234 |
|
| 235 |
+
| Step | DI-S | Full 0.1 | Outside DI-S | Note |
|
| 236 |
+
|---|---:|---:|---:|---|
|
| 237 |
+
| MiMo base, zero-shot | 47.09 | — | — | Baseline before decision training. |
|
| 238 |
+
| **blink-mimo-9b** (123,195 question rows; lr 5e-5; 615 steps) | 55.3 | 56.53 | 56.60 | Cleared DI-S gate; shipped. |
|
| 239 |
+
|
| 240 |
+
Training mix (123,195 rows): 61,394 public-source, 23,894 program-generated reasoning, 12,000 decision worlds, 7,860 teacher-written rows (Qwen3.8-27B), 7,000 judge-style, 6,000 chess move choices, and 5,047 exact-probability worlds.
|
| 241 |
+
|
| 242 |
+
### Data sources and licenses
|
| 243 |
|
| 244 |
| Source | Licence |
|
| 245 |
|---|---|
|
|
|
|
| 255 |
| SciQ | CC BY-NC 3.0 |
|
| 256 |
| iSarcasmEval | MIT (upstream repository licence) |
|
| 257 |
| VAST, Humicroedit, OpenBookQA | None stated by source |
|
| 258 |
+
| Code-generated worlds and teacher-written documents (Qwen3.8-27B) | See LICENSE.md |
|
|
|
|
|
|
|
| 259 |
|
| 260 |
+
Source repository licenses do not settle rights in underlying texts.
|
|
|
|
|
|
|
| 261 |
|
| 262 |
### Evaluation notes
|
| 263 |
|
| 264 |
+
- **Scorer parity:** `blink.py` and the lab scorer agree within 1e-7 on this checkpoint (p99 |Δp| 1.4e-8). Published weights match the evaluated adapter on 231 public JevBench items (0 argmax changes, max |Δp| 0.0). Median per-request latency on 1,000 suite requests was 68.3 ms via HTTP vs 66.9 ms in-process. `serve.py` matched evaluation answers on public items (easy 48/48, standard 70/72, hard 77/111; 0 argmax changes; max |Δp| about 0.03).
|
| 265 |
+
- **Selection:** Prompt format came from earlier DI-S reads. The DI-S sample served as a pre-registered gate. The remaining 129,422 suite requests outside DI-S were not used for selection and scored 56.60.
|
| 266 |
+
- **Training overlap:** Public train splits in the 0.1 index: ContractNLI, iSarcasmEval, VAST, Amazon ESCI, Humicroedit, ChessBench (training positions; none of the 5,000 test positions), GSM8K train split. ANLI and BANKING77 train splits were also used (in 0.1 suite but outside index; in 0.2 panel). Public-source data included training and development partitions.
|
| 267 |
+
- **Mixture audit:** Every question row was screened against the 132,422 requests in 0.1 and 231 public JevBench items for exact matches (>=30 normalized characters) and shared 13-word passages (ignoring strings appearing in 20 or more requests as templates). No public JevBench items or chess positions matched.
|
| 268 |
+
- **Overlap findings:** 16 BANKING77/VAST training rows share a 13-word passage with 31 suite requests (23 VAST, 8 BANKING77). Two VAST training posts are near-duplicates of a test post, but these rows had no exact normalised-text match under the audit. Dropping these requests leaves the index at 56.53 (VAST 0.7805 -> 0.7803); BANKING77 is outside the 0.1 index.
|
| 269 |
+
- **Audit limits:** The 13-word check did not search long-document bodies or option text. Semantic or pretraining overlap cannot be ruled out. Private JevBench items were unavailable to check.
|
| 270 |
+
- **Generated data:** Program-computed labels were used for CRUXEval-style and CLadder-style questions; no benchmark items were used. Suite GSM8K distractors were not reused. Teacher documents (Qwen3.8-27B) were kept only when a blind re-solve by the teacher agreed.
|
| 271 |
+
- **MMLU-Pro and GPQA:** Neither benchmark was used in training; SuperGPQA is a separate source.
|
|
|
|
|
|
|
|
|
|
| 272 |
|
| 273 |
### Limits
|
| 274 |
|
| 275 |
+
- **English-centric:** Training included Arabic iSarcasmEval rows (Arabic task A scored 0.321, task C pairs 0.79 on 0.1). Multilingual performance is not established.
|
| 276 |
+
- **No text generation:** Does not chat or explain answers.
|
| 277 |
+
- **Prompt sensitivity:** Text in the state can influence the answer.
|
| 278 |
+
- **Reasoning gaps:** Weakest public hard cases are date/number reasoning, trade-offs, and long policies.
|
| 279 |
|
| 280 |
</details>
|
| 281 |
|
| 282 |
+
## License
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 283 |
|
| 284 |
+
Code (`blink.py`, `serve.py`, Dockerfile): Apache-2.0. Weights: non-commercial research only; see [LICENSE.md](LICENSE.md). The MiMo model card declares MIT without a separate upstream licence file or copyright line ([LICENSE-MiMo.md](LICENSE-MiMo.md)); its Qwen/Qwen3.5-9B base is Apache-2.0 ([LICENSE-Qwen](LICENSE-Qwen)).
|
|
|
|
|
|