--- base_model: Qwen/Qwen3.8-27B base_model_relation: finetune license: other license_name: nace-ai-open-rail-m license_link: LICENSE library_name: pytorch pipeline_tag: zero-shot-classification tags: - decision-model - system-one - multimodal - drex - qwen3.8 private: true --- # Drex-Max A **multimodal decision model** on a fine-tuned [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) hybrid backbone. It reads a state (text, JSON, and optional images) plus typed questions (`choice`, `noul`, ordinal `score`) and returns a probability for every option in one forward pass per question. Nothing is generated. The 16 full-attention layers are **non-causal**: every token can attend to the whole sequence. The other 48 layers are **Gated DeltaNet linear-attention** blocks and stay left-to-right. A dedicated 255-way readout sits on the last hidden state. Images go through the native vision tower (up to 1.6 MP each) and are visible to every question on the request. ## Architecture | | | |---|---| | Base | Qwen3.8-27B, then decision fine-tune | | Layers | 64: 48 Gated DeltaNet (linear attention) + 16 full attention (every 4th) | | Hidden / MLP | 5120 / 17408 | | Full attention | 24 query heads, 4 KV heads, head dim 256 — **non-causal** | | Linear attention | causal Gated DeltaNet scan | | Vision | native tower, several images per request | | Decision head | `readout.safetensors` `[255 × 5120]`, last-token pooling | | Options per pass | up to 255 | | Context | 262,144 tokens | | Size | 49 GB (bf16) | Default causal chat inference does not reproduce these scores. Use the bundled loader. ## Inference ```bash pip install -r requirements.txt python inference.py --model NischayDnk/Drex-Max python inference.py --model NischayDnk/Drex-Max --request req.json ``` ```python from pathlib import Path import sys from huggingface_hub import snapshot_download ckpt = Path(snapshot_download("NischayDnk/Drex-Max")) sys.path.insert(0, str(ckpt / "code")) from drex_max.model import DecisionModel, answer model = DecisionModel(ckpt, device="cuda") row = { "state": {"ticket": "I was charged twice for order A-104."}, "question": { "type": "choice", "instructions": "Which team handles this?", "criteria": { "billing": "Charges and refunds", "shipping": "Delivery", "other": None, }, }, # "images": ["receipt.jpg"], } print(answer(row["question"], model.predict([row])[0])) ``` Request shape (System One): ```json {"state": {"ticket": "I was charged twice for order A-104."}, "images": ["receipt.jpg"], "questions": {"team": {"type": "choice", "instructions": "Which team handles this?", "criteria": {"billing": "Charges and refunds", "shipping": "Delivery", "other": null}}, "refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"}}} ``` Needs a CUDA GPU with room for ~49 GiB of weights plus activations. Python 3.12+, `transformers==5.17.0`. ## Benchmarks ### Decision Index 0.3 (official kit, public suite) 140,178 scored requests. Area columns are chance-corrected skill × 100. Unanswered requests count as wrong. | Model | Index | Raw | Knowledge | Language | Retrieval | Tools | Arts | |---|---|---|---|---|---|---|---| | **Drex-Max** | **67.59** | **75.48** | **57.2** | **72.2** | **68.8** | **82.3** | **53.3** | | Drex v2.0 (0.2.1) | 60.78 | 69.70 | 47.7 | 65.9 | 62.9 | 76.0 | 49.1 | | Jev 1.13.0 (0.2.1) | 57.91 | 68.09 | 51.4 | 62.0 | 55.4 | 75.1 | 37.7 | Drex v2.0 and Jev are the published 0.2.1 public index. Drex-Max is the official 0.3 public index (ForecastBench retired, GSM8K added). Same scoring rule: chance-corrected skill, area-balanced.

Decision Index

Areas

### JevBench (231 public items) Choice = top option, yes/no = noul ≥ 0.5, ordinal = top level. | Model | Accuracy | Easy | Standard | Hard | |---|---|---|---|---| | **Drex-Max** | **92.2%** (213/231) | 100% | 100% | **83.8%** | | Drex v2.0 | 85.7% (198/231) | 100% | 95.8% | 73.0% | | Jev 1.13.0 | 86.6% (200/231) | 100% | 98.6% | 73.0% | ### Decision Bench v4 | Model | Accuracy | |---|---| | **Drex-Max** | **94.30%** (1010/1071) | | Drex v2.0 | — | ### Vision (public hold-out) 400 held-out items from the Decision Index vision-board public sets (CV-Bench, BLINK, RealWorldQA, R-Bench-M, MMMU-Pro vision). Skill is chance-corrected accuracy. | | Accuracy | Skill | |---|---|---| | **Drex-Max** | **71.5%** (286/400) | **61.9** | | Bench | n | Acc | Skill | |---|---|---|---| | CV-Bench | 80 | 93.8 | 88.9 | | RealWorldQA | 80 | 82.5 | 73.2 | | BLINK | 80 | 67.5 | 48.3 | | MMMU-Pro vision | 80 | 57.5 | 51.7 | | R-Bench-M | 80 | 56.3 | 47.5 |

Vision

## Files | | | |---|---| | `model-*.safetensors`, `config.json` | Qwen3.8-27B backbone after decision fine-tune (bf16) | | `readout.safetensors` | 255-way decision head | | `decision_config.json` | option codes, temperature, non-causal full attention | | `code/drex_max/` | loader: hybrid mask + readout + images | | `inference.py` | System One scorer | | `decision_index.png`, `areas.png`, `vision.png` | card plots |