|
Download README.md from nace-ai/drex-max-v1: direct link, hf CLI and curl.
- Browser
- Download file 5.3 kB
-
https://huggingface.co/nace-ai/drex-max-v1/resolve/753f55d919843471eaa5518d9a9d5ec53e574bcb/README.md
- Command line
-
hf download hf://nace-ai/drex-max-v1@753f55d919843471eaa5518d9a9d5ec53e574bcb/README.md
-
curl -L -o README.md https://huggingface.co/nace-ai/drex-max-v1/resolve/753f55d919843471eaa5518d9a9d5ec53e574bcb/README.md
5.3 kB
| base_model: Qwen/Qwen3.8-27B | |
| base_model_relation: finetune | |
| license: other | |
| license_name: nace-ai-open-rail-m | |
| license_link: LICENSE | |
| library_name: pytorch | |
| pipeline_tag: zero-shot-classification | |
| tags: | |
| - decision-model | |
| - system-one | |
| - multimodal | |
| - drex | |
| - qwen3.8 | |
| private: true | |
| # Drex-Max | |
| A **multimodal decision model** on a fine-tuned [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) hybrid backbone. It reads a state (text, JSON, and optional images) plus typed questions (`choice`, `noul`, ordinal `score`) and returns a probability for every option in one forward pass per question. Nothing is generated. | |
| The 16 full-attention layers are **non-causal**: every token can attend to the whole sequence. The other 48 layers are **Gated DeltaNet linear-attention** blocks and stay left-to-right. A dedicated 255-way readout sits on the last hidden state. Images go through the native vision tower (up to 1.6 MP each) and are visible to every question on the request. | |
| ## Architecture | |
| | | | | |
| |---|---| | |
| | Base | Qwen3.8-27B, then decision fine-tune | | |
| | Layers | 64: 48 Gated DeltaNet (linear attention) + 16 full attention (every 4th) | | |
| | Hidden / MLP | 5120 / 17408 | | |
| | Full attention | 24 query heads, 4 KV heads, head dim 256 — **non-causal** | | |
| | Linear attention | causal Gated DeltaNet scan | | |
| | Vision | native tower, several images per request | | |
| | Decision head | `readout.safetensors` `[255 × 5120]`, last-token pooling | | |
| | Options per pass | up to 255 | | |
| | Context | 262,144 tokens | | |
| | Size | 49 GB (bf16) | | |
| Default causal chat inference does not reproduce these scores. Use the bundled loader. | |
| ## Inference | |
| ```bash | |
| pip install -r requirements.txt | |
| python inference.py --model NischayDnk/Drex-Max | |
| python inference.py --model NischayDnk/Drex-Max --request req.json | |
| ``` | |
| ```python | |
| from pathlib import Path | |
| import sys | |
| from huggingface_hub import snapshot_download | |
| ckpt = Path(snapshot_download("NischayDnk/Drex-Max")) | |
| sys.path.insert(0, str(ckpt / "code")) | |
| from drex_max.model import DecisionModel, answer | |
| model = DecisionModel(ckpt, device="cuda") | |
| row = { | |
| "state": {"ticket": "I was charged twice for order A-104."}, | |
| "question": { | |
| "type": "choice", | |
| "instructions": "Which team handles this?", | |
| "criteria": { | |
| "billing": "Charges and refunds", | |
| "shipping": "Delivery", | |
| "other": None, | |
| }, | |
| }, | |
| # "images": ["receipt.jpg"], | |
| } | |
| print(answer(row["question"], model.predict([row])[0])) | |
| ``` | |
| Request shape (System One): | |
| ```json | |
| {"state": {"ticket": "I was charged twice for order A-104."}, | |
| "images": ["receipt.jpg"], | |
| "questions": {"team": {"type": "choice", "instructions": "Which team handles this?", | |
| "criteria": {"billing": "Charges and refunds", "shipping": "Delivery", "other": null}}, | |
| "refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"}}} | |
| ``` | |
| Needs a CUDA GPU with room for ~49 GiB of weights plus activations. Python 3.12+, `transformers==5.17.0`. | |
| ## Benchmarks | |
| ### Decision Index 0.3 (official kit, public suite) | |
| 140,178 scored requests. Area columns are chance-corrected skill × 100. Unanswered requests count as wrong. | |
| | Model | Index | Raw | Knowledge | Language | Retrieval | Tools | Arts | | |
| |---|---|---|---|---|---|---|---| | |
| | **Drex-Max** | **67.59** | **75.48** | **57.2** | **72.2** | **68.8** | **82.3** | **53.3** | | |
| | Drex v2.0 (0.2.1) | 60.78 | 69.70 | 47.7 | 65.9 | 62.9 | 76.0 | 49.1 | | |
| | Jev 1.13.0 (0.2.1) | 57.91 | 68.09 | 51.4 | 62.0 | 55.4 | 75.1 | 37.7 | | |
| Drex v2.0 and Jev are the published 0.2.1 public index. Drex-Max is the official 0.3 public index (ForecastBench retired, GSM8K added). Same scoring rule: chance-corrected skill, area-balanced. | |
| <p><img src="decision_index.png" alt="Decision Index" width="720"></p> | |
| <p><img src="areas.png" alt="Areas" width="720"></p> | |
| ### JevBench (231 public items) | |
| Choice = top option, yes/no = noul ≥ 0.5, ordinal = top level. | |
| | Model | Accuracy | Easy | Standard | Hard | | |
| |---|---|---|---|---| | |
| | **Drex-Max** | **92.2%** (213/231) | 100% | 100% | **83.8%** | | |
| | Drex v2.0 | 85.7% (198/231) | 100% | 95.8% | 73.0% | | |
| | Jev 1.13.0 | 86.6% (200/231) | 100% | 98.6% | 73.0% | | |
| ### Decision Bench v4 | |
| | Model | Accuracy | | |
| |---|---| | |
| | **Drex-Max** | **94.30%** (1010/1071) | | |
| | Drex v2.0 | — | | |
| ### Vision (public hold-out) | |
| 400 held-out items from the Decision Index vision-board public sets (CV-Bench, BLINK, RealWorldQA, R-Bench-M, MMMU-Pro vision). Skill is chance-corrected accuracy. | |
| | | Accuracy | Skill | | |
| |---|---|---| | |
| | **Drex-Max** | **71.5%** (286/400) | **61.9** | | |
| | Bench | n | Acc | Skill | | |
| |---|---|---|---| | |
| | CV-Bench | 80 | 93.8 | 88.9 | | |
| | RealWorldQA | 80 | 82.5 | 73.2 | | |
| | BLINK | 80 | 67.5 | 48.3 | | |
| | MMMU-Pro vision | 80 | 57.5 | 51.7 | | |
| | R-Bench-M | 80 | 56.3 | 47.5 | | |
| <p><img src="vision.png" alt="Vision" width="720"></p> | |
| ## Files | |
| | | | | |
| |---|---| | |
| | `model-*.safetensors`, `config.json` | Qwen3.8-27B backbone after decision fine-tune (bf16) | | |
| | `readout.safetensors` | 255-way decision head | | |
| | `decision_config.json` | option codes, temperature, non-causal full attention | | |
| | `code/drex_max/` | loader: hybrid mask + readout + images | | |
| | `inference.py` | System One scorer | | |
| | `decision_index.png`, `areas.png`, `vision.png` | card plots | | |