drex-max-v1 / README.md
NischayDnk's picture
Duplicate from NischayDnk/Drex-Max
753f55d
|
Raw History Blame
5.3 kB
metadata
base_model: Qwen/Qwen3.8-27B
base_model_relation: finetune
license: other
license_name: nace-ai-open-rail-m
license_link: LICENSE
library_name: pytorch
pipeline_tag: zero-shot-classification
tags:
  - decision-model
  - system-one
  - multimodal
  - drex
  - qwen3.8
private: true

Drex-Max

A multimodal decision model on a fine-tuned Qwen3.8-27B hybrid backbone. It reads a state (text, JSON, and optional images) plus typed questions (choice, noul, ordinal score) and returns a probability for every option in one forward pass per question. Nothing is generated.

The 16 full-attention layers are non-causal: every token can attend to the whole sequence. The other 48 layers are Gated DeltaNet linear-attention blocks and stay left-to-right. A dedicated 255-way readout sits on the last hidden state. Images go through the native vision tower (up to 1.6 MP each) and are visible to every question on the request.

Architecture

Base Qwen3.8-27B, then decision fine-tune
Layers 64: 48 Gated DeltaNet (linear attention) + 16 full attention (every 4th)
Hidden / MLP 5120 / 17408
Full attention 24 query heads, 4 KV heads, head dim 256 — non-causal
Linear attention causal Gated DeltaNet scan
Vision native tower, several images per request
Decision head readout.safetensors [255 × 5120], last-token pooling
Options per pass up to 255
Context 262,144 tokens
Size 49 GB (bf16)

Default causal chat inference does not reproduce these scores. Use the bundled loader.

Inference

pip install -r requirements.txt
python inference.py --model NischayDnk/Drex-Max
python inference.py --model NischayDnk/Drex-Max --request req.json
from pathlib import Path
import sys
from huggingface_hub import snapshot_download

ckpt = Path(snapshot_download("NischayDnk/Drex-Max"))
sys.path.insert(0, str(ckpt / "code"))
from drex_max.model import DecisionModel, answer

model = DecisionModel(ckpt, device="cuda")
row = {
    "state": {"ticket": "I was charged twice for order A-104."},
    "question": {
        "type": "choice",
        "instructions": "Which team handles this?",
        "criteria": {
            "billing": "Charges and refunds",
            "shipping": "Delivery",
            "other": None,
        },
    },
    # "images": ["receipt.jpg"],
}
print(answer(row["question"], model.predict([row])[0]))

Request shape (System One):

{"state": {"ticket": "I was charged twice for order A-104."},
 "images": ["receipt.jpg"],
 "questions": {"team": {"type": "choice", "instructions": "Which team handles this?",
                        "criteria": {"billing": "Charges and refunds", "shipping": "Delivery", "other": null}},
               "refund": {"type": "noul", "instructions": "Does the customer ask for a refund?"}}}

Needs a CUDA GPU with room for ~49 GiB of weights plus activations. Python 3.12+, transformers==5.17.0.

Benchmarks

Decision Index 0.3 (official kit, public suite)

140,178 scored requests. Area columns are chance-corrected skill × 100. Unanswered requests count as wrong.

Model Index Raw Knowledge Language Retrieval Tools Arts
Drex-Max 67.59 75.48 57.2 72.2 68.8 82.3 53.3
Drex v2.0 (0.2.1) 60.78 69.70 47.7 65.9 62.9 76.0 49.1
Jev 1.13.0 (0.2.1) 57.91 68.09 51.4 62.0 55.4 75.1 37.7

Drex v2.0 and Jev are the published 0.2.1 public index. Drex-Max is the official 0.3 public index (ForecastBench retired, GSM8K added). Same scoring rule: chance-corrected skill, area-balanced.

Decision Index

Areas

JevBench (231 public items)

Choice = top option, yes/no = noul ≥ 0.5, ordinal = top level.

Model Accuracy Easy Standard Hard
Drex-Max 92.2% (213/231) 100% 100% 83.8%
Drex v2.0 85.7% (198/231) 100% 95.8% 73.0%
Jev 1.13.0 86.6% (200/231) 100% 98.6% 73.0%

Decision Bench v4

Model Accuracy
Drex-Max 94.30% (1010/1071)
Drex v2.0 —

Vision (public hold-out)

400 held-out items from the Decision Index vision-board public sets (CV-Bench, BLINK, RealWorldQA, R-Bench-M, MMMU-Pro vision). Skill is chance-corrected accuracy.

Accuracy Skill
Drex-Max 71.5% (286/400) 61.9
Bench n Acc Skill
CV-Bench 80 93.8 88.9
RealWorldQA 80 82.5 73.2
BLINK 80 67.5 48.3
MMMU-Pro vision 80 57.5 51.7
R-Bench-M 80 56.3 47.5

Vision

Files

model-*.safetensors, config.json Qwen3.8-27B backbone after decision fine-tune (bf16)
readout.safetensors 255-way decision head
decision_config.json option codes, temperature, non-causal full attention
code/drex_max/ loader: hybrid mask + readout + images
inference.py System One scorer
decision_index.png, areas.png, vision.png card plots