Instructions to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0
# Run inference directly in the terminal:
llama cli -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0
# Run inference directly in the terminal:
llama cli -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0
# Run inference directly in the terminal:
./build/bin/llama cli -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Use Docker

docker model run hf.co/gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Llama new
LM Studio
Jan

vLLM

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "gbuzhf/Nex-N2.5-mini-Sangreal-ICE"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gbuzhf/Nex-N2.5-mini-Sangreal-ICE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Ollama
How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with Ollama:
```
ollama run hf.co/gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0
```
Unsloth Desktop

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with Docker Model Runner:
```
docker model run hf.co/gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0
```

Lemonade

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Run and chat with the model

lemonade run user.Nex-N2.5-mini-Sangreal-ICE-Q8_0

List all available models

lemonade list

Hermes Agent

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Nex-N2.5-mini — Sangreal ICE

Four ICE 1.6 tiers of nex-agi's Nex-N2.5-mini (35B total, 3B active, Qwen3.5-MoE architecture), quantized against Sangreal — a purpose-built 77-bucket calibration corpus — with an importance matrix measured on Nex itself, and shipping Qwen-Sharp v22.5.0-arrow4.0 — our agent-tuned chat template, a heavily modified derivative of peculiar-ragdoll's Qwen-Sharp (spec below). Vision works through Nex's own projector, mmproj-Nex-N2.5-mini-Q8_0.gguf.

At a glance — against bartowski at the same size

Mean KLD to BF16, one session. Size-matched = bartowski's imatrix ladder for the same model, interpolated (log-linear) at each ICE file's exact byte size; negative = ICE closer to BF16. Paired = the same 64 chunks scored for both files against the same reference.

tier	size	code @2K	code @16K	wiki @2K	paired, code @2K (t, chunks ICE better)
15G-ICE	14.51 GB	-15.1%	-7.8%	-12.5%	vs Q2_K (13.84 GB): -30.5% (t -9.5, 64/64); vs IQ3_XXS (15.58 GB): +16.5% (t +6.7, 4/64)
19G-ICE	18.47 GB	-16.1%	-12.7%	-11.2%	vs Q3_K_L (18.27 GB): -15.9% (t -5.0, 57/64); vs IQ3_M (18.96 GB): -16.8% (t -4.9, 58/64)
23G-ICE	22.52 GB	-11.0%	-7.1%	-6.6%	vs Q4_K_M (22.32 GB): -12.0% (t -6.0, 58/64); vs Q4_K_L (24.11 GB): -2.7% (t -1.3, 31/64)
25G-ICE	24.53 GB	-11.3%	-0.4%	-4.5%	vs Q4_K_L (24.11 GB): -18.1% (t -5.0, 62/64); vs Q5_K_S (25.14 GB): -0.4% (t -0.1, 29/64)

Code at 16K context (code.test.raw, n_ctx 16384 × 8), mean KLD to BF16 vs file size; lower-left is better. Raw numbers in Table 2 below and in measurements/2026-10-03/.

Demo — Coral Dragon Reef Rescue

A complete browser game — one self-contained index.html, no assets, no network calls — generated in a single shot by the sibling OrcaRouter build of this quant, Nex-N2.5-mini-OrcaRouter-Sangreal-23G-ICE.gguf, in about 45 minutes, running locally in the Nous Hermes desktop harness on Windows 11 with an RTX 3070 (8 GB VRAM), 32 GB RAM and no MTP.

Play: Hugging Face Space · GitHub Pages · Source

Measurements

One binary (llama.cpp 3653e6d), one reference (Nex-N2.5-mini BF16), one session (2026-10-03, RTX PRO 6000 96 GB). 64 chunks at n_ctx 2048; the long-context table uses 8 chunks at n_ctx 16384. Every file below was measured in this session. Raw logs: measurements/2026-10-03/.

Table 1 — Code (`code.test.raw`)

PPL(base) 2.322497 · BF16 = 100

file	size	mean KLD	99% KLD	99.9% KLD	PPL ratio	same top-1	active bpw	file bpw	overall
≈ 24–25 GB
Sangreal 25G-ICE	24.53 GB	0.013991	0.1950	1.0164	1.0032	97.03 %	7.385	5.662	98.1
bartowski Q5_K_S	25.14 GB	0.014041	0.2010	1.0056	1.0026	97.09 %	7.709	5.802	98.2
bartowski Q4_K_L	24.11 GB	0.017092	0.2651	1.2858	1.0031	96.73 %	7.622	5.566	97.8
≈ 22.5 GB
Sangreal 23G-ICE	22.52 GB	0.016628	0.2464	1.1798	1.0032	96.89 %	7.215	5.197	97.9
bartowski Q4_K_M	22.32 GB	0.018899	0.2867	1.4471	1.0034	96.70 %	7.470	5.151	97.7
≈ 18–19 GB
Sangreal 19G-ICE	18.47 GB	0.030944	0.4559	2.0955	1.0128	95.54 %	6.872	4.263	96.6
bartowski Q3_K_L	18.27 GB	0.036789	0.5981	2.4447	1.0117	95.34 %	6.763	4.216	96.1
bartowski IQ3_M	18.96 GB	0.037171	0.5992	2.3107	1.0180	95.25 %	7.068	4.376	96.1
≈ 14–15.5 GB
bartowski IQ3_XXS	15.58 GB	0.056271	0.9337	3.2597	1.0224	94.04 %	5.936	3.596	94.5
Sangreal 15G-ICE	14.51 GB	0.065540	1.0710	3.4991	1.0287	93.54 %	6.531	3.350	93.8
bartowski Q2_K	13.84 GB	0.094286	1.4952	4.3221	1.0429	92.22 %	5.299	3.194	91.6

Table 2 — Code at long context (`code.test.raw`, n_ctx 16384 × 8)

Compare rows within this table only.

PPL(base) 2.385381 · BF16 = 100

file	size	mean KLD	99% KLD	99.9% KLD	PPL ratio	same top-1	active bpw	file bpw	overall
≈ 24–25 GB
bartowski Q5_K_S	25.14 GB	0.070885	1.2557	11.8217	1.0073	96.33 %	7.709	5.802	94.3
Sangreal 25G-ICE	24.53 GB	0.073257	1.3994	11.6990	1.0035	96.19 %	7.385	5.662	94.1
bartowski Q4_K_L	24.11 GB	0.075456	1.3853	11.6240	1.0043	96.07 %	7.622	5.566	93.9
≈ 22.5 GB
Sangreal 23G-ICE	22.52 GB	0.075886	1.3280	11.9342	0.9980	96.07 %	7.215	5.197	93.9
bartowski Q4_K_M	22.32 GB	0.082457	1.3812	13.2635	0.9985	95.96 %	7.470	5.151	93.5
≈ 18–19 GB
Sangreal 19G-ICE	18.47 GB	0.088622	1.5877	12.1322	1.0153	94.98 %	6.872	4.263	92.8
bartowski Q3_K_L	18.27 GB	0.100775	1.9995	12.6043	1.0141	94.80 %	6.763	4.216	92.0
bartowski IQ3_M	18.96 GB	0.103120	2.2440	12.3204	1.0265	94.63 %	7.068	4.376	91.8
≈ 14–15.5 GB
bartowski IQ3_XXS	15.58 GB	0.123681	2.7842	13.2045	0.9976	93.85 %	5.936	3.596	90.5
Sangreal 15G-ICE	14.51 GB	0.137858	3.1925	13.5721	1.0407	93.11 %	6.531	3.350	89.5
bartowski Q2_K	13.84 GB	0.168550	3.8429	14.6754	0.9824	92.24 %	5.299	3.194	87.6

Table 3 — English text (WikiText-2)

PPL(base) 6.509631 · BF16 = 100

file	size	mean KLD	99% KLD	99.9% KLD	PPL ratio	same top-1	active bpw	file bpw	overall
≈ 24–25 GB
bartowski Q5_K_S	25.14 GB	0.023095	0.2071	1.1366	0.9995	93.95 %	7.709	5.802	96.6
Sangreal 25G-ICE	24.53 GB	0.024245	0.2151	1.0980	0.9995	93.79 %	7.385	5.662	96.5
bartowski Q4_K_L	24.11 GB	0.027083	0.2575	1.2347	1.0020	93.45 %	7.622	5.566	96.2
≈ 22.5 GB
Sangreal 23G-ICE	22.52 GB	0.028427	0.2583	1.1999	1.0007	93.09 %	7.215	5.197	96.0
bartowski Q4_K_M	22.32 GB	0.030890	0.2832	1.5537	1.0023	93.03 %	7.470	5.151	95.8
≈ 18–19 GB
Sangreal 19G-ICE	18.47 GB	0.047143	0.4644	1.8029	1.0125	90.94 %	6.872	4.263	94.1
bartowski Q3_K_L	18.27 GB	0.052879	0.5233	1.9749	1.0094	90.71 %	6.763	4.216	93.7
bartowski IQ3_M	18.96 GB	0.053638	0.5158	2.2097	1.0165	90.66 %	7.068	4.376	93.6
≈ 14–15.5 GB
bartowski IQ3_XXS	15.58 GB	0.079871	0.8234	2.6094	1.0286	88.64 %	5.936	3.596	91.4
Sangreal 15G-ICE	14.51 GB	0.097880	0.9787	3.2222	1.0469	87.18 %	6.531	3.350	89.9
bartowski Q2_K	13.84 GB	0.138367	1.3816	4.1705	1.0363	84.70 %	5.299	3.194	86.9

overall = 0.70/(1 + meanKLD) + 0.30 * sameTop1, ×100. BF16 = 100. The same composite as the Ornith-1.5 ICE card: 70% on how close the whole output distribution stays, 30% on agreement about the argmax.

Read the tail columns as shape, not order. 99.9% KLD is roughly the 33rd-worst token of 32,768 — an extreme order statistic with large sampling variance, so it inverts between adjacent files without meaning anything. Rank on mean KLD.

Two bpw columns. Only 8 of 256 experts fire per token, so a bit in ffn_*_exps is worth ~3% of a bit in attention, the shared expert or the output head. active bpw weights by that (token embedding excluded: it is a lookup, not a matmul); file bpw is just size ÷ parameters.

Don't compare the tables to each other. Code is more predictable text, so every file scores about half the divergence on it. Compare rows within a table.

Every row is measured against Nex-N2.5-mini's own BF16 — bartowski's unmodified BF16 conversion of nex-agi's weights (its tensor set, types and metadata match our own conversion plan tensor for tensor). Quantizations of other models are deliberately absent: scoring them here would measure the distance between models and call it quantization damage.

What Sangreal is

The calibration corpus these tiers were quantized against. Built from primary sources, not assembled from eaddario's set, bartowski's calibration_datav5, or any other ready-made calibration file. 77 buckets, 9,533 documents, 47,443,549 tokens, sha256 85a6b823a6762bf6….

The full render is 92,663 chunks. These tiers use a 10,000-chunk pass over it, measured on Nex-N2.5-mini itself - 17x the 573-chunk imatrix behind the stock quants (bartowski's calibration-v6).

block	buckets	share	documents	synthetic
code + cyber + spec	36	45.10%	5,858	0.1%
reasoning	5	13.16%	553	100.0%
agentic	7	11.95%	1,018	65.2%
science + domain	11	11.24%	1,183	0.3%
language	11	9.66%	596	0.0%
mathematics	7	8.89%	325	0.0%
total	77	100%	9,533	21.7%

Split: technical 70.2 · science+domain 20.1 · language 9.7.

Synthetic is 21.7% by bytes and sits in exactly three buckets — teacher:k2-horizon, teacher:deepseek-v4-pro, teacher:hermes. Everything outside reasoning and agentic is real text.

Language — 11 languages, 5 non-Latin scripts

bucket	share	docs	bucket	share	docs
`arabic`	0.92%	48	`russian`	0.87%	58
`chinese`	0.89%	31	`spanish`	0.87%	72
`hindi`	0.88%	44	`portuguese`	0.87%	70
`japanese`	0.88%	33	`turkish`	0.87%	56
`french`	0.88%	60	`german`	0.86%	73
`english`	0.87%	51

The spread across the whole block is 0.06 points, and that is deliberate: language's job here is activation, not ranking. Each bucket sits just above the saturation floor at 44.9% consumption, so these buckets select for the first time — the legacy v1 spec had consumed 96% of turkish and 92% of english.

Mathematics — absent from the pool until this cycle

bucket	share	docs
`math` (general)	3.16%	170
`analysis`	1.24%	28
`algebra`	1.16%	46
`geometry`	1.12%	30
`proof`	1.08%	28
`topology`	1.05%	20
`number-theory`	0.06%	3

Sourced from the LaTeX algebraic geometry, mathlib4, UniMath, math-comp, open-web-math, AutoMathText and proof-pile.

The imatrix and the recipes are not published with this release.

What's inside

ICE = Isolation of Compounding Error: allocate bits by how far a quantization error travels, not by how large the activations are. Routers stay F32, SSM decay gates stay F32, the KV-cached projections stay F16, the always-on dense path stays Q8_0, and the budget goes to the routed expert bank — most of the parameters, but only 8-of-256 active per token. ICE 1.6 keeps token_embd at Q8_0 (its error propagates with context) and moves the output head to Q6_K (its error stays flat from 2K to 16K). Method and the cases where it does not win: gbuzhf/ICE-quantization.

No MTP head. nex-agi's config declares one MTP layer, but the checkpoint carries no MTP weights. These tiers are the 40-block trunk; there is no built-in draft model.

Chat template: Qwen-Sharp v22.5.0-arrow4.0 (ours; spec below). Nex-N2.5-mini uses Qwen3.6's tokenizer unchanged (vocabulary, merges and special tokens; seven extra audio/TTS tokens), so the template's delimiters are native tokens — checked on every tier at signing.

Vision. Nex retrained its vision tower: every vision tensor we sampled differs from Qwen3.6's, projector included. This repo therefore ships Nex's own projector, converted from nex-agi's weights to Q8_0 (334 tensors). Load it with --mmproj mmproj-Nex-N2.5-mini-Q8_0.gguf.

Chat template — Qwen-Sharp v22.5.0-arrow4.0

The GGUFs still embed arrow.3.1; pass the file shipped here to use arrow4.0:

llama-server -m <model>.gguf --jinja --chat-template-file chat_template_sharp-v22.5.0-arrow4.0-nex.jinja

arrow4.0 is our agent layer on Qwen-Sharp v22.5.0. Plain chat stays byte-identical to Sharp; with tools it adds short agent rules, one note per tool result, a STATUS line, thinking only at decision points, .arrow project memory and a hand-back report when work stalls. The -nex file opens replies the Nex way, so the harness's reasoning level is respected (default medium = adaptive).

Full description and switches: gbuzhf/Qwen-Sharp-Arrow-Chat-Templates.

Files

file	size	sha256
`Nex-N2.5-mini-Sangreal-15G-ICE.gguf`	14.51 GB	`512572c63edd57c56742566a9b257ebb7a48d3a4cf734f7eb18c2d784c01327d`
`Nex-N2.5-mini-Sangreal-19G-ICE.gguf`	18.47 GB	`10d0c3609708dcca0cfbd150a4786eb80e06404285f0a83d421fc0bff93f86cc`
`Nex-N2.5-mini-Sangreal-23G-ICE.gguf`	22.52 GB	`aba72b748e97180db005e74379428c9cc401023424073648e9a58992286c077f`
`Nex-N2.5-mini-Sangreal-25G-ICE.gguf`	24.53 GB	`7e44f35eb33632212badc1d319be861b14a4bab3d8731ab7e03420df507922f9`
`mmproj-Nex-N2.5-mini-Q8_0.gguf`	0.61 GB	`ec9fffcc1a6d…`

measurements/2026-10-03/ — raw llama-perplexity output for every row of every table (4 ICE tiers + 7 bartowski files × 3 columns), the three BF16 base runs, and the session summary.

Credits


base model	nex-agi/Nex-N2.5-mini
BF16 master + comparison ladder	bartowski/nex-agi_Nex-N2.5-mini-GGUF
chat template	peculiar-ragdoll/Qwen-Sharp-Chat-Templates
ICE method	gbuzhf/ICE-quantization

KLD measures fidelity to this repo's reference and nothing else — not reasoning, tool use or speed. It is also within-model: every file here is measured against Nex-N2.5-mini's own BF16, so these values are not comparable to another repo's. Compare across models with PPL.

Downloads last month: 3,954

GGUF

Model size

35B params

Architecture

qwen35moe

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for gbuzhf/Nex-N2.5-mini-Sangreal-ICE

Base model

nex-agi/Nex-N2.5-mini

Quantized

(42)

this model

Space using gbuzhf/Nex-N2.5-mini-Sangreal-ICE 1

Collection including gbuzhf/Nex-N2.5-mini-Sangreal-ICE

ICE 1.6

Collection

2 items • Updated 7 days ago

Nex-N2.5-mini — Sangreal ICE

At a glance — against bartowski at the same size

Demo — Coral Dragon Reef Rescue

Measurements

Table 1 — Code (code.test.raw)

Table 2 — Code at long context (code.test.raw, n_ctx 16384 × 8)

Table 3 — English text (WikiText-2)

What Sangreal is

Language — 11 languages, 5 non-Latin scripts

Mathematics — absent from the pool until this cycle

What's inside

Chat template — Qwen-Sharp v22.5.0-arrow4.0

Files

Credits

Model tree for gbuzhf/Nex-N2.5-mini-Sangreal-ICE

Space using gbuzhf/Nex-N2.5-mini-Sangreal-ICE 1

Collection including gbuzhf/Nex-N2.5-mini-Sangreal-ICE

Table 1 — Code (`code.test.raw`)

Table 2 — Code at long context (`code.test.raw`, n_ctx 16384 × 8)