Instructions to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0
# Run inference directly in the terminal:
llama cli -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0
# Run inference directly in the terminal:
llama cli -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0
# Run inference directly in the terminal:
./build/bin/llama cli -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Use Docker

docker model run hf.co/gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Llama new
LM Studio
Jan

vLLM

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "gbuzhf/Nex-N2.5-mini-Sangreal-ICE"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gbuzhf/Nex-N2.5-mini-Sangreal-ICE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Ollama
How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with Ollama:
```
ollama run hf.co/gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0
```
Unsloth Desktop

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with Docker Model Runner:
```
docker model run hf.co/gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0
```

Lemonade

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Run and chat with the model

lemonade run user.Nex-N2.5-mini-Sangreal-ICE-Q8_0

List all available models

lemonade list

Hermes Agent

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use gbuzhf/Nex-N2.5-mini-Sangreal-ICE with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "gbuzhf/Nex-N2.5-mini-Sangreal-ICE:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Nex-N2.5-mini-Sangreal-ICE

File size: 7,183 Bytes

592307d

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 860 540" width="860" height="540" font-family="Inter,Segoe UI,Helvetica,Arial,sans-serif">
<rect width="860" height="540" fill="#fcfcfb"/>
<text x="80" y="30" font-size="19" font-weight="600" fill="#0b0b0b">Nex-N2.5-mini: KL divergence vs size (code, 16K context)</text>
<text x="80" y="52" font-size="13" fill="#52514e">Mean KLD vs BF16 on the same tokens, llama-perplexity, n_ctx 16384, 8 chunks. Lower-left is better.</text>
<line x1="80" x2="830" y1="470.0" y2="470.0" stroke="#e8e7e3" stroke-width="1"/>
<text x="72" y="474.0" font-size="12" fill="#898781" text-anchor="end">0.060</text>
<line x1="80" x2="830" y1="407.7" y2="407.7" stroke="#e8e7e3" stroke-width="1"/>
<text x="72" y="411.7" font-size="12" fill="#898781" text-anchor="end">0.080</text>
<line x1="80" x2="830" y1="345.3" y2="345.3" stroke="#e8e7e3" stroke-width="1"/>
<text x="72" y="349.3" font-size="12" fill="#898781" text-anchor="end">0.100</text>
<line x1="80" x2="830" y1="283.0" y2="283.0" stroke="#e8e7e3" stroke-width="1"/>
<text x="72" y="287.0" font-size="12" fill="#898781" text-anchor="end">0.120</text>
<line x1="80" x2="830" y1="220.7" y2="220.7" stroke="#e8e7e3" stroke-width="1"/>
<text x="72" y="224.7" font-size="12" fill="#898781" text-anchor="end">0.140</text>
<line x1="80" x2="830" y1="158.3" y2="158.3" stroke="#e8e7e3" stroke-width="1"/>
<text x="72" y="162.3" font-size="12" fill="#898781" text-anchor="end">0.160</text>
<line x1="80" x2="830" y1="96.0" y2="96.0" stroke="#e8e7e3" stroke-width="1"/>
<text x="72" y="100.0" font-size="12" fill="#898781" text-anchor="end">0.180</text>
<line x1="133.6" x2="133.6" y1="470" y2="475" stroke="#898781"/>
<text x="133.6" y="490" font-size="12" fill="#898781" text-anchor="middle">14</text>
<line x1="240.7" x2="240.7" y1="470" y2="475" stroke="#898781"/>
<text x="240.7" y="490" font-size="12" fill="#898781" text-anchor="middle">16</text>
<line x1="347.9" x2="347.9" y1="470" y2="475" stroke="#898781"/>
<text x="347.9" y="490" font-size="12" fill="#898781" text-anchor="middle">18</text>
<line x1="455.0" x2="455.0" y1="470" y2="475" stroke="#898781"/>
<text x="455.0" y="490" font-size="12" fill="#898781" text-anchor="middle">20</text>
<line x1="562.1" x2="562.1" y1="470" y2="475" stroke="#898781"/>
<text x="562.1" y="490" font-size="12" fill="#898781" text-anchor="middle">22</text>
<line x1="669.3" x2="669.3" y1="470" y2="475" stroke="#898781"/>
<text x="669.3" y="490" font-size="12" fill="#898781" text-anchor="middle">24</text>
<line x1="776.4" x2="776.4" y1="470" y2="475" stroke="#898781"/>
<text x="776.4" y="490" font-size="12" fill="#898781" text-anchor="middle">26</text>
<line x1="80" x2="830" y1="470" y2="470" stroke="#898781"/>
<text x="455.0" y="518" font-size="13" fill="#52514e" text-anchor="middle">File size (GB, lower is better)</text>
<text transform="translate(22,283.0) rotate(-90)" font-size="13" fill="#52514e" text-anchor="middle">Mean KL divergence (lower is better)</text>
<path d="M160.9,227.3 L373.0,380.8 L590.0,420.5 L697.7,428.7" fill="none" stroke="#2a78d6" stroke-width="2"/>
<line x1="160.9" x2="160.9" y1="238.2" y2="216.4" stroke="#2a78d6" stroke-width="1.5"/>
<circle cx="160.9" cy="227.3" r="5.5" fill="#2a78d6" stroke="#fcfcfb" stroke-width="2"/>
<text x="151.9" y="247.3" font-size="12" fill="#0b0b0b" text-anchor="end">ICE-15G <tspan fill="#52514e">0.138</tspan></text>
<line x1="373.0" x2="373.0" y1="389.7" y2="371.8" stroke="#2a78d6" stroke-width="1.5"/>
<circle cx="373.0" cy="380.8" r="5.5" fill="#2a78d6" stroke="#fcfcfb" stroke-width="2"/>
<text x="364.0" y="400.8" font-size="12" fill="#0b0b0b" text-anchor="end">ICE-19G <tspan fill="#52514e">0.089</tspan></text>
<line x1="590.0" x2="590.0" y1="429.3" y2="411.7" stroke="#2a78d6" stroke-width="1.5"/>
<circle cx="590.0" cy="420.5" r="5.5" fill="#2a78d6" stroke="#fcfcfb" stroke-width="2"/>
<text x="581.0" y="440.5" font-size="12" fill="#0b0b0b" text-anchor="end">ICE-23G <tspan fill="#52514e">0.076</tspan></text>
<line x1="697.7" x2="697.7" y1="437.3" y2="420.0" stroke="#2a78d6" stroke-width="1.5"/>
<circle cx="697.7" cy="428.7" r="5.5" fill="#2a78d6" stroke="#fcfcfb" stroke-width="2"/>
<text x="697.7" y="454.7" font-size="12" fill="#0b0b0b" text-anchor="middle">ICE-25G <tspan fill="#52514e">0.073</tspan></text>
<path d="M125.0,131.7 L218.2,271.5 L362.3,342.9 L399.3,335.6 L579.3,400.0 L675.2,421.8 L730.4,436.1" fill="none" stroke="#eb6834" stroke-width="2"/>
<line x1="125.0" x2="125.0" y1="143.9" y2="119.4" stroke="#eb6834" stroke-width="1.5"/>
<rect x="119.5" y="126.2" width="11" height="11" rx="1.5" fill="#eb6834" stroke="#fcfcfb" stroke-width="2"/>
<text x="134.0" y="123.7" font-size="12" fill="#0b0b0b" text-anchor="start">Q2_K <tspan fill="#52514e">0.169</tspan></text>
<line x1="218.2" x2="218.2" y1="282.0" y2="261.0" stroke="#eb6834" stroke-width="1.5"/>
<rect x="212.7" y="266.0" width="11" height="11" rx="1.5" fill="#eb6834" stroke="#fcfcfb" stroke-width="2"/>
<text x="227.2" y="263.5" font-size="12" fill="#0b0b0b" text-anchor="start">IQ3_XXS <tspan fill="#52514e">0.124</tspan></text>
<line x1="362.3" x2="362.3" y1="352.5" y2="333.3" stroke="#eb6834" stroke-width="1.5"/>
<rect x="356.8" y="337.4" width="11" height="11" rx="1.5" fill="#eb6834" stroke="#fcfcfb" stroke-width="2"/>
<text x="371.3" y="360.9" font-size="12" fill="#0b0b0b" text-anchor="start">Q3_K_L <tspan fill="#52514e">0.101</tspan></text>
<line x1="399.3" x2="399.3" y1="345.1" y2="326.1" stroke="#eb6834" stroke-width="1.5"/>
<rect x="393.8" y="330.1" width="11" height="11" rx="1.5" fill="#eb6834" stroke="#fcfcfb" stroke-width="2"/>
<text x="408.3" y="325.6" font-size="12" fill="#0b0b0b" text-anchor="start">IQ3_M <tspan fill="#52514e">0.103</tspan></text>
<line x1="579.3" x2="579.3" y1="409.4" y2="390.6" stroke="#eb6834" stroke-width="1.5"/>
<rect x="573.8" y="394.5" width="11" height="11" rx="1.5" fill="#eb6834" stroke="#fcfcfb" stroke-width="2"/>
<text x="588.3" y="390.0" font-size="12" fill="#0b0b0b" text-anchor="start">Q4_K_M <tspan fill="#52514e">0.082</tspan></text>
<line x1="675.2" x2="675.2" y1="430.5" y2="413.2" stroke="#eb6834" stroke-width="1.5"/>
<rect x="669.7" y="416.3" width="11" height="11" rx="1.5" fill="#eb6834" stroke="#fcfcfb" stroke-width="2"/>
<text x="675.2" y="407.8" font-size="12" fill="#0b0b0b" text-anchor="middle">Q4_K_L <tspan fill="#52514e">0.075</tspan></text>
<line x1="730.4" x2="730.4" y1="444.4" y2="427.7" stroke="#eb6834" stroke-width="1.5"/>
<rect x="724.9" y="430.6" width="11" height="11" rx="1.5" fill="#eb6834" stroke="#fcfcfb" stroke-width="2"/>
<text x="739.4" y="426.1" font-size="12" fill="#0b0b0b" text-anchor="start">Q5_K_S <tspan fill="#52514e">0.071</tspan></text>
<circle cx="86" cy="78" r="5.5" fill="#2a78d6"/>
<text x="97" y="82" font-size="13" fill="#0b0b0b">Sangreal ICE 1.6</text>
<rect x="290.5" y="72.5" width="11" height="11" rx="1.5" fill="#eb6834"/>
<text x="307" y="82" font-size="13" fill="#0b0b0b">bartowski imatrix GGUF</text>
<text x="830" y="82" font-size="12" fill="#898781" text-anchor="end">bars: ±1 s.e.</text>
</svg>