Instructions to use wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16

Use Docker

docker model run hf.co/wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16

LM Studio
Jan

vLLM

How to use wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16

Ollama
How to use wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with Ollama:
```
ollama run hf.co/wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16
```
Unsloth Desktop

How to use wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with Docker Model Runner:
```
docker model run hf.co/wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16
```

Lemonade

How to use wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16

Run and chat with the model

lemonade run user.Ornith-1.0-9B-ASHQ1-MTP-GGUF-BF16

List all available models

lemonade list

Hermes Agent

How to use wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Ornith-1.0-9B — ASHQ1 Quantization (MTP-Aware)

ASHQ1 quantization of Ornith-1.0-9B with explicit MTP (Multi-Token Prediction) handling — MTP heads are kept at Q8_0 and excluded from the classifier budget, preserving their quality while the priority queue optimises the main transformer layers.

Huge thanks to protoLabsAI/Ornith-1.0-9B-MTP-GGUF for providing the BF16 source with MTP heads — we adapted it with ASHQ1 to deliver both speed and quality.

Note: File names contain "BF16" for HuggingFace parser compatibility — these are ASHQ1 quants, not BF16.

Quants

File	Size	Method	Description
`Ornith-1.0-9B-MTP-BF16-ASHQ1-4850.gguf`	4.7 GB	ASHQ1 (v6, `--allow-q3-or-lower`)	Tiny — uses IQ tiers to fit 4.7 GB, MTP heads at Q8_0
`Ornith-1.0-9B-MTP-BF16-ASHQ1-5500.gguf`	5.4 GB	ASHQ1 (v6)	Compact — beats i1-Q6_K (7.0 GB), 23% smaller
`Ornith-1.0-9B-MTP-BF16-ASHQ1-6500.gguf`	6.4 GB	ASHQ1 (v6)	Balanced — near i1-Q8_0 quality at 33% smaller
`Ornith-1.0-9B-MTP-BF16-ASHQ1-7300.gguf`	7.2 GB	ASHQ1 (v6)	Best — surpasses i1-Q8_0 quality, 25% smaller

All comparisons are against i1 quants (uniform quantization with bartowski's Ornith imatrix). ASHQ1 consistently beats them at smaller sizes: the priority queue allocates bits where they matter most. See wepiqx/ASHQ1 for full benchmarks across architectures. MTP heads are deployed at Q8_0 and excluded from the classifier budget.

Note: The 4850 variant uses IQ tiers via --allow-q3-or-lower to fit the tight budget — quality is roughly on par with Q4_K_S, while 5500+ delivers a significant step up. It's best suited for lighter tasks, while coding and agentic workloads may prefer 5500+. Feel free to try it either way.

How ASHQ1 Works

See wepiqx/ASHQ1 for the full method, source code, and benchmarks.

Usage

llama.cpp

# Chat mode with built-in Jinja template (recommended)
llama-cli \
  -m Ornith-1.0-9B-MTP-BF16-ASHQ1-6500.gguf \
  --jinja \
  -ngl 99 \
  -c 8192

Recommended sampling: temperature 0.6–1.0, top_p 0.95, top_k 20.

Ollama

The model's built-in Jinja template is auto-detected by Ollama — no need to specify TEMPLATE manually.

Create a Modelfile:

FROM ./Ornith-1.0-9B-MTP-BF16-ASHQ1-6500.gguf

PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95

Build and run:

ollama create ornith-ashq1-mtp-6500 -f Modelfile
ollama run ornith-ashq1-mtp-6500

LM Studio

Open LM Studio
Drag the desired GGUF file (4850, 5500, 6500, or 7300) into the app
Set GPU Offload to 99 layers
Set context length to 8192+
Set preset sampling: temperature 0.6, top_p 0.95, top_k 20
Start chatting

Quantization Configs

Generated by ASHQ1 priority queue (v6) — files in this repo were produced with these exact llama-quantize arguments.

4850 Config (`--allow-q3-or-lower`)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=IQ4_XS"
--tensor-type "(blk|BLK)\.(30)\.ffn_down=IQ3_S"
--tensor-type "(blk|BLK)\.(32)\.attn_v=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_k=Q4_K"
--tensor-type "(blk|BLK)\.((?:26|27|28|29))\.ffn_down=IQ3_XXS"
--tensor-type "(blk|BLK)\.((?:22|23|24|25))\.ffn_down=IQ2_S"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21))\.ffn_down=IQ2_XXS"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q5_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q5_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.attn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.ssm_beta=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.ssm_alpha=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.attn_qkv=Q4_K"
--tensor-type "(blk|BLK)\.(28|30)\.ssm_out=IQ4_XS"
--tensor-type "(blk|BLK)\.((?:24|25)|29)\.ssm_out=IQ3_S"
--tensor-type "(blk|BLK)\.((?:21|22)|26)\.ssm_out=IQ3_XXS"
--tensor-type "(blk|BLK)\.(18|20)\.ssm_out=IQ2_S"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=IQ2_XXS"
--tensor-type ".*output_norm.*=F16"

5500 Config (base=Q5_K_M)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(3)\.attn_q=Q6_K"
--tensor-type "(blk|BLK)\.(3)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.(3|32)\.attn_v=Q6_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(31)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.(30)\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.(1)\.attn_gate=Q4_K"
--tensor-type "(blk|BLK)\.(1)\.ssm_alpha=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.(1)\.ssm_beta=Q4_K"
--tensor-type "(blk|BLK)\.(1)\.attn_qkv=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30)|32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:26|27|28|29|30|31))\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.attn_qkv=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.ssm_alpha=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23|27|(?:31|32))\.attn_k=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23|27|31)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.attn_gate=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23|27|31)\.attn_q=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.ssm_beta=Q5_K"
--tensor-type "(blk|BLK)\.((?:26|27|28|29|30|31))\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25)|32)\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25)|32)\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|32)\.attn_output=Q4_K"
--tensor-type ".*output_norm.*=F16"

6500 Config (base=Q5_K_M)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|15|19|23|27|31)\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|15|19|23|27|31)\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|15|19|23|27|31)\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.(11)\.attn_q=Q6_K"
--tensor-type "(blk|BLK)\.(11)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.(11)\.attn_v=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.((?:14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.((?:28|29|30))\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.((?:14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(27|31)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.(32)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(32)\.attn_k=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.((?:26|27|28|29|30))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25)|32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:24|25|26))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13))\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13))\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.(19|23)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.([0-3]|32)\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.([0-3]|32)\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|32)\.attn_output=Q4_K"
--tensor-type ".*output_norm.*=F16"

7300 Config (base=Q5_K_M)

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q8_0"
--tensor-type "(blk|BLK)\.(27|31)\.attn_output=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(28|30)\.ssm_out=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.((?:16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.((?:16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28|29|30))\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13|14|15))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13|14|15))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:24|25|26)|29)\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.(19|23)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.(32)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(32)\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.((?:18|19|20|21))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17)|32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.((?:9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|8)\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.(3|32)\.attn_output=Q4_K"
--tensor-type ".*output_norm.*=F16"

Hardware Requirements

Variant	Size (GGUF)	VRAM (ctx 8192)	Minimum GPU
4850	4.7 GB	~5.8 GB	8 GB
5500	5.4 GB	~6.5 GB	8 GB
6500	6.4 GB	~7.5 GB	8 GB
7300	7.2 GB	~8.3 GB	12 GB

Notes

Source BF16: protoLabsAI/Ornith-1.0-9B-MTP-GGUF — huge thanks to protoLabsAI for the MTP-ready BF16
Imatrix: bartowski's Ornith imatrix (248 entries, 6 datasets)
Architecture: Qwen3.5-9B (32 layers, GQA, SSM, MTP, 262k context)
MTP heads deployed at Q8_0
License: MIT
Quantization and tuning by wepiqx
Built with llama.cpp

Downloads last month: 218

GGUF

Model size

9B params

Architecture

qwen35

Hardware compatibility

16-bit

Model tree for wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF

Base model

ornith-ai/Ornith-1.0-9B

Quantized

(101)

this model

Collection including wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF

Ornith 1.5 & 1.0 ASHQ1

Collection

Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement. • 3 items • Updated Sep 6 • 2

Ornith-1.0-9B — ASHQ1 Quantization (MTP-Aware)

Quants

How ASHQ1 Works

Usage

llama.cpp

Ollama

LM Studio

Quantization Configs

4850 Config (--allow-q3-or-lower)

5500 Config (base=Q5_K_M)

6500 Config (base=Q5_K_M)

7300 Config (base=Q5_K_M)

Hardware Requirements

Notes

Model tree for wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF

Collection including wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF

4850 Config (`--allow-q3-or-lower`)