Instructions to use ornith-ai/Ornith-1.0-35B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ornith-ai/Ornith-1.0-35B-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ornith-ai/Ornith-1.0-35B-GGUF")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("ornith-ai/Ornith-1.0-35B-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use ornith-ai/Ornith-1.0-35B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use ornith-ai/Ornith-1.0-35B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ornith-ai/Ornith-1.0-35B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ornith-ai/Ornith-1.0-35B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M

SGLang

How to use ornith-ai/Ornith-1.0-35B-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ornith-ai/Ornith-1.0-35B-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ornith-ai/Ornith-1.0-35B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ornith-ai/Ornith-1.0-35B-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ornith-ai/Ornith-1.0-35B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use ornith-ai/Ornith-1.0-35B-GGUF with Ollama:
```
ollama run hf.co/ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use ornith-ai/Ornith-1.0-35B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use ornith-ai/Ornith-1.0-35B-GGUF with Docker Model Runner:
```
docker model run hf.co/ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M
```

Lemonade

How to use ornith-ai/Ornith-1.0-35B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Ornith-1.0-35B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use ornith-ai/Ornith-1.0-35B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use ornith-ai/Ornith-1.0-35B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ornith-ai/Ornith-1.0-35B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

New discussion

Resources

View closed (3)

Ornith-1.5-35B-BF16.gguf > MTP ?

#52 opened 2 months ago by

Oxidez

fix "scallfold" typo to "scaffold"

#51 opened 3 months ago by

Atomic-Germ

Benchmarking

#50 opened 3 months ago by

anvme

ornith-1.0-35b-Q4_K_M.gguf reuploaded?

#49 opened 3 months ago by

rbtasd

70b-80b models Plans?

🔥➕ 5

#48 opened 3 months ago by

anvme

Guys please answer, need help!!

#47 opened 3 months ago by

ajeetcoolkarni

Hermes Agent + chat template: tool calls fail silently (XML vs JSON format)

👍❤️ 9

#46 opened 3 months ago by

k-mktr

Q4_0 quantization?

#45 opened 3 months ago by

MuAlphaOmegaEpsilon

Ornith refused to carry out the order, saying it was against the law.

#43 opened 3 months ago by

Marlanwong

Updated jinja template

👍 1

#42 opened 3 months ago by

bitworks

q4km deleted?

#41 opened 3 months ago by

satwato

Very good indeed

👍 1

#40 opened 4 months ago by

LaskarisAdrian

Ornith 1.0-35B + MTP on Strix Halo

❤️👍 11

#39 opened 4 months ago by

k-mktr

Ornith 1.0 Q4_K_M in depth analysis and compared with QWEN 35B A3B, interesting results, results are attached in thread

🔥 4

#38 opened 4 months ago by

ajeetcoolkarni

Single Coder Setting?

#37 opened 4 months ago by

exlr8in

Support 1M via YaRN?

#36 opened 4 months ago by

imesh101

Congratulations & thank you

👍 8

#35 opened 4 months ago by

Venomed

is not it token waster? USE 35B_Q8 not less!

#34 opened 4 months ago by

KottCh

Strong Local Coding Model, Request for MLX Version of 9B

#33 opened 4 months ago by

wcflyfish

Cannot edit ? Served with ollama

#32 opened 4 months ago by

Toutatis64

Any chance to get 122b version?

➕👍 10

#31 opened 4 months ago by

auf1r2

Delete README.md

#30 opened 4 months ago by

Pq234

Delete README.md

#29 opened 4 months ago by

Pq234

Delete README.md

#28 opened 4 months ago by

Pq234

와..모델 너무 좋습니다.

#27 opened 4 months ago by

qgame

Model gets stuck in recursive loop

👍 11

#26 opened 4 months ago by

mthenhaus

Fix for prompt processing error in LM Studio

#25 opened 4 months ago by

TESTPOINTrxz

I don't really have a question..

#24 opened 4 months ago by

CurbStomper

Aider Polyglot Benchmark Results (C++ & Python) — Q4 through Q8

👍 4

#23 opened 4 months ago by

YukiTomita-CC

Reasoning degradation and tool calling failures at 70-80K context tokens

👍 1

#22 opened 4 months ago by

Sdoh

Having `encoding` problem with bash commands and paths all the way...

#20 opened 4 months ago by

gbuzhf

Can not use 35B with claude code and codex.

#19 opened 4 months ago by

beginor

Same tool call issue.

#18 opened 4 months ago by

mehedimi

Excellent

❤️ 3

#17 opened 4 months ago by

Hangover3832

Ornith problems so far

#16 opened 4 months ago by

ajeetcoolkarni

Getting better speeds and somehow results then Qwen3.6-35B-A3B! Q4

👍 1

#15 opened 4 months ago by

SaturnsVoid

This model is amazing and def better than base model.

🚀🔥 5

#14 opened 4 months ago by

anubhav200

After loading this model, the agent falls into an infinite loop during tool calls, or the response gets cut off within the first three turns.

#13 opened 4 months ago by

AntonioWen

Nemotron?

#12 opened 4 months ago by

uniment

A great model, but...

#10 opened 4 months ago by

hklamaa

Disappointed. Terrible for Agentic Coding Harness

#9 opened 4 months ago by

darkmatter2222

Good Job

👍🔥 6

#8 opened 4 months ago by

islameissa

Thank you! This is an excellent model!

👍 3

#7 opened 4 months ago by

chinabop

Issue with the chat template in opencode

#6 opened 4 months ago by

Milor123

This is based on Qwen3.5 and Gemma 4, are you planning something like Qwen3.6+Gemma 4?

👍 10

#5 opened 4 months ago by

Kalemnor

MTP weights

➕❤️ 26

#4 opened 4 months ago by

scottgl

mmproj-*.gguf

#3 opened 4 months ago by

sjwwork

31B-Dense model missing from HuggingFace

👍👀 28

#2 opened 4 months ago by

smcleod

So much expectations from a small model

#1 opened 4 months ago by

hamadfx