Instructions to use mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF with Transformers:

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M

Llama new
LM Studio
Jan
Ollama
How to use mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF with Ollama:
```
ollama run hf.co/mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M
```
Unsloth Desktop

How to use mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF with Docker Model Runner:
```
docker model run hf.co/mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M
```

Lemonade

How to use mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Atomight-V2.2-0.5B-UltraThink-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

mradermacher commited on May 29

Commit

660467f

verified ·

1 Parent(s): 3085c46

uploaded from leia

Browse files

Files changed (13) hide show

.gitattributes +12 -0
Atomight-V2.2-0.5B-UltraThink.IQ4_XS.gguf +3 -0
Atomight-V2.2-0.5B-UltraThink.Q2_K.gguf +3 -0
Atomight-V2.2-0.5B-UltraThink.Q3_K_L.gguf +3 -0
Atomight-V2.2-0.5B-UltraThink.Q3_K_M.gguf +3 -0
Atomight-V2.2-0.5B-UltraThink.Q3_K_S.gguf +3 -0
Atomight-V2.2-0.5B-UltraThink.Q4_K_M.gguf +3 -0
Atomight-V2.2-0.5B-UltraThink.Q4_K_S.gguf +3 -0
Atomight-V2.2-0.5B-UltraThink.Q5_K_M.gguf +3 -0
Atomight-V2.2-0.5B-UltraThink.Q5_K_S.gguf +3 -0
Atomight-V2.2-0.5B-UltraThink.Q6_K.gguf +3 -0
Atomight-V2.2-0.5B-UltraThink.Q8_0.gguf +3 -0
Atomight-V2.2-0.5B-UltraThink.f16.gguf +3 -0

@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+Atomight-V2.2-0.5B-UltraThink.f16.gguf filter=lfs diff=lfs merge=lfs -text

+version https://git-lfs.github.com/spec/v1
+oid sha256:b3f4b5de7cfcd614d5bed1d7eaf4f00a72e862aae414fafaf34f17f43470406f
+size 351445824

+version https://git-lfs.github.com/spec/v1
+oid sha256:0f42f6ad48fc514b5f6dd0e50c067e8010f9f5154a74f852b664a722be357338
+size 338607936

+version https://git-lfs.github.com/spec/v1
+oid sha256:0f20036ee8ab55572e37593c0cfc950b23fe8db93717e5a7886c0dea4a3551db
+size 369358656

+version https://git-lfs.github.com/spec/v1
+oid sha256:7703bf0a1194e0e810eb9f308dd2697708a1e0aba10a40fd7576c999211674cb
+size 355467072

+version https://git-lfs.github.com/spec/v1
+oid sha256:ed26a6e219e59ff66656625ed04b34a10099ba9aba5a5701a673d1612df91669
+size 338263872

+version https://git-lfs.github.com/spec/v1
+oid sha256:4da81b0e502499f8eb826a77ec920e281d5f200c9be4e278ee35e31b286f402e
+size 397808448

+version https://git-lfs.github.com/spec/v1
+oid sha256:4c46c2ab8c56b4348ac1c90a89aad2c5b34dfeb5a2f3c1e066c2e29e248cc2f0
+size 385472320

+version https://git-lfs.github.com/spec/v1
+oid sha256:9ad91266fae255e5469d59130bede63d3f944da5a3f0af74cf4d0114417e5a5a
+size 420086592

+version https://git-lfs.github.com/spec/v1
+oid sha256:141294c460060ab3b24a27974fe6b53c8d1cbd33c9470d8328f69e747ebd3f67
+size 412710720

+version https://git-lfs.github.com/spec/v1
+oid sha256:4d41df7730efdb2332931137af34a0c1b116d316e8a3eff3bef2ab26aa295360
+size 505737024

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6b800e44e26afa49e482b819bf89065b1f8964c3e04fa2388bc1a267eee33357
+size 531068736

+version https://git-lfs.github.com/spec/v1
+oid sha256:3947f88bccaba741a92f6bafd2db9944bd8caf8d3ffd1828d506993678806125
+size 994157376