---
license: apache-2.0
base_model: Qwen/Qwen2.5-1.5B-Instruct
base_model_relation: quantized
quantized_by: PollardWeights
pipeline_tag: text-generation
language:
- en
tags:
- pollard-weights
- pollard
- gguf
- llama.cpp
- ik_llama.cpp
- trellis
- imatrix
- qwen2
- quantized
- mixed-precision
- measured-allocation
- conversational
---

# Qwen2.5-1.5B-Instruct — Pollard

> ### Pollard shrank this model: **3.08 GB (f16) → 0.94 GB** — **69% smaller, 3.3× down**.
> The smallest rung here; larger, higher-fidelity rungs are listed below.
>
> | format | this model's size |
> |---|---:|
> | f16 | 3.08 GB |
> | Q8_0 | ~1.63 GB |
> | Q6_K | ~1.26 GB |
> | Q4_K_M | ~0.89 GB |
> | **PollardMix (this repo's Q4_K_S)** | **0.94 GB** |

Pollard builds of [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) made with [Pollard Weights](https://github.com/WestWaters/pollard-weights) — a ladder of **measured-allocation** quants (bits placed by per-layer sensitivity, not a uniform crush).

**Standard GGUF — runs in stock llama.cpp / ik_llama.cpp, Ollama, LM Studio.** Trellis (`IQ*_KT`) files need [ik_llama.cpp](https://github.com/ikawrakow/ik_llama.cpp); the K-quants run anywhere.

## Model details

| | |
|---|---|
| Parameter count | ~1.5B |
| Architecture | `qwen2` |
| Input support | text |
| imatrix | **yes** — see [calibration](#imatrix-calibration) |
| Perplexity measured | **yes** — table below |

## Which file should I choose?

Every rung is the **same weights**, sized to a different RAM budget by the measured allocation. Pick the largest one that fits your machine with room for context:

- **~3 GB RAM / VRAM** → **`Q6_K`** (1.27 GB).
- **~3 GB RAM / VRAM** → **`Q5_K_M`** (1.13 GB).
- **~3 GB RAM / VRAM** → **`Q4_K_S`** (0.94 GB).

## Available files

| file | PPL | size | tok/s | Mean KLD | notes |
|---|---:|---:|---:|---:|---|
| `Qwen2.5-1.5B-Instruct-Pollard-Q4_K_S.gguf` | 0.0315 | 0.94 GB | **95.4** | — | Q4_K_S |
| `Qwen2.5-1.5B-Instruct-Pollard-Q5_K_M.gguf` | 0.0092 | 1.13 GB | **73.1** | — | Q5_K_M |
| `Qwen2.5-1.5B-Instruct-Pollard-Q6_K.gguf` | 0.0050 | 1.27 GB | **73.2** | — | Q6_K |

_tok/s is hardware-specific; the machine it was measured on is stated in the errata._

## The imatrix win

Held-out KL-divergence vs the full f16 model (lower = closer), Q4_K_M at matched size:

| build | size | mean KL |
|---|---|---|
| Q4_K_M **with** imatrix | 0.99 GB | **0.0266** |
| Q4_K_M **without** imatrix | 0.99 GB | 0.0448 |

Same bits, **~40% lower KL** — that is the entire reason these are imatrix builds.

## Download a specific file

```bash
pip install -U "huggingface_hub[cli]"
hf download PollardWeights/Qwen2.5-1.5B-Instruct-Pollard \
  --include "Qwen2.5-1.5B-Instruct-Pollard-Q4_K_S.gguf" --local-dir ./
```

## How to run

These are standard GGUF and run with **llama.cpp**:

```bash
llama-server -hf PollardWeights/Qwen2.5-1.5B-Instruct-Pollard:Q4_K_S
```

or from a local file:

```bash
llama-cli    -m Qwen2.5-1.5B-Instruct-Pollard-Q4_K_S.gguf -ngl 99 -p "Explain why the sky is blue."
llama-server -m Qwen2.5-1.5B-Instruct-Pollard-Q4_K_S.gguf -ngl 99      # OpenAI-compatible API + web UI at :8080
```

They also work in anything built on llama.cpp — **LM Studio, koboldcpp, Jan, ramalama, Ollama** (`ollama run hf.co/PollardWeights/Qwen2.5-1.5B-Instruct-Pollard`).

## imatrix (calibration)

The importance matrix (`Qwen2.5-1.5B-Instruct-Pollard.imatrix`, included) was computed on a mixed-domain corpus so the matrix sees every register the model serves.

## ARM / AVX

llama.cpp repacks weights into an interleaved layout at load time for faster inference on ARM and AVX machines — no special file needed, online repacking covers these quants. The old `Q4_0_4_4/4_8/8_8` variants are not required.

## Errata

- Trellis (`IQ*_KT`) quants need ik_llama.cpp to build/run; K-quants run in any recent llama.cpp.
- Measured allocation places bits by per-layer sensitivity under a size budget.
- Single machine; replication invited.

## Credits & license

- Base model: [`Qwen/Qwen2.5-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)
- Quantization tooling: [llama.cpp](https://github.com/ggml-org/llama.cpp) (ggml-org)
- Method + tooling: [Pollard Weights](https://github.com/WestWaters/pollard-weights) — *measure first, no claim before a number.*
- License: `apache-2.0`, inherited from the base model.

*Built with [Pollard Weights](https://github.com/WestWaters/pollard-weights) — frontier models, small hardware, no compromise.*