---
base_model: mstrasser/jeff-base
base_model_relation: adapter
library_name: gguf
license: apache-2.0
language:
- en
pipeline_tag: text-classification
tags:
- jeff
- gguf
- llama.cpp
- lora
- decision-model
- voice
---

# jeff-adapter-nav-gguf

The LoRA GGUF of [jeff-adapter-nav](https://huggingface.co/mstrasser/jeff-adapter-nav) (Jeff **v1.3**), for
[llama.cpp](https://github.com/ggml-org/llama.cpp).

**Voice navigation.** Turns a spoken command, even a misheard one, into an item on the current app screen, a question, or none of these.

This repository holds only the adapter. Load it on top of the base GGUF from
[mstrasser/jeff-base-gguf](https://huggingface.co/mstrasser/jeff-base-gguf); there is no separate full model per adapter.

It works on either base format: Q8_0 (recommended, within 0.4 points of full precision on every adapter test) or Q4_K_M (within 0.4 points).

## Files

| File | What it is |
|---|---|
| `loras/nav.gguf` | The LoRA (169 MB), the same file for every base format. It stays at full precision; quantisation applies to the base weights only |
| `models/nav.jeff.json` | What a client needs: the answer codes and their token ids, the prompt layout (`live-last`) and the temperature fitted for each format |

`nav.jeff.json` has a temperature for each format (`temperature_by_format`: f16, q8_0, q4_k_m); use the one for
your base format, so the probabilities stay calibrated.

## Results

The same test, through llama.cpp: the base GGUF plus this LoRA GGUF, with the temperature refitted for each format. [Running Jeff with llama.cpp](https://jeffhub.ai/docs/llama-cpp)

| Test set                | Full precision | Q8_0          | Q4_K_M        |
|-------------------------|----------------|---------------|---------------|
| `heldout_synthetic`     | 97.3% · 0.006  | 97.3% · 0.007 | 97.4% · 0.009 |
| `heldout_synthetic-v12` | 97.3% · 0.006  | 97.3% · 0.008 | 97.3% · 0.008 |

All results, and the same test at full precision against the untrained model and the base alone: [jeffhub.ai/adapters/nav](https://jeffhub.ai/adapters/nav).

## How to run it

Use llama.cpp commit `cb7934c52ca8710994b2ecc19775ebefcfdb8d01` or newer: it needs the `qwen35` architecture and LoRA
on the output layer.

```bash
hf download mstrasser/jeff-base-gguf --local-dir jeff-gguf
hf download mstrasser/jeff-adapter-nav-gguf --local-dir jeff-gguf
cd jeff-gguf
llama-server -m models/base-q8_0.gguf -c 8192 -np 1 --lora-init-without-apply --lora loras/nav.gguf
```

Jeff is not a chat model. For each decision you run one forward pass over the prompt and read the probabilities of
the answer codes; you never sample text.

1. **Build the prompt exactly as Jeff does** (the question, the state, the options as answer codes, and the changing
   state field under "Latest", then the chat template with thinking off). The Jeff repository builds it for you
   (`jeff.model.decision_messages`).
2. **Tokenize** without a beginning-of-sequence token, with special tokens parsed.
3. **Run one forward pass** over the whole prompt from an empty state, with this adapter active. Clear the state
   between prompts: Qwen3.5 has recurrent layers, so a reused state would carry over.
4. **Read the probabilities.** Take the logits of the first N answer-code tokens (`token_ids[:N]`, N = the number of
   options), divide by the temperature for this adapter and your format, and apply a softmax over those N.

With llama-server, each decision is one `POST /completion` with `"n_predict": 1`, `"cache_prompt": false`, a `lora`
list that names **every** loaded adapter (scale 1 for this one, 0 for the rest; an adapter left out of the list keeps
scale 1.0), a logit bias of +1000 on every answer-code token of the request, and `n_probs` set to the number of
options. From the llama.cpp library: load the base once with `llama_model_load_from_file`, this adapter once with
`llama_adapter_lora_init`, and per request call `llama_set_adapters_lora` with scale 1, clear the memory, decode once
and read `llama_get_logits_ith(ctx, -1)`.

The full guide, with an example prompt and request: [Running Jeff with llama.cpp](https://jeffhub.ai/docs/llama-cpp).

## Data and licence

**Adapter licence: Apache-2.0.**

**Qwen3.5-0.8B notice:** these weights were modified from [Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) by the Jeff project: [jeff-base](https://huggingface.co/mstrasser/jeff-base) is a fine-tune of Qwen3.5-0.8B, and this adapter was trained on top of it. Qwen3.5-0.8B is Copyright 2026 Alibaba Cloud and licensed under the Apache License, Version 2.0; a copy of that licence is in [`LICENSE`](LICENSE).

It was trained on:

- **Generated apps, screens and spoken commands**. Licence: Released with the adapter under Apache-2.0 (made for this adapter) · Made by Qwen3.8-Max (hosted) and Qwen3.8-Flash-Next (local), per stage; see the data card

  About 340 made-up apps and websites, their screens and items, and how people say each command aloud, written by a language model and code, and checked by a second blind pass (see the data card). No real user data. Sound-alike speech-recognition errors were made in code with the CMU Pronouncing Dictionary (BSD-2-Clause, commit 74790861f652).

## Links

- The adapter at full precision: [mstrasser/jeff-adapter-nav](https://huggingface.co/mstrasser/jeff-adapter-nav) (revision v1.3)
- The base GGUF: [mstrasser/jeff-base-gguf](https://huggingface.co/mstrasser/jeff-base-gguf)
- Adapter page: [jeffhub.ai/adapters/nav](https://jeffhub.ai/adapters/nav)
- Running Jeff with llama.cpp: [jeffhub.ai/docs/llama-cpp](https://jeffhub.ai/docs/llama-cpp)

Jeff is an independent project. It uses the same request format as Jev but is not affiliated with or endorsed by
TypeSafe, the makers of Jev.