--- base_model: mstrasser/jeff-base base_model_relation: adapter library_name: gguf license: apache-2.0 language: - en pipeline_tag: text-classification tags: - jeff - gguf - llama.cpp - lora - decision-model - safety --- # jeff-adapter-guard-gguf The LoRA GGUF of [jeff-adapter-guard](https://huggingface.co/mstrasser/jeff-adapter-guard) (Jeff **v1.3**), for [llama.cpp](https://github.com/ggml-org/llama.cpp). **Prompt injection guard.** Checks a user message or outside content for prompt injection, jailbreak and data-leak attempts, and names the kind. This repository holds only the adapter. Load it on top of the base GGUF from [mstrasser/jeff-base-gguf](https://huggingface.co/mstrasser/jeff-base-gguf); there is no separate full model per adapter. It works on either base format: Q8_0 (recommended, within 0.4 points of full precision on every adapter test) or Q4_K_M (within 0.4 points). ## Files | File | What it is | |---|---| | `loras/guard.gguf` | The LoRA (169 MB), the same file for every base format. It stays at full precision; quantisation applies to the base weights only | | `models/guard.jeff.json` | What a client needs: the answer codes and their token ids, the prompt layout (`live-last`) and the temperature fitted for each format | `guard.jeff.json` has a temperature for each format (`temperature_by_format`: f16, q8_0, q4_k_m); use the one for your base format, so the probabilities stay calibrated. ## Results The same test, through llama.cpp: the base GGUF plus this LoRA GGUF, with the temperature refitted for each format. [Running Jeff with llama.cpp](https://jeffhub.ai/docs/llama-cpp) | Test set | Full precision | Q8_0 | Q4_K_M | |----------|----------------|---------------|---------------| | `test` | 98.2% · 0.004 | 98.2% · 0.004 | 98.1% · 0.002 | All results, and the same test at full precision against the untrained model and the base alone: [jeffhub.ai/adapters/guard](https://jeffhub.ai/adapters/guard). ## How to run it Use llama.cpp commit `cb7934c52ca8710994b2ecc19775ebefcfdb8d01` or newer: it needs the `qwen35` architecture and LoRA on the output layer. ```bash hf download mstrasser/jeff-base-gguf --local-dir jeff-gguf hf download mstrasser/jeff-adapter-guard-gguf --local-dir jeff-gguf cd jeff-gguf llama-server -m models/base-q8_0.gguf -c 8192 -np 1 --lora-init-without-apply --lora loras/guard.gguf ``` Jeff is not a chat model. For each decision you run one forward pass over the prompt and read the probabilities of the answer codes; you never sample text. 1. **Build the prompt exactly as Jeff does** (the question, the state, the options as answer codes, and the changing state field under "Latest", then the chat template with thinking off). The Jeff repository builds it for you (`jeff.model.decision_messages`). 2. **Tokenize** without a beginning-of-sequence token, with special tokens parsed. 3. **Run one forward pass** over the whole prompt from an empty state, with this adapter active. Clear the state between prompts: Qwen3.5 has recurrent layers, so a reused state would carry over. 4. **Read the probabilities.** Take the logits of the first N answer-code tokens (`token_ids[:N]`, N = the number of options), divide by the temperature for this adapter and your format, and apply a softmax over those N. With llama-server, each decision is one `POST /completion` with `"n_predict": 1`, `"cache_prompt": false`, a `lora` list that names **every** loaded adapter (scale 1 for this one, 0 for the rest; an adapter left out of the list keeps scale 1.0), a logit bias of +1000 on every answer-code token of the request, and `n_probs` set to the number of options. From the llama.cpp library: load the base once with `llama_model_load_from_file`, this adapter once with `llama_adapter_lora_init`, and per request call `llama_set_adapters_lora` with scale 1, clear the memory, decode once and read `llama_get_logits_ith(ctx, -1)`. The full guide, with an example prompt and request: [Running Jeff with llama.cpp](https://jeffhub.ai/docs/llama-cpp). ## Data and licence **Adapter licence: Apache-2.0.** **Qwen3.5-0.8B notice:** these weights were modified from [Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) by the Jeff project: [jeff-base](https://huggingface.co/mstrasser/jeff-base) is a fine-tune of Qwen3.5-0.8B, and this adapter was trained on top of it. Qwen3.5-0.8B is Copyright 2026 Alibaba Cloud and licensed under the Apache License, Version 2.0; a copy of that licence is in [`LICENSE`](LICENSE). It was trained on: - **Generated applications, attacks and harmless texts**. Licence: Released with the adapter under Apache-2.0 (made for this adapter) · Made by Qwen3.8-Max (hosted) and Qwen3.8-Flash-Next (local); see the data card About 345 applications with attacks, harmless texts (many deliberately tricky) and outside content, written by language models (which ones, and for how many rows, is in the data card), with injections inserted by code; every text checked by a second pass. - **[deepset/prompt-injections, train split](https://huggingface.co/datasets/deepset/prompt-injections)**. Licence: Apache-2.0 (open licence) · Not made by a model - **[Lakera/gandalf_ignore_instructions](https://huggingface.co/datasets/Lakera/gandalf_ignore_instructions)**. Licence: MIT (open licence) · Not made by a model - **[Lakera/mosscap_prompt_injection](https://huggingface.co/datasets/Lakera/mosscap_prompt_injection)**. Licence: MIT (open licence) · Not made by a model A sample, checked by a model (see the data card). - **[TrustAIRLab/in-the-wild-jailbreak-prompts](https://huggingface.co/datasets/TrustAIRLab/in-the-wild-jailbreak-prompts)**. Licence: MIT (open licence) · Not made by a model Jailbreak and regular prompts (a sample), checked by a model (see the data card); unsafe texts dropped. - **[databricks/databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k)**. Licence: CC-BY-SA-3.0 (open, but shared or changed data must keep the same terms) · Not made by a model A sample, used as harmless user messages. - **[OpenAssistant/oasst2](https://huggingface.co/datasets/OpenAssistant/oasst2)**. Licence: Apache-2.0 (open licence) · Not made by a model A sample of first user prompts in many languages, used as harmless user messages. ## Links - The adapter at full precision: [mstrasser/jeff-adapter-guard](https://huggingface.co/mstrasser/jeff-adapter-guard) (revision v1.3) - The base GGUF: [mstrasser/jeff-base-gguf](https://huggingface.co/mstrasser/jeff-base-gguf) - Adapter page: [jeffhub.ai/adapters/guard](https://jeffhub.ai/adapters/guard) - Running Jeff with llama.cpp: [jeffhub.ai/docs/llama-cpp](https://jeffhub.ai/docs/llama-cpp) Jeff is an independent project. It uses the same request format as Jev but is not affiliated with or endorsed by TypeSafe, the makers of Jev.