Arete-27B-NVFP4-GGUF

Arete-27B is Nous Research's supervised fine-tune of Qwen3.8-27B, built to change the model's behavior on politically sensitive topics. Nous says Qwen3.8-27B gave refusals, deflections and CCP-framed answers there, and describes the goal as reinforcing "western values", which the card frames as giving users a choice of values. Training used about 20,000 generated prompts across seven categories (freedom of expression, state ideology, state versus free market, education, data privacy and surveillance, democracy, environmental policy). The teacher was Inkling, an open model chosen so as to avoid "illicit distillation" terms-of-use issues. It was trained for two epochs (1,202 steps, about 52.8M tokens at 64k context) on all language-model weights, with the vision encoder frozen and no RL or preference optimization. On Nous's own 803-prompt Propaganda_Eval, scored by a Grok 4.7 judge, the fact score rises from 2.32 to 2.98 out of 5 and answers containing propaganda fall from 22% to 0.9%. That is still below the Inkling teacher (3.96 and 0.3%), and the card says the gains held in all seven categories. The biggest change is in freedom of expression, where the fact score goes from 0.89 to 2.44 and propaganda from 60% to 1%. General capability is reported as unchanged: the internal Hermes Bench is 65.77% against 65.60%, and Terminal-Bench-2.1 is 67% against 70%, though Nous takes no credit for that gain. The evaluation is the authors' own, and the settings are not identical across the models compared. The training dataset is announced in the blog post, and the linked dataset is only the held-out evaluation. Weights are Apache 2.0.

MTP — Speculative Decoding: No

Model Files

File Name	Quant Type	File Size	File Link	Description
Arete-27B-NVFP4.gguf	NVFP4	20.2 GB	Link	NVFP4 quantization. Highly optimized for NVIDIA Blackwell hardware accelerators (e.g., RTX Pro 6000, RTX 50-series, B200, B300), offering native hardware-accelerated FP4 Tensor Core speeds and a strong balance of size and performance for this 27B parameter model.
Arete-27B.mmproj.gguf	mmproj	931 MB	Link	Multimodal projection file. Used for vision/language models.

NVFP4 Test Run

Tested Arete-27B-NVFP4.gguf on an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with 96 GB of VRAM, using llama-cli with full GPU offloading.

Performance Metrics

Metric	Value
Average Prompt Processing Speed	319.51 tokens/s
Average Generation Speed	65.22 tokens/s
Total Wall Time	76.2 seconds
Context Length	16,384 tokens
KV Cache (K)	q8_0
GPU	NVIDIA RTX PRO 6000 Blackwell Server Edition
GPU VRAM Capacity	96 GB
Peak GPU VRAM Usage	18.28 GiB
Peak GPU Utilization	97%
GPU Layers Offloaded	99 (-ngl 99)
Random Seed	42
Speculative Decoding	No
Test Runs	5

Test Shots

Shot	Test
Shot 1	C: Insert Node at End of Singly Linked List
Shot 2	Jogger and Train Passing Time
Shot 3	Two Pipes Filling a Tank
Shot 4	Students in Examination Rooms
Shot 5	Father and Son Ages

Run Inference

./build/bin/llama-cli \
  -m /path/to/models/Arete-27B-NVFP4.gguf \
  -ngl 99 \
  -c 16384 \
  --cache-type-k q8_0 \
  --seed 42 \
  -n 8192 \
  -st \
  -p "Your prompt here"

Performance results reflect this specific hardware and inference configuration. Actual throughput may vary depending on the prompt, model, software build, and runtime settings.

Snapshots (5)

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp

Quantized using llama.cpp by prithivMLmods.