cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end Β· GGUF

GGUF quantizations of logan7000/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end.

Quantized by tinyopsec using llama.cpp.


Model Overview

This is a 3.2B parameter language model fine-tuned from meta-llama/Llama-3.2-3B-Instruct using the GRPO (Group Relative Policy Optimization) method introduced in the DeepSeekMath paper.

It is part of the CogRPO N3 Ring experiment by logan7000, which explores heterogeneous model merging and multi-group reinforcement learning for mathematical reasoning. The groupB-llama32-end variant represents the Llama-3.2-3B branch of the ring fine-tuning pipeline, trained on q1716523669/MATH-Level345.

Architecture: LlamaForCausalLM Context length: 32,768 tokens Training data: MATH Level 3–4–5 competition problems


Quantization Table

File Bits Approx. Size Use Case
model_f16.gguf 16 6.4 GB Full precision, reference
model_q8_0.gguf 8 3.4 GB Best quality, needs more RAM
model_q6_k.gguf 6 2.6 GB Near-lossless, recommended
model_q5_k_m.gguf 5 2.3 GB Balanced quality/size
model_q5_k_s.gguf 5 2.1 GB Slightly smaller Q5
model_q4_k_m.gguf 4 2.0 GB Recommended default
model_q4_k_s.gguf 4 1.9 GB Smaller Q4, minor quality loss
model_q3_k_l.gguf 3 1.7 GB Low RAM, acceptable quality
model_q3_k_m.gguf 3 1.6 GB Lower RAM
model_q3_k_s.gguf 3 1.4 GB Minimum viable quality
model_q2_k.gguf 2 1.2 GB Extreme compression, lowest quality

VRAM / RAM Requirements

Quant Min RAM Recommended
F16 8 GB 10 GB
Q8_0 5 GB 6 GB
Q6_K 4 GB 5 GB
Q5_K_M 4 GB 5 GB
Q4_K_M 3 GB 4 GB
Q3_K_M 3 GB 4 GB
Q2_K 2 GB 3 GB

All quants run on CPU-only. GPU is optional.


Usage

llama.cpp (CLI)

./llama-cli \
  -m model_q4_k_m.gguf \
  -p "Solve: If x^2 + 5x + 6 = 0, find x." \
  -n 512 \
  --temp 0.6 \
  --top-p 0.95 \
  --min-p 0.0

llama.cpp (Server)

./llama-server \
  -m model_q4_k_m.gguf \
  --port 8080 \
  --ctx-size 8192

llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="tinyopsec/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end-GGUF",
    filename="model_q4_k_m.gguf",
)

output = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Solve: x^2 - 7x + 12 = 0"}],
    temperature=0.6,
    top_p=0.95,
    min_p=0.0,
    max_tokens=512,
)
print(output["choices"][0]["message"]["content"])

LM Studio

  1. Open LM Studio β†’ Search
  2. Search: tinyopsec/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end-GGUF
  3. Download model_q4_k_m.gguf
  4. Load and chat

Ollama

ollama run hf.co/tinyopsec/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end-GGUF:Q4_K_M

Recommended Sampling Parameters

This model benefits from low min_p (set to 0.0) to avoid repetition loops, consistent with the base Llama-3.2 family:

Parameter Value
temperature 0.6
top_p 0.95
min_p 0.0
repeat_penalty 1.0

Original Model

Downloads last month
1,749
GGUF
Model size
3B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for tinyopsec/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end-GGUF

Collection including tinyopsec/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end-GGUF

Paper for tinyopsec/cogrpo-n3-ring-qwen25-3b-x-llama32-3b-x-phi4mini-math345-groupB-llama32-end-GGUF