Text Generation
Transformers
Safetensors
English
Chinese
qwen3
qwen
math
grpo
verl
rl
reinforcement-learning
on-policy-distillation
full-parameter-rl
reasoning
conversational
text-generation-inference
Instructions to use Thinking-Space/Qwen3-4B-Base-GRPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Thinking-Space/Qwen3-4B-Base-GRPO with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Thinking-Space/Qwen3-4B-Base-GRPO") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Thinking-Space/Qwen3-4B-Base-GRPO") model = AutoModelForCausalLM.from_pretrained("Thinking-Space/Qwen3-4B-Base-GRPO", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Thinking-Space/Qwen3-4B-Base-GRPO with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Thinking-Space/Qwen3-4B-Base-GRPO" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Thinking-Space/Qwen3-4B-Base-GRPO", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Thinking-Space/Qwen3-4B-Base-GRPO
- SGLang
How to use Thinking-Space/Qwen3-4B-Base-GRPO with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Thinking-Space/Qwen3-4B-Base-GRPO" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Thinking-Space/Qwen3-4B-Base-GRPO", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Thinking-Space/Qwen3-4B-Base-GRPO" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Thinking-Space/Qwen3-4B-Base-GRPO", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Thinking-Space/Qwen3-4B-Base-GRPO with Docker Model Runner:
docker model run hf.co/Thinking-Space/Qwen3-4B-Base-GRPO
|
Download README.md from Thinking-Space/Qwen3-4B-Base-GRPO: direct link, hf CLI and curl.
- Browser
- Download file 4.37 kB
-
https://huggingface.co/Thinking-Space/Qwen3-4B-Base-GRPO/resolve/main/README.md
- Command line
-
hf download hf://Thinking-Space/Qwen3-4B-Base-GRPO/README.md
-
curl -L -o README.md https://huggingface.co/Thinking-Space/Qwen3-4B-Base-GRPO/resolve/main/README.md
4.37 kB
| license: other | |
| language: | |
| - en | |
| - zh | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| tags: | |
| - qwen | |
| - qwen3 | |
| - math | |
| - grpo | |
| - verl | |
| - rl | |
| - reinforcement-learning | |
| - on-policy-distillation | |
| - full-parameter-rl | |
| - reasoning | |
| - safetensors | |
| - arxiv:2604.13016 | |
| base_model: Qwen/Qwen3-4B-Base | |
| base_model_relation: finetune | |
| <h1 align="center">Qwen3-4B-Base-GRPO</h1> | |
| <div align="center" style="line-height: 1;"> | |
| <a href="https://arxiv.org/abs/2604.13016" style="margin: 2px;"> | |
| <img alt="Paper" src="https://img.shields.io/badge/paper-A42C25?style=for-the-badge&logo=arxiv&logoColor=white" style="display: inline-block; vertical-align: middle;"/> | |
| </a> | |
| <a href="https://github.com/thunlp/OPD" style="margin: 2px;"> | |
| <img alt="Github" src="https://img.shields.io/badge/OPD-000000?style=for-the-badge&logo=github&logoColor=white" style="display: inline-block; vertical-align: middle;"/> | |
| </a> | |
| <a href="https://huggingface.co/papers/2604.13016" style="margin: 2px;"> | |
| <img alt="HF Papers" src="https://img.shields.io/badge/HF--Paper-%23FFD14D?style=for-the-badge&logo=huggingface&logoColor=black" style="display: inline-block; vertical-align: middle;"/> | |
| </a> | |
| <a href="https://x.com/HBX_hbx/status/2044464414829777354" style="margin: 2px;"> | |
| <img alt="Twitter" src="https://img.shields.io/badge/Twitter-%23000000.svg?style=for-the-badge&logo=x&logoColor=white" style="display: inline-block; vertical-align: middle;"/> | |
| </a> | |
| </div> | |
| <br> | |
| Qwen3-4B-Base-GRPO is a post-RL checkpoint trained with the **verl** framework. | |
| It starts from **Qwen3-4B-Base** and applies GRPO on the **DAPO-Math-17k-Processed** dataset for mathematical reasoning and problem-solving. | |
| This model is associated with the paper: | |
| **Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe** | |
| Paper link: https://arxiv.org/abs/2604.13016 | |
| ## Model Description | |
| This model is obtained by applying GRPO reinforcement learning to `Qwen3-4B-Base` with verl. The training is intended to improve math-focused reasoning performance under the on-policy distillation setting. | |
| ### Key characteristics | |
| - **Base model**: Qwen3-4B-Base | |
| - **Training framework**: verl | |
| - **Training stage**: Reinforcement Learning (GRPO) | |
| - **Parameter update**: Full-parameter actor update | |
| - **Primary domain**: Mathematical reasoning | |
| - **Reward model**: Not used (`reward_model.enable: false`) | |
| - **Rollout engine**: vLLM | |
| - **Context length**: 32768 tokens | |
| - **Responses per prompt**: 8 | |
| ## Training Details | |
| ### Training configuration | |
| - **Framework**: verl | |
| - **Algorithm**: `grpo` | |
| - **GRPO outcome weight**: `1.0` | |
| - **Learned reward model**: disabled (`reward_model.enable: false`) | |
| - **Reward source**: custom rule-based math reward function | |
| - **Training dataset**: `DAPO-Math-17k-Processed` | |
| - **Training file**: `datasets/DAPO-Math-17k-Processed/DAPO-Math.parquet` | |
| - **Validation datasets**: `AIME25`, `AMC23`, `AIME24` | |
| - **Prompt length**: `1024` | |
| - **Response length**: `7168` | |
| - **Validation response length**: `31744` | |
| - **Max model length**: `32768` | |
| - **Rollout temperature**: `1.0` | |
| - **Repetition penalty**: `1.0` | |
| - **KL loss**: disabled | |
| - **Format reward**: disabled | |
| - **Loss aggregation**: `token-mean` | |
| - **Learning rate**: `1e-6` | |
| - **PPO mini-batch size**: `64` | |
| - **PPO micro-batch size per GPU**: `1` | |
| - **Tensor parallel size**: `1` | |
| - **Number of GPUs**: `8` | |
| - **Number of epochs**: `1` | |
| - **Save frequency**: every `20` steps | |
| - **Test frequency**: every `20` steps | |
| ### Dataset | |
| - **Training dataset**: `DAPO-Math-17k-Processed` | |
| - **Validation datasets**: `AIME25`, `AMC23`, `AIME24` | |
| ## Usage | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| model_id = "lllyx/Qwen3-4B-Base-GRPO" | |
| tokenizer = AutoTokenizer.from_pretrained(model_id) | |
| model = AutoModelForCausalLM.from_pretrained( | |
| model_id, | |
| torch_dtype="auto", | |
| device_map="auto", | |
| ) | |
| ``` | |
| ## Citation | |
| If you use this model, please consider citing the related paper: | |
| ```bibtex | |
| @article{li2026rethinking, | |
| title={Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe}, | |
| author={Li, Yaxuan and Zuo, Yuxin and He, Bingxiang and Zhang, Jinqian and Xiao, Chaojun and Qian, Cheng and Yu, Tianyu and Gao, Huan-ang and Yang, Wenkai and Liu, Zhiyuan and Ding, Ning}, | |
| journal={arXiv preprint arXiv:2604.13016}, | |
| year={2026} | |
| } | |
| ``` | |