Image-Text-to-Text
PEFT
Safetensors
trl
grpo
async-grpo
lora
agents
harbor
openenv
pi
conversational
Instructions to use harness-hub/Qwen3.8-27B-pi-adapted with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use harness-hub/Qwen3.8-27B-pi-adapted with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-27B") model = PeftModel.from_pretrained(base_model, "harness-hub/Qwen3.8-27B-pi-adapted") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -48,7 +48,7 @@ changed outcome. Tool calls are compared over every task, and on the tasks both
|
|
| 48 |
- **SWE-bench Pro (hard 51):** the 51 hardest tasks of [SWE-bench Pro](https://huggingface.co/datasets/harness-hub/swebench-pro-v2-hard51-harbor), 2 h and 100 agent steps per task.
|
| 49 |
|
| 50 |
Step 160 was picked by looking at the checkpoints' scores on SWE-bench Pro and on an 18-task subset of Terminal-Bench
|
| 51 |
-
2.1, tracked during training.
|
| 52 |
|
| 53 |
## Use with pi
|
| 54 |
|
|
@@ -145,12 +145,4 @@ The adapter must be loaded on top of Qwen3.8-27B. It does not work with other ba
|
|
| 145 |
The training script is
|
| 146 |
[`examples/async_grpo_harbor`](https://github.com/huggingface/trl/tree/main/examples/async_grpo_harbor) in TRL.
|
| 147 |
|
| 148 |
-
## Limitations
|
| 149 |
|
| 150 |
-
- **Small benchmarks.** With 51 to 89 tasks and one attempt per task, the 95% interval on a paired difference is
|
| 151 |
-
±10 points or more. The sign test counts only the tasks that changed outcome.
|
| 152 |
-
- **Thinking off.** The adapter was trained and evaluated without thinking. With thinking on, expect different behaviour.
|
| 153 |
-
- **Timeouts count as failures.** A rollout past the time or step limit is graded 0, for the base and the adapter alike.
|
| 154 |
-
- **One harness.** The adapter was trained inside pi's prompts and tools. With another agent, expect it to
|
| 155 |
-
behave like the base model, or worse; [harness-hub/Qwen3.8-27B-multi-harness](https://huggingface.co/harness-hub/Qwen3.8-27B-multi-harness) is one adapter
|
| 156 |
-
for pi, Hermes and OpenCode.
|
|
|
|
| 48 |
- **SWE-bench Pro (hard 51):** the 51 hardest tasks of [SWE-bench Pro](https://huggingface.co/datasets/harness-hub/swebench-pro-v2-hard51-harbor), 2 h and 100 agent steps per task.
|
| 49 |
|
| 50 |
Step 160 was picked by looking at the checkpoints' scores on SWE-bench Pro and on an 18-task subset of Terminal-Bench
|
| 51 |
+
2.1, tracked during training.
|
| 52 |
|
| 53 |
## Use with pi
|
| 54 |
|
|
|
|
| 145 |
The training script is
|
| 146 |
[`examples/async_grpo_harbor`](https://github.com/huggingface/trl/tree/main/examples/async_grpo_harbor) in TRL.
|
| 147 |
|
|
|
|
| 148 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|