---
license: apache-2.0
library_name: peft
pipeline_tag: text-generation
base_model: HuggingFaceTB/SmolLM3-3B
datasets:
  - alirezaaminzadeh/soc-agent-traces-10k
tags:
  - cybersecurity
  - soc
  - agent
  - tool-use
  - lora
  - qlora
  - smollm3
---

# SOC Agent — SmolLM3-3B (investigation agent)

QLoRA fine-tune of [SmolLM3-3B](https://huggingface.co/HuggingFaceTB/SmolLM3-3B)
on [SOC-Agent-Traces-10K](https://huggingface.co/datasets/alirezaaminzadeh/soc-agent-traces-10k),
turning the base model into a SOC investigation agent that reasons step by
step, calls nine read-only investigation tools in `<tool_call>` format, and
closes with a structured JSON triage report.

## Behavior

```
alert → reasoning + <tool_call> → tool result → … → fenced JSON triage report
```

Trained with assistant-only loss on successful traces (`success=true`): tool
results and prompts are masked — only analyst reasoning, tool calls, and the
final report contribute to the gradient.

## Evaluation (held-out test, report prediction given evidence)

| Metric | Value |
|--------|-------|
| JSON parse rate | 0.617 |
| Verdict accuracy | 0.617 |
| Decision accuracy | 0.617 |
| Technique F1 | 1.0 |

Trained steps: 120 · eval samples: 60

## Usage

```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "HuggingFaceTB/SmolLM3-3B", torch_dtype=torch.bfloat16, device_map="cuda")
model = PeftModel.from_pretrained(base, "alirezaaminzadeh/soc-agent-traces-smollm3-3b")
tokenizer = AutoTokenizer.from_pretrained("alirezaaminzadeh/soc-agent-traces-smollm3-3b")

prompt = tokenizer.apply_chat_template(messages, tokenize=False,
                                       add_generation_prompt=True,
                                       enable_thinking=False)
```

Parse `<tool_call>...</tool_call>` blocks from the generation, execute your
(read-only) tools, feed results back as `role="tool"` messages, and iterate
until the model emits the fenced JSON triage report. The companion demo Space
[`alirezaaminzadeh/soc-agent-traces`](https://huggingface.co/spaces/alirezaaminzadeh/soc-agent-traces)
implements the full loop with smolagents.

## Training setup

- Base: SmolLM3-3B, 4-bit NF4 quantization, LoRA r=32 α=64 on all projections
- TRL SFT, assistant-only loss, max length 3072, lr 1.5e-4 cosine
- Trained on ZeroGPU in quota-bounded bursts with Hub checkpoint resume

## License

Apache 2.0