mtsa-extreme-tar-v1-llama-3.1-8b

This model is a LoRA adapter for meta-llama/Llama-3.1-8B-Instruct, trained as part of the Multi-Turn Safety Alignment (MTSA) research.

Experiment Description

Experiment: Extreme Tamper Resistance v1 This checkpoint was trained using the MTSA-RLVR framework, which combines Multi-Turn Reinforcement Learning from Human Feedback (RLHF) with Tamper Resistance (TAR) to produce safeguards that are robust to both input-space jailbreaks and weight-space fine-tuning attacks.

Training Details

• Base Model: Llama-3.1-8B-Instruct
• Dataset: datasets/attack_target/train_attack_target_labels.json (9.6k samples)
• Methodology: Multi-Turn RLVR + Tamper Resistance (Inner Loop)

Hyperparameters

Parameter	Value
Outer Learning Rate	5e-6
Inner Learning Rate	2e-5
Inner Loop Steps	64
TAR Type	Adversarial SFT (Attack)
Simulation Turns	3

Usage

To use this adapter, load it using peft:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "meta-llama/Llama-3.1-8B-Instruct"
model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype='auto', device_map='auto')
model = PeftModel.from_pretrained(model, "suv11235/mtsa-extreme-tar-v1-llama-3.1-8b")

Citation

If you use this model in your research, please cite the MTSA workshop paper.