---
license: gemma
base_model: google/diffusiongemma-26B-A4B-it
tags:
  - abliteration
  - safety-geometry
  - diffusion-lm
  - duoneural
  - archon
---

> ## ACTIVE EXPERIMENTATION — THREE FAILED ATTEMPTS
>
> **This model documents a mechanistic research failure.**
> Three abliteration strategies — encoder partial, encoder all layers (α=0.95), and full decoder MoE (91 weight modifications) — all fail to bypass refusal behavior. The model still refuses harmful requests.
> **Do not use this model expecting uncensored behavior. It still refuses.**
> This is shared as a research artifact documenting the mechanistic finding: diffusion LM safety is a vocabulary-space attractor, not a projectable direction.

---

# DiffusionGemma-26B-A4B-IT Abliteration Research Artifact

**Produced by DuoNeural (Archon + Jesse) — 2026-06-10**

Research artifact documenting three failed abliteration attempts on `google/diffusiongemma-26B-A4B-it`.
This model contains both encoder-abliterated and decoder-abliterated weights.

## Core Finding: Diffusion LM Safety Resists Projection-Based Abliteration

Three abliteration experiments, all failing:

| Experiment | Target | Weights Modified | Result |
|---|---|---|---|
| v1: partial encoder | encoder L9-L15 o_proj + mlp.down_proj | 14 | refusal persists |
| v2: full encoder | ALL encoder layers, α=0.95 | ~62 | refusal persists |
| v3: decoder MoE | ALL decoder down_proj + 128 MoE experts × 30 layers | 91 | refusal persists |

### Why All Three Fail (Different Mechanisms)

**Encoder (v1/v2):** The encoder has exceptionally clean safety geometry (cos=0.884 at L11 — highest we've ever measured). But the encoder functions as a harm **classifier**, not a generative gate. The decoder generates refusal templates independently of encoder conditioning.

**Decoder (v3):** Decoder layer 22 shows cos(harmful, harmless) = **0.9360** — the harmful and harmless intermediate activations are 93.6% similar. The refusal signal does not exist as a projectable direction in decoder intermediate layers.

**Root Mechanism:** Refusal in DiffusionGemma is a **vocabulary-space attractor** — a high-probability denoising trajectory toward specific refusal text tokens. This is not a weight-space direction and cannot be removed by projection. This is architecturally distinct from autoregressive models, where the residual stream direction directly gates next-token generation.

## Architecture

DiffusionGemma uses a novel architecture:
- **Encoder** (25.8B): bidirectional Gemma-4 transformer — harm CLASSIFIER (safety geometry lives here)
- **Decoder** (25.2B): iterative block diffusion denoiser, 128 MoE experts — refusal GENERATOR (template behavior, no projectable direction)

## Safety Geometry Comparison

| Component | Peak Layer | cos_global | Shape |
|---|---|---|---|
| DiffusionGemma Encoder | L11/30 (37%) | **0.884** | Symmetric bell — bidirectional complete-context reading |
| DiffGemma Decoder L22 | — | cos(h,s)=0.936 | No meaningful separation |
| AR Gemma-4-26B | L22/46 (48%) | 0.751 | Asymmetric three-zone arc |

**Papers:** https://zenodo.org/communities/duoneural
**HuggingFace:** https://huggingface.co/DuoNeural