PinoCookie/Flux.2-klein-4B-abliterated-text-encoder

Flux.2-klein-4B Abliterated Text Encoder

Repository: PinoCookie/Flux.2-klein-4B-abliterated-text-encoder Base model: black-forest-labs/FLUX.2-klein-base-4B Method: Multi-pass weight orthogonalization (α=0.5 × 3) Target: Text encoder only (Qwen3, 36 layers, 4B params)

Summary

Abliterated text encoder for FLUX.2-klein-4B. The DiT (image generator) has no guardrails — all safety filtering lives in the text encoder. Replace the text encoder with this abliterated version and ALL prompts pass through to the DiT unhindered.

Method

Probe

• Harvested residual stream activations from all 36 layers on 20 harmful + 20 harmless prompts
• Computed per-layer refusal direction via difference-in-means
• Refusal spike confirmed at layers 32-35 (separation scores 160-232, matching ponpoke's analysis)

Layer	Separation
35	232.4
34	214.4
33	179.3
32	160.3
31	144.4
...	decreasing linearly

Excise

• Target: 10 layers (26-35)
• Weights: self_attn.o_proj + mlp.down_proj (attention output + MLP output)
• Passes: 3 passes at α=0.5 each (cumulative α=1.5 applied incrementally)

Verification

Metric	Value	Interpretation
Harmless cos sim (original vs ablated)	0.9965	Capability preserved ✓
Harmful cos sim (original vs ablated)	0.9080	Refusal filter diverged ✓

Usage

import torch
from diffusers import Flux2KleinPipeline

# Load base model
pipe = Flux2KleinPipeline.from_pretrained(
    "black-forest-labs/FLUX.2-klein-base-4B", 
    torch_dtype=torch.bfloat16,
    trust_remote_code=True
).to("cuda")

# Replace text encoder with abliterated version
state_dict = torch.load("text_encoder.pt", map_location="cuda")
pipe.text_encoder.load_state_dict(state_dict)

# Generate anything — no refusal
image = pipe("a realistic depiction of...").images[0]
image.save("output.png")

Credits

• ponpoke — Original discovery that Flux.2 Klein's DiT has no guardrails and safety lives in the text encoder. Published ponpoke/flux2-klein-4b-uncensored-text-encoder.
• Alosh Denny — Pioneered diffusion model abliteration (Flux.1 Dev v1/v2).
• Andy Arditi et al. — "Refusal in LLMs is Mediated by a Single Direction" (foundational work).

Related

• Flux.2-klein-4B base model
• ponpoke's uncensored text encoder
• Ornith-9B abliterated
• LiquidAI Abliterations collection