jlens-amalia

Pre-fitted Jacobian Lenses for the Amalia model family, fitted on the amalia-llm/CorEGe-PT dataset.

Trained using Anthropic's Jacobian Lens library via the Neuronpedia fitting utilities. The same library can be used to load a lens and run the local visualizer. Jacobian Lens can also be explored on Neuronpedia.

Contents

Lens	Base model	Architecture	Prompts	Path
AMALIA-9B-0626-DPO / CorEGe-PT	amalia-llm/AMALIA-9B-0626-DPO	LlamaForCausalLM, 42 layers, d_model=4096	783	amalia-9b-0626-dpo/corege-pt/
AMALIA-VL-DPO / CorEGe-PT	amalia-llm/AMALIA-VL-DPO	LlavaNextForConditionalGeneration, 42 layers, d_model=4096	740	amalia-vl-dpo/corege-pt/

Each subfolder contains:

• <MODEL>_jacobian_lens.pt — fitted lens weights
• <MODEL>_convergence.csv — full convergence curve (Δmean vs. prompt count)

Shared fitting configuration

Both lenses were fit with the same recipe:

• Dataset: amalia-llm/CorEGe-PT, split=train, field=text
• Filter: pt.auto=true,pt.pt.auto=true
• Text cleaning: dropped rows matching [˙ˆ˜œ], max_chars=2000
• Target prompts: 1000 (both converged and stopped early)
• Source layers: 0–40 (41 layers), dtype=bfloat16
• Levels: 1e-2, 5e-3, 1e-3
• Stopping criterion: stop_at_delta=0.002 over a 10-prompt running-mean window (min_prompts=100)
• Batch/seq: dim_batch=32, max_seq_len=128

Per-lens convergence

AMALIA-9B-0626-DPO

Δmean threshold	First reached at
< 0.01	123 prompts
< 0.005	268 prompts
< 0.001	not reached

Stopped early at 783 prompts, Δmean = 2.11e-03. Peak CUDA memory: 34.13 GB.

AMALIA-VL-DPO

Δmean threshold	First reached at
< 0.01	106 prompts
< 0.005	211 prompts
< 0.001	not reached

Stopped early at 740 prompts, Δmean = 1.63e-03. Peak CUDA memory: 34.94 GB.

Usage

from jacobian_lens import HFLensModel, JacobianLens

lens = JacobianLens.from_pretrained(
    "Helder5788/jlens-amalia",
    subfolder="amalia-9b-0626-dpo/corege-pt",  # or "amalia-vl-dpo/corege-pt"
)

model = HFLensModel.from_pretrained("amalia-llm/AMALIA-9B-0626-DPO")
model.attach_lens(lens)

See the Jacobian Lens library for full API details and the local visualizer.

License

MIT