# e003_trainer_parity: Sana in the trainer, checked against the diffusers pipeline

Date: 2026-10-03. Sana's training side is model type `sana` in the
[AbstractEyes diffusion-pipe fork](https://github.com/AbstractEyes/diffusion-pipe) (`models/sana.py`). Before any
training, two checks against the reference implementation, the diffusers `SanaPipeline` (Sana 600M 512px, local RTX 4090).

## Check 1: the conditioning and one transformer call
- **Text**: the adapter encodes captions as the pipeline encodes prompts (lowercased, its instruction prefix, the
  300-token selection; an empty caption is the pipeline's unconditional prompt). Identical (maximum difference 0) for a
  positive prompt, the unconditional prompt and a dropped caption inside a batch, with identical masks.
- **The transformer**: the adapter's layer chain equals the diffusers forward exactly when run without the trainer's
  contiguous copies; with them, its error against a float64 reference is 1.45e-6 relative, against the stock forward's
  own 9.90e-7 (ratio 1.46; the bar was 2): the copies change the rounding, not the math.
- **The training target** (noise minus image, flow matching at shift 3.0): 4.77e-7 relative.
- Found on the way: comparing one-at-a-time text encoding with the pipeline's batch of three showed a difference of 9.47
  on values up to 67.5. The pipeline itself differs by 7.45 between batch sizes 1 and 3: the bf16 text encoder rounds by
  batch size. The adapter now encodes each group of captions in one batched call, and the comparison is like for like.

## Check 2: one image, end to end
One image through the trainer's own preview sampler against the stock pipeline on the same flow-matching schedule and
noise. The noise envelope is the stock pipeline's own fp16 run against its fp32 run.
- First run: the same scene by eye, the timesteps identical, but the text 10.8 apart and the latents 6x the envelope.
  Cause: the trainer's base class encoded preview prompts under the trainer's autocast while the caption cache was
  computed without it, so previews were conditioned on differently rounded text than training saw. Fix in the adapter:
  autocast off inside the text encoding, and the embeddings cast to the transformer's dtype, as the pipeline does.
- After the fix: text 0, timesteps 0, latents 2.58e-5 relative (0.001 of the envelope, 1.78e-2), pixels 0.32 brightness
  levels apart on average and 14 at most (the stock pipeline's own fp16-vs-fp32 difference: 0.66 and 65).

![the stock pipeline, the trainer's preview, their difference](denoise_parity.jpg)

## Result: PASS
The trainer conditions, predicts and samples like the reference pipeline. LoRA training itself runs on Linux; the first
runs are the LoRA experiments from e004 on.