---
license: other
license_name: minimax-h3-community-license
license_link: LICENSE
base_model: MiniMaxAI/MiniMax-H3
tags:
- level-of-token
- lora
- video-generation
- research
---

# MiniMax-H3 Level-of-Token adapter: run 3: isometric stills, teacher distillation

> [!WARNING]
> # ⚠️ THIS MODEL NEEDS MORE TRAINING — HELP WANTED
> **These are research checkpoints, not a finished model.** LoT output is still visibly worse than dense MiniMax H3: coarse regions are soft, faces in talking-head shots distort, and the LoRA can bend bodies even with LoT off. If you have GPU time, data, or ideas, join the discussion: **[help-wanted discussion](https://github.com/scrya-com/lot-diffusion/discussions/1)**.

> [!CAUTION]
> **License and territory.** Derived from MiniMax H3 under the [MiniMax H3 Community License Agreement](https://huggingface.co/johndpope/MiniMax-H3-LoT/blob/main/LICENSE) (see also [`NOTICE`](https://huggingface.co/johndpope/MiniMax-H3-LoT/blob/main/NOTICE)). That license allows use and redistribution **only outside its Excluded Territories: the European Union, the United Kingdom, the Republic of Korea and the United States of America.** Do not download or use these files there. Commercial use above US$20M yearly revenue needs MiniMax's written authorization. The [Acceptable Use Policy](https://huggingface.co/johndpope/MiniMax-H3-LoT/blob/main/LICENSE) applies, including: no impersonation without consent, and machine-generated output must be disclosed as such.

> [!NOTE]
> **Training data.** Run 4 includes clips of "Nikki", a character whose reference face comes from a Grok Imagine generation (per the dataset's own notes), plus isometric-3D stills. Do not use these weights to impersonate anyone.

**Train it yourself / help improve it:** [cheatsheet](https://github.com/scrya-com/lot-diffusion/blob/main/lot/CHEATSHEET.md) · [train from your own mp4s](https://github.com/scrya-com/lot-diffusion/blob/main/lot/README.md#training-h3-for-lot-from-your-own-mp4s) · [results and renders (issue 1)](https://github.com/johndpope/MiniMax-H3/issues/1)


Research weights for running **MiniMax-H3**'s DiT on a **Level-of-Token (LoT)** layout: fewer, larger tokens where detail is low, with the full-resolution velocity recovered afterwards ([Nakayama et al., arXiv 2610.05816](https://arxiv.org/abs/2610.05816)). The VAE latent stays full size; only the transformer sequence shrinks. On one 24 GB card a 124-frame 768×1344 DiT forward (37 latent frames) measured 52.3 s dense vs 18.3 s LoT (2.85×, `center` layout). The VAE decode is unchanged.

**Status: research.** Coarse regions are still softer than dense H3. Progress, renders and caveats are tracked in [the issue](https://github.com/johndpope/MiniMax-H3/issues/1).

## Files

| File | What |
|---|---|
| `adapter.safetensors` | LoT adapter: per-extent input/output heads, shape MLP, and the Procrustes extent bank (`A`, `s`) it was trained with |
| `lora.safetensors` | Rank-16 LoRA on `blocks.*.attn.qkv_proj`, `attn.out_proj`, `mlp.fc1`, `mlp.fc2` (200 modules) of the FL2VA DiT |
| `train_log.jsonl` | Per-step loss and held-out evals |

## Use

Needs [lot-diffusion](https://github.com/scrya-com/lot-diffusion) (it bundles the [musubi-tuner](https://github.com/kohya-ss/musubi-tuner) H3 runtime) and the pruned int8 FL2VA DiT (`hf download Comfy-Org/MiniMax-H3 diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors`).

```bash
hf download johndpope/MiniMax-H3-LoT --local-dir runs/lot_hf
python3 lot/time_lot_h3.py --shape clip --trained runs/lot_hf/run3_iso3d_distill      # timing per layout
python3 lot/render_h3.py --trained runs/lot_hf/run3_iso3d_distill --swap 4 \
    --layout bands --variants dense,dense_lora,lot_trained                      # needs a cache of encoded prompts
```

ComfyUI (stills and video): [ComfyUI-MiniMax-H3-Image-Lane](https://github.com/scrya-com/ComfyUI-MiniMax-H3-Image-Lane). Quick start and training from your own mp4s: [cheatsheet](https://github.com/scrya-com/lot-diffusion/blob/main/lot/CHEATSHEET.md).

## Training

Warm start chain: run 1 (2,000 steps, shift 12) → run 2 (2,000 steps, LoT shift 3) → **run 3 (2,000 steps, LoT shift 3, `--distill 1`)**. Data: 777 isometric-3D stills with cached text (`cache_iso3d`), 24 held out. Layouts per step: 20% dense, uniform extents, detail-driven 4×4 mosaics. Rank 16, AdamW 8-bit, lr 1e-4, batch 1, swap 4, one RTX PRO 4000 Blackwell (24 GB), 2 h 50 min.

Loss: eq. 17 with H3's `x0 − ε` head (velocity MSE in y-space), plus a distillation term that pulls LoT's clean estimate toward the frozen dense base (LoRA off) from the same noisy input.

## Held-out eval

`dense` / `uniform2` / `mosaic` are the data loss on those layouts. `gap_*` is the weighted distance from LoT to the frozen dense teacher. Lower is better.

| step | dense | uniform2 | mosaic | gap_uniform2 | gap_mosaic |
|---|---|---|---|---|---|
| 0 | 0.2959 | 0.1835 | 0.2496 | 0.6732 | 0.6934 |
| 250 | 0.3076 | 0.1878 | 0.2551 | 0.6376 | 0.6475 |
| 500 | 0.3094 | 0.1894 | 0.2536 | 0.6172 | 0.6119 |
| 750 | 0.3094 | 0.1846 | 0.2516 | 0.6092 | 0.6071 |
| 1000 | 0.3091 | 0.1836 | 0.2513 | 0.5969 | 0.6044 |
| 1250 | 0.3179 | 0.1883 | 0.2567 | 0.5833 | 0.5826 |
| 1500 | 0.3078 | 0.1822 | 0.2512 | 0.5794 | 0.5833 |
| 1750 | 0.3091 | 0.1851 | 0.2537 | 0.5609 | 0.5726 |
| 2000 | 0.3056 | 0.1830 | 0.2523 | 0.5688 | 0.5670 |

## License

Derived from MiniMax H3 weights; use is governed by the [MiniMax H3 Community License Agreement](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE).