VRAM requirements table / optional README badge
#2
by donaldJJ - opened
Estimated VRAM for one request (weights + FP16 KV cache + ~10% runtime overhead, GiB):
| 8K ctx | 32K ctx | |
|---|---|---|
| Q4_K_M | 1.28 | 1.59 |
| Q8_0 | 1.77 | 2.08 |
| BF16 | 2.76 | 3.07 |
BF16 weights are 1.93 GiB (2.07 GB of safetensors, 1.0B params). 36 of 48 layers use a 1,025-token sliding window, so even BF16 at 32K fits in ~3 GB, well below the H100 it was validated on.
If useful, an optional README badge (renders like this):
Markdown to paste:
[](https://modelvram.com/llm-vram-calculator/limite-1b-violetto/)
I maintain ModelVRAM, a free calculator; feel free to ignore/close.