--- language: - en tags: - gguf - nvfp4 - dense - vision - muse-glimmer - 30b - quantized inference: false pipeline_tag: text-generation base_model: meta-models/Muse-Glimmer-30B --- # Muse-Glimmer-30B NVFP4 GGUF ## Base Model **[Muse-Glimmer-30B](https://huggingface.co/meta-models/Muse-Glimmer-30B)** by Meta — a 27.8B-parameter dense causal transformer with integrated vision encoder (ViT-G/14, ~1.8B params). Apache 2.0 licensed. Supports images, screenshots, charts, documents, and 131K+ context. No MTP heads — uses DFlash speculative decoding instead. ## Vision Support This model includes a full vision encoder (mmproj) embedded in the GGUF. Use `llama-mtmd-cli` or `llama-server --mmproj` for vision capabilities. The vision encoder uses dynamic-size input with patch size 14 and temporal patch 2. ## Quantization - **Format:** NVIDIA FP4 (NVFP4) — 4-bit E2M1 values with E4M3 scaling per 16 values + FP32 tensor scale - **BPW:** 4.69 bits per weight - **Size:** 15.6 GB - **Target hardware:** NVIDIA Blackwell GPUs (RTX 50 series) — requires CUDA kernels at inference ## Tested Settings (llama.cpp build 1647+, RTX 5060 Ti 16GB) ``` llama-cli -m muse-glimmer-30b-nvfp4.gguf \ -ngl 99 \ --ctx-size 131072 \ --flash-attn on \ --cache-type-k q8_0 --cache-type-v q8_0 \ --reasoning off \ -t 8 --temp 0.7 --top-k 20 --top-p 0.95 ``` ### Performance Results | Metric | Result | |--------|--------| | Prompt processing | 80.2 t/s | | Token generation | 8.7 t/s | | Context | 128K | | KV Cache | Q8_0 | | Test prompt | "Explain to me in detail what is superpositioning and Quantum computing" | | Output quality | Coherent, detailed, accurate — covered Dirac notation, Bloch sphere, interference, quantum parallelism | ## Usage ### llama.cpp CLI (text generation) ```bash llama-cli -m muse-glimmer-30b-nvfp4.gguf -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --reasoning off -p "Your prompt here" ``` ### llama.cpp Server (API) ```bash llama-server -m muse-glimmer-30b-nvfp4.gguf -ngl 99 -c 131072 --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --reasoning off --host 0.0.0.0 --port 8080 ``` ### Vision (requires llama-mtmd-cli) ```bash llama-mtmd-cli -m muse-glimmer-30b-nvfp4.gguf -ngl 99 --flash-attn on --reasoning off --image /path/to/image.png -p "Describe this image" ``` ## Requirements - llama.cpp build 1647+ with CUDA support (Muse-Glimmer architecture support merged Aug 2026) - NVIDIA GPU with 16GB+ VRAM (tested on RTX 5060 Ti 16GB) - Full GPU offload works — model fits entirely in VRAM - `--reasoning off` recommended to suppress thinking token leakage ## Notes - Dense model (all 27.8B params active per token) — no `--cpu-moe` needed - Sliding window attention (2048 for 75% of layers, full for 25%) keeps KV cache small at 128K - For faster generation (~3x), see the [MXFP4 variant](https://huggingface.co/FreedomAISVR/Muse-Glimmer-30B-MXFP4-GGUF) - Reasoning cannot be fully disabled via config — use `--reasoning off` at inference ## License Apache 2.0 (inherited from base model)