muse-glimmer-30b on Tenstorrent p300x2

Serves the text decoder of meta-models/Muse-Glimmer-30B from the meta-models organization at revision f84ecc3a. See the original model card for training and evaluation details.

Muse-Glimmer-30B, a dense transformer of about 29.6B parameters including its vision encoder, served with vLLM as a text-only endpoint for long-context agentic coding with tool calling and separated reasoning output.

Runs on Tenstorrent p300x2 (mesh P300x2) — 131,072-token context, up to 32 concurrent sequences.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Hardware	p300x2
Context	131,072 tokens
License	apache-2.0

Intended use

Agentic coding and tool calling on text with long contexts, one request at a time.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p300x2

Quickstart

tt serve tt-hous/muse-glimmer-30b

tt serve downloads the Docker image and the meta-models/Muse-Glimmer-30B at f84ecc3a0ea984a4c04542a84269e3d065350a6e weights (into your HF cache; they are not in the image), then starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve tt-hous/muse-glimmer-30b.

Capabilities

The server speaks the OpenAI API at http://127.0.0.1:20000/v1 (or whichever port your serve command reported; chat completions, completions, and /v1/models). Pass "model": "meta-models/Muse-Glimmer-30B" — the weights id, not this package's name.

Tool calling is enabled (--tool-call-parser muse_glimmer): a request that passes tools comes back with tool_calls and finish_reason: tool_calls.

Reasoning output is separated (--reasoning_parser muse_glimmer): the thinking text arrives in reasoning_content, apart from content.

The chat template defaults to Reasoning strength: high, so the model reasons before every reply. A short factual question needs about 500 to 700 completion tokens. Put Reasoning strength: low in the system prompt, or send chat_template_kwargs: {"reasoning_strength": "low"}, when a short budget is required. The model card's sampling is temperature 1.0, top_p 0.95, top_k 64.

Prefix caching is on. With eight 4,096-token prefixes each reused by eight requests (vllm bench serve --dataset-name prefix_repetition, one request at a time), median TTFT is 98 ms with caching and 496 ms without, and p99 TTFT is 952 ms with caching and 526 ms without.

Expected performance

eval	this package	reference	prompts	notes
GPQA Diamond, CoT zero-shot, exact match (flexible extract)	77.27	83.5	198	reference is "GPQA Diamond (AA)" on the original card, a different harness; 30 of 198 responses had no extractable answer or ran on without stopping
IFEval, prompt-level strict	88.72	—	541	the original card reports IFBench, a different benchmark

Evals ran through tt-inference-server's eval workflow (its lm-eval command and scoring) against this package at 131,072-token context, with the model card's sampling: temperature 1.0, top_p 0.95, top_k 64.

profile	ISL	OSL	concurrency	N	TTFT (ms)	prefill tok/s/u	decode tok/s/u	E2EL (s)
default	128	512	1	1	70	1,842	42.4	12.10
default	1,024	512	1	1	145	7,082	40.0	12.90
default	4,096	512	1	1	454	9,012	37.5	14.10
default	8,192	512	1	1	913	8,975	35.9	15.10
default	16,384	512	1	1	2,080	7,877	33.0	17.50
default	32,768	512	1	1	4,480	7,314	28.3	22.50
default	65,536	512	1	1	10,170	6,444	22.2	33.20
default	130,560	512	1	1	25,120	5,197	15.4	58.20

One request at a time on two P300 boards (four Blackhole chips), 512 output tokens, input length swept to the full context. Values are single measurements, or the median of three runs where a point was re-measured. Prefill tok/s/u is ISL divided by TTFT. The last row fills the 131,072-token context.

Limitations

• Text input only: the checkpoint's vision encoder is not served. Tested on two P300 boards only.
• Not for concurrent long-context serving: admission is limited by the KV cache, so serve one request at a time at long context.
• A max_tokens budget that ends inside the reasoning returns the reasoning as reasoning and an empty string as content.
• A request that sends tools with tool_choice: "none" gets no tool calls, but if the model still writes one, a non-streaming response carries its markup in content.
• Through lm-eval's aime25 task, 18 of 30 responses had no extractable answer and one ran to 294,912 characters, so no AIME score is reported.
• Prefix caching raises p99 TTFT on cold prefixes (952 ms against 526 ms without caching).

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/tt-hous/muse-glimmer-30b/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: [email protected].

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	a local checkout — commit not published
vLLM	v0.24.0
vllm-tt-plugin	a local checkout — commit not published
code/ digest	7931f069dd3c0235 (sha256, first 16 hex digits)
built	2026-10-05T16:40:10+00:00 by tt-model 0.1.0