jashansinghTT/olmo-3.1-32b-instruct-blackhole

olmo-3.1-32b-instruct on Tenstorrent p150 and p300 and p300x2

Derived from allenai/Olmo-3.1-32B-Instruct by AI2, unmodified. See the original model card for training and evaluation details.

Olmo 3.1 32B Instruct, a fully open 32B dense decoder with hybrid sliding-window / full attention and 65k context, served with vLLM.

Runs on Tenstorrent p150 or p300 or p300x2 — see the serving profiles below.

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

At a glance

Hardware	p150, p300, p300x2
Context	65,536 tokens
License	apache-2.0

Intended use

General chat and tool-calling assistance through an OpenAI-compatible API.

Prerequisites

• The Tenstorrent CLI, tt: uv tool install tenstorrent (or use tt-model alone, see the Quickstart)
• Docker
• Tenstorrent hardware: p150 or p300 or p300x2

Quickstart

tt serve jashansinghTT/olmo-3.1-32b-instruct-blackhole

tt serve downloads the Docker image and the allenai/Olmo-3.1-32B-Instruct weights at ac0587e4a7744a551c059d8cd17ba220bc940dae (into your HF cache; they are not in the image), then starts an OpenAI-compatible server on port 20000 (or the next free port, if that one is busy). The first start compiles kernels for your device, which takes several minutes; the server is ready when it logs Application startup complete.

Without tt-cli: tt-model serve jashansinghTT/olmo-3.1-32b-instruct-blackhole.

Serving profiles

Select a different serving profile using --profile.

profile	hardware	mesh	max_num_seqs	max_model_len	description
p150 (default)	p150	P150	32	16384	Single Blackhole chip (TP1), bfp4 MLP; 16k context. Widest compatibility. [validated context 16384, precision performance]
p300	p300	P300	32	65536	One p300 board, both chips (TP2); 64k context. Needs a P300 host (2 chips); on a QuietBox-2 use p300x2. [validated context 65536, precision accuracy]
p300x2	p300x2	P300x2	32	65536	Two p300 boards / QuietBox 2, 4 chips (TP4); 64k context, 32 users. [validated context 65536, precision accuracy]

Capabilities

The server speaks the OpenAI API at http://127.0.0.1:20000/v1 (or whichever port your serve command reported; chat completions, completions, and /v1/models). Pass "model": "allenai/Olmo-3.1-32B-Instruct" — the weights id, not this package's name.

Tool calling is enabled (--tool-call-parser olmo3_tt): a request that passes tools comes back with tool_calls and finish_reason: tool_calls.

Expected performance

eval	this package (p150)	reference	prompts
IFEval (0-shot, prompt-level strict)	89.4	88.8	151

profile	ISL	OSL	concurrency	N	TTFT (ms)	prefill tok/s/u	decode tok/s/u	E2EL (s)
p150	128	128	1	4	177	723	10.8	11.89
p150	128	128	32	128	2,465	52	10.6	14.49
p150	1,024	256	1	4	472	2,168	7.4	34.91
p150	4,096	256	1	4	1,873	2,187	3.7	70.02
p300	128	128	1	4	123	1,042	14.6	8.80
p300	128	128	32	128	1,598	80	14.5	10.35
p300	4,096	256	1	4	1,208	3,391	7.6	34.57
p300	32,768	256	1	1	15,952	2,054	4.1	77.95
p300x2	128	128	1	4	80	1,601	22.9	5.63
p300x2	128	128	32	128	993	129	22.7	6.58
p300x2	4,096	256	1	4	711	5,763	13.4	19.74
p300x2	32,768	256	1	1	8,537	3,838	7.6	41.95

IFEval with lm-evaluation-harness 0.4.10 through the chat endpoint (one sample per prompt, seed 42); reference is the upstream model card. Speed from the tt-model latency sweep on a QuietBox 2, September 2026; N is the number of requests.

Limitations

• On p150, about 1k-token prompts at 32 concurrent users, or 4k-token prompts at 8 or more, stop the engine returning tokens. Keep long-prompt concurrency low on that profile, or use p300/p300x2.
• Prompts above 16k tokens were measured at one user only; at 16k tokens and 32 users on p300, a request takes about 317 s end to end.
• Only IFEval was measured for accuracy; math, reasoning and code benchmarks were not.
• Text only; no image input.

Feedback

Questions or problems with this package: open a discussion at https://huggingface.co/jashansinghTT/olmo-3.1-32b-instruct-blackhole/discussions — that is what reaches its author. A problem with the tt tooling itself: tt report issue (collects your environment and opens a prefilled issue against tenstorrent/tt-cli). Product feedback: [email protected].

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	262a2382c7d638f2cb66eb231ae15fa8ad69fbc4 (dirty tree — the image includes uncommitted changes)
vLLM	v0.26.0
vllm-tt-plugin	a local checkout — commit not published
code/ digest	8053b141cc976bb5 (sha256, first 16 hex digits)
built	2026-09-12T03:10:32+00:00 by tt-model 0.1.0