# Multi-chip benchmarks (2x p300c, 4 Blackhole chips, QuietBox 2)

Thermal note: the QB2's fans are fixed. Under sustained load the hottest chip's GDDR reaches 89 °C and the firmware
clamps its AI clock from 1350 to 800 MHz (UMD logs "AICLK clamped by max-arbiter index 7"); with 4 busy chips more of
them clamp within ~2-9 min (the hottest after 2-3 min). A busy chip draws ~125 W, an open idle chip 60-80 W. "Cool" = the first minutes from
< 50 °C; "steady" = after the clamp (≥ 12 min of load, interleaved A / B arms, every run from < 50 °C), except the energy-work and extra-bucket tables,
whose "hot" phase is after 5 min of load (one run each, not interleaved). Every number below says which.

## This revision, cool (`tests/perf/bench_pi05_sp.py`, 2026-10-13, serial + acknowledged K / V feed)

L224, 64 rows, N = 10; 300 replays per figure; p50 / p99 in ms, mean of 2 interleaved runs, every run after a reset and
from <= 50 °C (`notes/hangs/verify/perf`). *SP h2h*: `write_inputs` + replay + readback with the prompt validity
changing every call; *1 chip*: the same programs on one chip of the mesh (`debug_grouped_on_ec`), h2h with a cached
validity. Every SP output equal to the 1-chip launches.

| cameras | SP mesh trace | SP mesh h2h | 1 chip trace | 1 chip h2h |
|---:|---|---|---|---|
| 1 | 35.56 / 35.67 | 36.04 / 36.20 | 40.87 / 41.58 | 41.24 / 42.27 |
| 2 | 37.87 / 38.29 | 38.47 / 39.17 | 52.05 / 52.31 | 52.56 / 52.91 |
| 3 | 42.87 / 43.35 | 43.63 / 44.23 | 71.80 / 72.41 | 72.43 / 73.01 |
| 4 | 49.19 / 49.91 | 50.02 / 50.79 | 89.86 / 90.70 | 90.61 / 91.44 |

One chip opened alone (`--single`, device 1): c2 trace 52.06 / 52.39, h2h 52.34 / 52.70; c4 trace 89.48 / 90.16, h2h
89.99 / 90.68.

The K / V feed fix alone (serial vs the previous concurrent feed, trace p50, 2 runs per arm, cool): one chip -0.23 ms
(c2) / -0.42 ms (c4), SP mesh -0.14 / -0.21 ms; 4 robots in-process cool p50 53.1 vs 53.3 ms, 75.1 vs 74.8 requests/s.
The acknowledged handoff on top of the serial feed: within 0.2 ms (c2 / c4 one chip, c1-c4 mesh). Against the older
table below (before the thermal placement, the energy work and the feed fix), c3 on the mesh is 1.2 ms slower and
everything else 1.1-3.5 ms faster; that difference is not attributed to one change.

## Single-robot mesh vs one chip (cool, `tests/perf/bench_pi05_sp.py`, before the thermal placement, energy work and K / V feed fix)

N = 10, 1000 replays after 50 warm-up, p50 / p99 in ms. *trace*: blocking `execute_trace`; *h2h*: `write_inputs` +
replay + readback (the mesh column with the prompt validity changing every call, the 1-chip column with one cached
validity). Every SP output equal to the 1-chip launches of the same request. Measured before the thermal placement
and the energy work (the placement costs +0.4 ms cool at c2 and nothing at c4; the energy work saves ~1.5 ms at c2:
the mesh's c2 L224 call is 37.5 ms cool at the current kernels, `energy_profile`).

| cameras | prompt | rows | SP mesh trace | 1 chip trace | SP mesh h2h | 1 chip h2h |
|---:|---:|---:|---|---|---|---|
| 1 | 224 | 64 | 36.87 / 36.97 | 42.85 / 43.21 | 37.35 / 37.58 | 43.03 / 43.45 |
| 1 | 224 | 32 | 34.44 / 34.57 | 40.00 / 40.34 | 34.91 / 35.13 | 40.28 / 40.63 |
| 1 | 32 | 64 | 34.83 / 34.93 | 39.87 / 39.98 | 35.28 / 35.50 | 40.30 / 40.91 |
| 1 | 32 | 32 | 32.40 / 32.51 | 37.54 / 37.78 | 32.87 / 33.04 | 37.83 / 38.08 |
| 2 | 224 | 64 | 38.99 / 39.25 | 55.52 / 55.81 | 39.58 / 39.87 | 55.78 / 56.10 |
| 2 | 224 | 32 | 36.43 / 36.67 | 52.33 / 52.60 | 36.98 / 37.31 | 52.58 / 52.94 |
| 2 | 32 | 64 | 37.46 / 37.62 | 52.14 / 52.38 | 38.06 / 38.28 | 52.39 / 52.71 |
| 2 | 32 | 32 | 35.03 / 35.17 | 49.30 / 49.54 | 35.61 / 35.83 | 49.59 / 50.00 |
| 3 | 224 | 64 | 41.72 / 42.25 | 75.12 / 75.62 | 42.40 / 43.11 | 75.37 / 75.84 |
| 3 | 224 | 32 | 36.88 / 37.32 | 70.15 / 70.62 | 37.53 / 38.28 | 70.45 / 71.03 |
| 3 | 32 | 64 | 40.15 / 40.86 | 71.77 / 72.16 | 40.80 / 41.65 | 71.81 / 72.32 |
| 3 | 32 | 32 | 35.66 / 36.36 | 66.82 / 67.39 | 36.21 / 37.07 | 67.32 / 67.93 |
| 4 | 224 | 64 | 51.56 / 51.98 | 93.08 / 93.64 | 52.25 / 52.78 | 93.48 / 93.94 |
| 4 | 224 | 32 | 44.40 / 44.83 | 86.35 / 86.82 | 45.14 / 45.68 | 86.72 / 87.14 |
| 4 | 32 | 64 | 45.45 / 45.99 | 85.65 / 86.08 | 46.21 / 46.86 | 85.87 / 86.38 |
| 4 | 32 | 32 | 43.08 / 43.59 | 82.63 / 83.08 | 43.82 / 44.41 | 82.94 / 83.47 |

Stages (trace, L224 / 64 rows, c1..c4): VISION 7.08-7.13 ms (cameras in parallel); relay +0.02-0.05 ms; SP PREFIX with
the K / V exchange 12.70 / 14.34 / 16.23 / 20.77 ms; EXPERT 17.01 / 17.47 / 18.22 / 23.56 ms.

## Single-robot mesh at thermal steady state (thermal placement A / B, `tests/perf/energy_profile.py --mode sp`)

L224, 64 rows, N = 10, 143 tokens; 2 runs per arm, interleaved, from < 50 °C; steady windows with the hot chip at
800 MHz. Whole-call p50 / p99, energy per call over the 4 chips:

| preset | placement | steady p50 / p99 ms | steady J / call | cool p50 ms | cool J / call | hot chip first clamped |
|---|---|---|---|---|---|---|
| c2 | legacy | 61.38 / 61.63 | 20.14 | 37.54 | 14.06 | 165 s |
| c2 | thermal (rows 1/8/7/7, EXPERT tt2, cameras on coordinates 1 / 3) | 42.13 / 42.82 | 15.48 | 37.98 | 14.14 | 391-421 s |
| c4 | legacy | 80.59 / 80.80 | 27.76 | 49.21 | 19.23 | 180 s |
| c4 | thermal (rows 10 x 4, EXPERT tt2) | 64.77 / 66.35 | 23.97 | 49.11 | 19.26 | 285 s |

B - A over the steady windows (95 % CI over runs): c2 -31.2 % time (+-1.7 %), -21.1 % energy (+-5.2 %); c4 -19.5 %
time (+-1.6 %), -12.4 % energy (+-3.2 %). At steady state the other chips stay at ~1312-1350 MHz.

## Multi-robot at thermal steady state (`energy_profile --mode mr --robots 4`)

One closed-loop robot per chip, c2 L224 64 rows, N = 10, 2 runs from < 50 °C, in-process:

| | p50 ms | p99 ms | req / s | J / request | chips at steady state |
|---|---:|---:|---:|---:|---|
| cool | 53.1 | | | | |
| steady (run 1 / run 2) | 83.03 / 83.02 | 84.22 / 83.94 | 52.18 / 52.37 | 6.65 / 6.64 | tt0, tt1, tt3 at 800 MHz, tt2 1118 MHz |

First clamp: tt1 at 120 s, tt3 at 195-210 s, tt0 at 255-270 s, tt2 dips at 525 s. This is why the `multi-robot`
profile calibrates with `PI05_SLO_CLOCK_GUARD=1.69` (1350 / 800): its p50 100 / p99 110 ms SLO admits these 4 robots
(R_max 4: the guarded cycle + host time of one robot per chip was 93.3-93.9 ms at the packaged server's starts, two
robots per chip 174.9-177.8 ms; 95.5 / 174.9 ms in a host run from the worktree).

Over HTTP, cool (image `71b34a39bc25`, 2026-10-11, served with `tt-model serve`, 2 cameras, H 50; prompts of 200 tokens in the 1-client rows and 143 tokens in the 4-robot rows, both bucket 224; client time per request):

| profile | load | requests | req / s | client p50 / p99 ms | server-side p50 / p99 ms |
|---|---|---:|---:|---|---|
| `single-robot` | 1 client | 200 | | 41.63 / 42.86 | 40.16 / 40.85 (request time) |
| `multi-robot` | 1 robot | 100 | | 56.38 / 56.92 | 54.72 / 55.26 (request time) |
| `multi-robot` | 4 robots, 60 s (a 5th robot got 503) | 4130 | 68.8 | 57.13 / 61.39 | 55.25 / 58.74 (HTTP counter, last 4096) |
| `disaggregated` | 1 client | 100 | | 58.43 / 59.37 | 56.40 / 57.22 (request time) |
| `disaggregated` | 4 clients, 30 s | 1500 | 49.9 | 78.7 / 101.3 | |

Re-check on the previous published image (`513c62c2b626`, same server code; only test-path and comment changes): `single-robot` 1
client, 200 requests: client p50 41.67 / p99 42.49 ms; `multi-robot` 4 robots, 60 s: 4070 requests, 67.8 req/s, client p50
57.97 / p99 62.39 ms, 0 errors, a 5th robot 503. The served actions equal the table run's (bit-identical to one chip).

This revision's image (`d918b7cb130a`: serial + acknowledged K / V feed, hang watchdog; 2026-10-13, cool, same clients;
the served actions of all 3 profiles equal the previous release's, 6 / 6 requests each):

| profile | load | requests | req / s | client p50 / p99 ms | server-side p50 / p99 ms |
|---|---|---:|---:|---|---|
| `single-robot` | 1 client | 200 | | 41.40 / 42.38 | 39.93 / 40.84 (request time) |
| `multi-robot` | 1 robot | 100 | | 55.33 / 56.01 | 53.84 / 54.33 (request time) |
| `multi-robot` | 4 robots, 60 s (a 5th robot got 503) | 4182 | 69.7 | 56.39 / 60.36 | 54.53 / 57.85 (HTTP counter, last 4096); 0 over the SLO |
| `disaggregated` | 1 client | 100 | | 57.83 / 58.68 | 55.90 / 56.45 (request time) |
| `disaggregated` | 4 clients, 30 s | 1522 | 50.7 | 77.5 / 99.8 | |

## Multi-robot with looser SLOs (`tests/perf/bench_multirobot.py`, in-process, hot boards)

H 50, 2 cameras, 224 tokens; worst repeat p50 / p99 ms; the boards ran hot, so the cycles include throttling; `v1` used
least-loaded routing, `v2` robot routing (the server's default loop):

| think time | robots | v1 + work | v2 + work | v2 + lateness |
|---|---:|---:|---:|---:|
| 0 | 12 | 287 / 314 | 253 / 258 | 253 / 257 |
| 0 | 16 | 378 / 401 | 336 / 356 | 347 / 352 |
| 100 ms | 12 | 206 / 285 | 251 / 253 | 252 / 258 |
| 100 ms | 16 | 340 / 421 | 335 / 349 | 254 / 272 |
| 100 ms | 20 | 349 / 692 | 419 / 467 | 335 / 346 |

With an SLO of 300 ms: 12 robots at zero think time, 16 with 100 ms think time under the lateness rule.

## Energy work (current kernels vs 89d0ceb, `energy_profile`, one run each, back to back, from < 50 °C)

L224, 64 rows, N = 10; "hot (5 min)" = after 300 s of load (not the ≥ 12 min steady state above); energy over the loaded
chips.

| config | phase | p50 ms | energy mJ / request |
|---|---|---|---|
| 1 chip c2 | cool | 55.76 -> 52.75 (-5.4 %) | 7001 -> 6656 (-4.9 %) |
| 1 chip c2 | hot (5 min) | 57.63 -> 54.70 (-5.1 %) | 7154 -> 6810 (-4.8 %) |
| 1 chip c4 | cool | 93.57 -> 89.16 (-4.7 %) | 11637 -> 11030 (-5.2 %) |
| 1 chip c4 | hot (5 min) | 96.66 -> 92.05 (-4.8 %) | 11711 -> 11499 (-1.8 %, within noise) |
| SP mesh c2 (legacy placement) | cool | 39.03 -> 37.54 (-3.8 %) | 14402 -> 14159 (-1.7 %, within noise) |
| SP mesh c2 (legacy placement) | hot (5 min) | 63.94 -> 61.37 (-4.0 %) | 20268 -> 19650 (-3.0 %) |
| SP mesh c4 (legacy placement) | cool | 51.61 -> 49.19 (-4.7 %) | 20046 -> 18892 (-5.8 %) |
| SP mesh c4 (legacy placement) | hot (5 min) | 84.43 -> 80.58 (-4.6 %) | 28384 -> 26782 (-5.6 %) |
| 4 robots (dp) c2 | cool | 56.20 -> 53.16 (-5.4 %) | 7049 -> 6614 (-6.2 %) |
| 4 robots (dp) c2 | hot (5 min) | 59.19 -> 56.61 (-4.4 %) | 7093 -> 6733 (-5.1 %) |

4 robots throughput: 71.0 -> 75.0 req/s cool (+5.6 %), 58.6 -> 61.4 req/s hot (+4.8 %). Interleaved A / B with CIs
(earlier runs): dp 4 robots steady -5.3 % energy / request (+-0.6 %), +5.4 % throughput; SP mesh steady -3.3 % energy
(+-0.9 %), -4.0 % time. (The 4-robot runs here were 5 min; the 12 min multi-robot run above is hotter, 3 chips clamped.)

## Extra prompt bucket (c2 L160 vs L224, 1 chip, `energy_profile --mode single`, 143 tokens)

PREFIX p50 28.83 -> 27.46 ms cool, 29.89 -> 28.38 ms after 5 min; a whole call 55.75 -> 54.38 / 57.24 -> 55.92 ms; actions
bit-identical to L224. Energy not resolved.

## Earlier measurements (2026-10-04, the previous release's kernels, cool, `bench_multichip.py`)

These were measured before the energy work, the extra buckets, the SLO scheduler (least-loaded routing) and the
single-robot mesh; kept for the layout comparison (dp vs pd vs hybrid).

Closed-loop robots, in-process (`code/models/experimental/pi0/tests/perf/bench_multichip.py`, 10 s per point): each
robot sends a request, waits for its actions, then sends the next one (no think time). Every request: random 224 x 224
images, a right-padded prompt of 143 real tokens (prompt bucket 224), fixed noise, N = 10. Each point: throughput
(action chunks / s over the window), p50 and p90 latency (submit -> actions on the host), the mean number of requests in
the expert batch. Measured 2026-10-04 with tt-metal f856a38a361 and the programs of this release (wide gate|up, fused
norms, one VISION program for 3-4 cameras, the expert's merge weights on the diagonal faces).

`multi-robot`: a chip prefills requests as they arrive and runs one batched expert for the prefilled ones when no
request waits (or 4 are prefilled). `disaggregated`: chips 0-2 prefill (shared queue), chip 3 runs the batched expert
on the requests whose slabs arrived (K / V copied through the host, 1.3 ms for 7.8 MB).


### 2 cameras, action chunk 50 (64 action rows)

| layout | robots | chunks/s | p50 ms | p90 ms | mean expert batch |
|---|---:|---:|---:|---:|---:|
| multi-robot (dp, max batch 4, work-conserving) | 1 | 17.9 | 55.5 | 56.0 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 2 | 35.9 | 55.5 | 55.9 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 4 | 70.7 | 56.3 | 57.6 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 6 | 71.7 | 80.0 | 110.3 | 1.59 |
| multi-robot (dp, max batch 4, work-conserving) | 8 | 72.6 | 109.3 | 111.0 | 2.00 |
| multi-robot (dp, max batch 4, work-conserving) | 12 | 72.9 | 163.1 | 165.3 | 3.00 |
| multi-robot (dp, max batch 4, work-conserving) | 16 | 73.2 | 217.2 | 219.6 | 4.00 |
| data parallel, no batching | 1 | 18.0 | 55.4 | 55.7 | 1.00 |
| data parallel, no batching | 2 | 35.4 | 56.2 | 57.1 | 1.00 |
| data parallel, no batching | 4 | 70.5 | 56.4 | 57.5 | 1.00 |
| data parallel, no batching | 6 | 70.5 | 87.6 | 113.0 | 1.00 |
| data parallel, no batching | 8 | 70.4 | 113.1 | 114.9 | 1.00 |
| data parallel, no batching | 12 | 70.2 | 169.8 | 172.8 | 1.00 |
| data parallel, no batching | 16 | 70.5 | 226.2 | 229.0 | 1.00 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 1 | 17.3 | 57.4 | 57.9 | 1.00 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 2 | 34.2 | 58.1 | 58.9 | 1.00 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 4 | 51.6 | 75.9 | 79.0 | 1.51 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 6 | 56.7 | 102.6 | 107.1 | 2.99 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 8 | 56.7 | 153.3 | 159.9 | 3.00 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 12 | 57.0 | 207.8 | 211.6 | 3.00 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 16 | 56.7 | 264.4 | 316.9 | 3.00 |
| disaggregated, max batch 4 | 1 | 17.3 | 57.6 | 58.2 | 1.00 |
| disaggregated, max batch 4 | 2 | 33.8 | 58.5 | 59.5 | 1.00 |
| disaggregated, max batch 4 | 4 | 50.9 | 76.7 | 79.7 | 1.52 |
| disaggregated, max batch 4 | 6 | 56.4 | 103.1 | 107.1 | 2.99 |
| disaggregated, max batch 4 | 8 | 58.0 | 120.7 | 183.8 | 3.61 |
| disaggregated, max batch 4 | 12 | 59.1 | 198.9 | 201.5 | 3.97 |
| disaggregated, max batch 4 | 16 | 59.1 | 266.1 | 270.2 | 3.97 |
| disaggregated, max batch 6 | 1 | 17.1 | 58.1 | 58.5 | 1.00 |
| disaggregated, max batch 6 | 2 | 33.5 | 58.9 | 60.5 | 1.00 |
| disaggregated, max batch 6 | 4 | 50.2 | 78.1 | 81.0 | 1.50 |
| disaggregated, max batch 6 | 6 | 55.8 | 104.2 | 108.6 | 2.99 |
| disaggregated, max batch 6 | 8 | 57.3 | 133.3 | 186.2 | 3.63 |
| disaggregated, max batch 6 | 12 | 59.4 | 197.4 | 200.8 | 5.93 |
| disaggregated, max batch 6 | 16 | 58.8 | 295.5 | 301.9 | 5.94 |
| disaggregated + the expert chip prefills when idle, max batch 3 | 1 | 17.3 | 57.4 | 57.8 | 1.00 |
| disaggregated + the expert chip prefills when idle, max batch 3 | 2 | 34.2 | 57.9 | 58.9 | 1.01 |
| disaggregated + the expert chip prefills when idle, max batch 3 | 4 | 35.6 | 111.2 | 114.3 | 2.49 |
| disaggregated + the expert chip prefills when idle, max batch 3 | 6 | 56.7 | 102.9 | 107.0 | 3.00 |
| disaggregated + the expert chip prefills when idle, max batch 3 | 8 | 56.7 | 153.4 | 159.5 | 3.00 |
| disaggregated + the expert chip prefills when idle, max batch 3 | 12 | 56.7 | 208.2 | 213.8 | 3.00 |
| disaggregated + the expert chip prefills when idle, max batch 3 | 16 | 57.0 | 261.7 | 313.9 | 3.00 |

### 2 cameras, action chunk 10 (32 action rows)

| layout | robots | chunks/s | p50 ms | p90 ms | mean expert batch |
|---|---:|---:|---:|---:|---:|
| multi-robot (dp, max batch 4, work-conserving) | 1 | 18.5 | 53.8 | 54.2 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 2 | 37.0 | 53.9 | 54.5 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 4 | 72.3 | 54.9 | 56.4 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 6 | 79.0 | 74.4 | 96.6 | 1.66 |
| multi-robot (dp, max batch 4, work-conserving) | 8 | 81.8 | 96.8 | 99.0 | 2.00 |
| multi-robot (dp, max batch 4, work-conserving) | 12 | 79.8 | 149.2 | 152.3 | 3.00 |
| multi-robot (dp, max batch 4, work-conserving) | 16 | 84.0 | 188.7 | 191.9 | 4.00 |
| data parallel, no batching | 1 | 18.5 | 53.8 | 54.2 | 1.00 |
| data parallel, no batching | 2 | 36.8 | 53.9 | 54.6 | 1.00 |
| data parallel, no batching | 4 | 72.4 | 54.8 | 56.6 | 1.00 |
| data parallel, no batching | 6 | 72.3 | 82.7 | 109.5 | 1.00 |
| data parallel, no batching | 8 | 72.4 | 110.0 | 112.2 | 1.00 |
| data parallel, no batching | 12 | 72.1 | 166.0 | 169.2 | 1.00 |
| data parallel, no batching | 16 | 71.8 | 221.5 | 226.4 | 1.00 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 1 | 18.0 | 55.5 | 55.8 | 1.00 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 2 | 35.2 | 56.3 | 57.2 | 1.00 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 4 | 60.8 | 61.2 | 78.2 | 1.38 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 6 | 71.8 | 81.2 | 92.0 | 1.80 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 8 | 71.2 | 107.1 | 128.6 | 1.88 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 12 | 71.8 | 163.7 | 178.3 | 1.85 |
| disaggregated (3 prefix + 1 expert chip), max batch 3 | 16 | 72.0 | 220.8 | 236.0 | 1.79 |
| disaggregated, max batch 4 | 1 | 17.9 | 55.5 | 56.0 | 1.00 |
| disaggregated, max batch 4 | 2 | 35.5 | 55.8 | 56.9 | 1.02 |
| disaggregated, max batch 4 | 4 | 61.6 | 61.3 | 77.5 | 1.40 |
| disaggregated, max batch 4 | 6 | 72.7 | 80.3 | 93.6 | 1.75 |
| disaggregated, max batch 4 | 8 | 71.1 | 110.2 | 128.7 | 1.94 |
| disaggregated, max batch 4 | 12 | 71.6 | 166.2 | 178.9 | 1.84 |
| disaggregated, max batch 4 | 16 | 71.5 | 221.6 | 240.1 | 1.84 |
| disaggregated, max batch 6 | 1 | 18.0 | 55.3 | 55.7 | 1.00 |
| disaggregated, max batch 6 | 2 | 35.2 | 56.1 | 57.0 | 1.00 |
| disaggregated, max batch 6 | 4 | 59.9 | 61.7 | 77.0 | 1.33 |
| disaggregated, max batch 6 | 6 | 72.2 | 81.2 | 92.4 | 1.79 |
| disaggregated, max batch 6 | 8 | 72.7 | 110.2 | 123.8 | 1.74 |
| disaggregated, max batch 6 | 12 | 71.6 | 165.3 | 180.1 | 1.85 |
| disaggregated, max batch 6 | 16 | 71.9 | 220.3 | 237.9 | 1.83 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 1 | 17.9 | 55.6 | 56.1 | 1.00 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 2 | 35.1 | 56.3 | 58.6 | 1.02 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 4 | 47.6 | 79.1 | 96.4 | 3.32 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 6 | 61.6 | 95.9 | 99.1 | 3.31 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 8 | 72.0 | 107.5 | 147.6 | 2.49 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 12 | 75.6 | 152.7 | 186.2 | 2.27 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 16 | 75.4 | 211.0 | 239.7 | 2.19 |

### 1 camera, action chunk 10 (32 action rows)

| layout | robots | chunks/s | p50 ms | p90 ms | mean expert batch |
|---|---:|---:|---:|---:|---:|
| multi-robot (dp, max batch 4, work-conserving) | 1 | 23.7 | 42.1 | 42.4 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 2 | 47.4 | 42.0 | 42.4 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 4 | 93.2 | 42.6 | 43.6 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 6 | 104.0 | 56.7 | 72.4 | 1.66 |
| multi-robot (dp, max batch 4, work-conserving) | 8 | 110.2 | 72.2 | 74.0 | 2.00 |
| multi-robot (dp, max batch 4, work-conserving) | 12 | 106.2 | 111.8 | 114.2 | 3.00 |
| multi-robot (dp, max batch 4, work-conserving) | 16 | 113.6 | 139.4 | 141.8 | 4.00 |
| disaggregated, max batch 4 | 1 | 22.9 | 43.5 | 43.9 | 1.00 |
| disaggregated, max batch 4 | 2 | 45.4 | 43.7 | 44.2 | 1.00 |
| disaggregated, max batch 4 | 4 | 71.1 | 55.0 | 58.1 | 1.50 |
| disaggregated, max batch 4 | 6 | 85.2 | 68.1 | 72.5 | 3.00 |
| disaggregated, max batch 4 | 8 | 97.2 | 68.9 | 102.2 | 3.42 |
| disaggregated, max batch 4 | 12 | 105.5 | 100.7 | 133.9 | 3.62 |
| disaggregated, max batch 4 | 16 | 105.9 | 137.1 | 167.8 | 3.63 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 1 | 23.0 | 43.4 | 43.7 | 1.00 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 2 | 45.3 | 43.8 | 44.3 | 1.01 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 4 | 62.4 | 62.3 | 64.9 | 3.89 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 6 | 100.0 | 58.4 | 61.3 | 2.00 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 8 | 97.1 | 69.8 | 102.6 | 3.42 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 12 | 105.6 | 100.2 | 133.4 | 3.62 |
| disaggregated + the expert chip prefills when idle, max batch 4 | 16 | 106.8 | 135.3 | 167.2 | 3.66 |

### 1 camera, action chunk 50 (64 action rows)

| layout | robots | chunks/s | p50 ms | p90 ms | mean expert batch |
|---|---:|---:|---:|---:|---:|
| multi-robot (dp, max batch 4, work-conserving) | 1 | 22.8 | 43.7 | 44.0 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 2 | 45.0 | 44.3 | 45.1 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 4 | 89.7 | 44.3 | 45.5 | 1.00 |
| multi-robot (dp, max batch 4, work-conserving) | 6 | 91.8 | 63.4 | 86.3 | 1.58 |
| multi-robot (dp, max batch 4, work-conserving) | 8 | 93.4 | 84.9 | 86.9 | 2.00 |
| multi-robot (dp, max batch 4, work-conserving) | 12 | 93.9 | 126.6 | 128.9 | 3.00 |
| multi-robot (dp, max batch 4, work-conserving) | 16 | 94.4 | 167.8 | 170.4 | 4.00 |

### Batched expert, one chip (previous kernels, before the energy work; trace replay of the EXPERT program, ms for the whole batch, N = 10)

| cameras | prompt bucket | H | nb 1 | nb 2 | nb 3 | nb 4 | per request at nb 4 |
|---:|---:|---:|---:|---:|---:|---:|---:|
| 1 | 224 | 50 | 17.47 | 30.07 | 44.38 | 58.68 | 14.7 |
| 2 | 64 | 50 | 17.48 | 30.09 | 44.42 | 58.72 | 14.7 |
| 2 | 224 | 50 | 17.92 | 31.04 | 45.90 | 60.70 | 15.2 |
| 3 | 224 | 50 | 18.74 | 32.98 | 48.87 | 64.79 | 16.2 |
| 4 | 32 | 50 | 18.76 | 33.00 | 48.90 | 64.81 | 16.2 |
| 4 | 224 | 50 | 23.97 | 43.58 | 64.73 | 85.85 | 21.5 (row loop) |
| 1 | 224 | 10 | 16.16 | 17.49 | 30.09 | 30.09 | 7.5 (two per slot) |
| 2 | 32 | 10 | 16.18 | 17.49 | 30.09 | 30.12 | 7.5 (two per slot) |
| 2 | 128 | 10 | 16.57 | 17.90 | 31.05 | 31.07 | 7.8 (two per slot) |
| 3 | 224 | 10 | 17.38 | 18.73 | 32.98 | 32.98 | 8.2 (two per slot) |
| 4 | 32 | 10 | 17.40 | 28.36 | 39.60 | 50.80 | 12.7 |
| 4 | 224 | 10 | 17.83 | 29.31 | 41.01 | 52.78 | 13.2 |

The single-request EXPERT program (K / V caches in L1) of 2 cameras / 224 tokens: 17.5 ms (64 rows).

### One chip, every program of a request (previous kernels, before the energy work; trace replay, H 33-64, N = 10)

The single-request programs (one chip, `PI05MegakernelTTNN`). At the current kernels (`energy_profile`, cool, L224):
c2 VISION 9.37 / PREFIX 26.43 / EXPERT 17.48 ms; c4 VISION 18.27 / PREFIX 48.48 / EXPERT 23.51 ms. "first release" = the first release of this package (the p150 programs).

| cameras | prompt bucket | VISION | PREFIX | EXPERT | total (one trace) | first release | p150a |
|---:|---:|---:|---:|---:|---:|---:|---:|
| 1 | 32 | 6.96 | 16.54 | 16.32 | 39.86 | - | - |
| 1 | 64 | 6.97 | 16.94 | 16.33 | 39.87 | - | - |
| 1 | 128 | 6.98 | 16.94 | 16.64 | 40.88 | - | - |
| 1 | 224 | 6.97 | 18.98 | 17.04 | 42.13 | - | - |
| 2 | 32 | 9.97 | 24.68 | 17.05 | 51.36 | 54.14 | 53.68 |
| 2 | 64 | 9.96 | 24.82 | 17.06 | 51.48 | 54.32 | 53.72 |
| 2 | 128 | 9.99 | 26.24 | 17.46 | 52.85 | 55.65 | 54.85 |
| 2 | 224 | 9.95 | 27.60 | 17.46 | 54.09 | 56.79 | 55.57 |
| 3 | 32 | 16.92 | 36.20 | 17.88 | 70.35 | - | - |
| 3 | 64 | 16.93 | 36.55 | 17.88 | 70.73 | - | - |
| 3 | 128 | 16.94 | 36.96 | 17.90 | 71.07 | - | - |
| 3 | 224 | 16.91 | 39.44 | 18.32 | 73.46 | - | - |
| 4 | 32 | 19.90 | 46.62 | 18.32 | 84.00 | - | - |
| 4 | 64 | 19.89 | 46.77 | 22.69 | 88.30 | - | - |
| 4 | 128 | 19.90 | 48.87 | 23.52 | 90.46 | - | - |
| 4 | 224 | 19.91 | 49.83 | 23.51 | 91.65 | - | - |