tt-model push ornith-1.5-35b-a3b (container) (part 3)

.gitattributes

@@ -108,3 +108,17 @@ image/blobs/sha256/c89021dfdc17c56095df3fa75bd153f2905a5fdf013021f7b6b69df4e7be6
 image/blobs/sha256/f1e1d8efd066a8614ec11d3cdaaa344ec94ea6a837dd6fc97fab6c891d7bfc92 filter=lfs diff=lfs merge=lfs -text
 image/blobs/sha256/f75bb4510cf8bf2f78d6dd2fd74dff9c4c99c23fd1c4753cda908446f04dcef0 filter=lfs diff=lfs merge=lfs -text
 image/blobs/sha256/fa321c822076f2819f538dfe916ccc0dbf6ebf757f0cb7b49a21b9147a5763a5 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/0a2d1d3ac4dd4a9b97b4b36d3b9c33fd7000be09be485d76a275cde6b4746e59 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/0ab7ba472d2c55edefa1cf5f7f3e6531b8053eeff029c88ff48dc2fce8052c9a filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/2fb2fadb0dd1bdb50f6dc1675cc4de2f6a15a625f5db9771c9cd3584ba5a524c filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/4bd0ed151188eb03b1c73f6f1a45c5e91ca14ea4ebb07c0e3a3570ca3cb7cb62 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/4c5662049740585a4856802bfe87058375a729d038f90304d86be187c0f22f32 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/60a24d7bdb99b37e3524309c9a45b649f0869a088137c6396e33bcf88a837f3d filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/7ff93218a1f461f0ff7de29caaae299027fab21dccd93fd62d14156e127755d6 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/940f6ae42c997504c9b9e88c251ac61e84464c9df9f7891766696355cbfe0da9 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/97c8a3d25d09f696e6e83b7473d7e14b2ea600392714355b93223d98e52696d8 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/a69718c502a0ae46d1f411856c7b1386309f93266af2300428077847bd558ee3 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/bcdb3d1b52e2726654acc1fe4fb0358e26fecd2e4ddfb3425696e98bc0cb1a51 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/c080bfa67ae4d74a8da698e7749cd6d68552e3080cb7118f91c2033161097d43 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/ebde235144ede87b0d3ac60b1ddf31e48d09fdebcfa2ec2f0063b94a62de5b83 filter=lfs diff=lfs merge=lfs -text
+image/blobs/sha256/f99c891cf7810365d8f2f0f08d964da3b6a207fc911cdfb161c1b61b1c7f7c75 filter=lfs diff=lfs merge=lfs -text

image/blobs/sha256/0a2d1d3ac4dd4a9b97b4b36d3b9c33fd7000be09be485d76a275cde6b4746e59

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a2d1d3ac4dd4a9b97b4b36d3b9c33fd7000be09be485d76a275cde6b4746e59
+size 766469

image/blobs/sha256/0ab7ba472d2c55edefa1cf5f7f3e6531b8053eeff029c88ff48dc2fce8052c9a

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ab7ba472d2c55edefa1cf5f7f3e6531b8053eeff029c88ff48dc2fce8052c9a
+size 1388656960

image/blobs/sha256/1cdaabe0a2d653efb168fd2a169808b75b553a8fd092af096a5e7b17af2610bd

@@ -0,0 +1,126 @@
+{
+  "schemaVersion": 2,
+  "mediaType": "application/vnd.oci.image.manifest.v1+json",
+  "config": {
+    "mediaType": "application/vnd.oci.image.config.v1+json",
+    "digest": "sha256:b6cf338c8b67f26a22cf5cf521de5384f082fc6f8b6277fc3a825af6f5a35180",
+    "size": 15930
+  },
+  "layers": [
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:0626d3cc316c9fea3d4891e2ec19b8f371fe22ffb915622f6072a2ee2ca9017f",
+      "size": 29752488
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:c080bfa67ae4d74a8da698e7749cd6d68552e3080cb7118f91c2033161097d43",
+      "size": 69478212
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:2ae30a9fd16710d03b00efdfc79603fd5281ea25e0b5d104dbd43f690481bca5",
+      "size": 4400
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:4c5662049740585a4856802bfe87058375a729d038f90304d86be187c0f22f32",
+      "size": 30938232
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:940f6ae42c997504c9b9e88c251ac61e84464c9df9f7891766696355cbfe0da9",
+      "size": 138994710
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:97c8a3d25d09f696e6e83b7473d7e14b2ea600392714355b93223d98e52696d8",
+      "size": 65895425
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:0ab7ba472d2c55edefa1cf5f7f3e6531b8053eeff029c88ff48dc2fce8052c9a",
+      "size": 1388656960
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:8abd8ce75f0ca026a80af1fbc547e289505ad0d37b453b2816951524afda8d3d",
+      "size": 115
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:60a24d7bdb99b37e3524309c9a45b649f0869a088137c6396e33bcf88a837f3d",
+      "size": 139077062
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:7ff93218a1f461f0ff7de29caaae299027fab21dccd93fd62d14156e127755d6",
+      "size": 38849281
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:f99c891cf7810365d8f2f0f08d964da3b6a207fc911cdfb161c1b61b1c7f7c75",
+      "size": 38849127
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:4bd0ed151188eb03b1c73f6f1a45c5e91ca14ea4ebb07c0e3a3570ca3cb7cb62",
+      "size": 58403387
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:2fb2fadb0dd1bdb50f6dc1675cc4de2f6a15a625f5db9771c9cd3584ba5a524c",
+      "size": 11818018
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:a69718c502a0ae46d1f411856c7b1386309f93266af2300428077847bd558ee3",
+      "size": 1729216
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:e1d70dd4872043b59108086de8c84903998e542c9cfba4679ff1f22f4089ee96",
+      "size": 6755
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:ebde235144ede87b0d3ac60b1ddf31e48d09fdebcfa2ec2f0063b94a62de5b83",
+      "size": 3609277
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:bcdb3d1b52e2726654acc1fe4fb0358e26fecd2e4ddfb3425696e98bc0cb1a51",
+      "size": 26673453
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:b34bde9dfb9910f28026e11d228d3b1239a3d5a30b65137811c6e17a787dad5a",
+      "size": 1079
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:58e4beb1037631cccf62c362c01c579f402d94671765a32741c4a917197f8232",
+      "size": 1393
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
+      "size": 32
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:7f1fcf02808f34fbdae045185153d7c65ee5144f9f13b55118b6bb4134209519",
+      "size": 869
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:0a2d1d3ac4dd4a9b97b4b36d3b9c33fd7000be09be485d76a275cde6b4746e59",
+      "size": 766469
+    },
+    {
+      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
+      "digest": "sha256:ef6a9fc8dfbb5f0d324a39b73fd5f7ae958a1e69256c47bac74a34cb694feaa8",
+      "size": 4253
+    }
+  ]
+}

image/blobs/sha256/2fb2fadb0dd1bdb50f6dc1675cc4de2f6a15a625f5db9771c9cd3584ba5a524c

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2fb2fadb0dd1bdb50f6dc1675cc4de2f6a15a625f5db9771c9cd3584ba5a524c
+size 11818018

image/blobs/sha256/4bd0ed151188eb03b1c73f6f1a45c5e91ca14ea4ebb07c0e3a3570ca3cb7cb62

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4bd0ed151188eb03b1c73f6f1a45c5e91ca14ea4ebb07c0e3a3570ca3cb7cb62
+size 58403387

image/blobs/sha256/4c5662049740585a4856802bfe87058375a729d038f90304d86be187c0f22f32

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c5662049740585a4856802bfe87058375a729d038f90304d86be187c0f22f32
+size 30938232

image/blobs/sha256/60a24d7bdb99b37e3524309c9a45b649f0869a088137c6396e33bcf88a837f3d

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60a24d7bdb99b37e3524309c9a45b649f0869a088137c6396e33bcf88a837f3d
+size 139077062

image/blobs/sha256/78602e7aa61c680b9692967f3000ddef670cfbebf5346c9e346aa2cacd939bbe

@@ -0,0 +1 @@
+{"_type":"https://in-toto.io/Statement/v0.1","predicateType":"https://slsa.dev/provenance/v1","subject":[{"name":"pkg:docker/tt-model/ornith-1.5-35b-a3b@build-d9ed49c49?platform=linux%2Famd64","digest":{"sha256":"1cdaabe0a2d653efb168fd2a169808b75b553a8fd092af096a5e7b17af2610bd"}}],"predicate":{"buildDefinition":{"buildType":"https://github.com/moby/buildkit/blob/master/docs/attestations/slsa-definitions.md","resolvedDependencies":[{"uri":"pkg:docker/docker/[email protected]","digest":{"sha256":"b99fecfe00268a8b556fad7d9c37ee25d716ae08a5d7320e6d51c4dd83246894"}},{"uri":"pkg:docker/[email protected]?platform=linux%2Famd64","digest":{"sha256":"e0f80c829509ef4df9e76bd8944eb7b8ec25cbb379b112a7f9448b2c77a77c9b"}},{"uri":"pkg:docker/ghcr.io/tenstorrent/tt-metal/tt-metalium/ubuntu-22.04-dev-amd64@latest?platform=linux%2Famd64","digest":{"sha256":"d2de3da0f7ed3afbdebfe0eae042ab4da6bee0e92c66e440ca2ff485ba621e67"}}],"externalParameters":{"configSource":{"path":"Dockerfile"},"request":{"frontend":"gateway.v0","args":{"cmdline":"docker/dockerfile:1.7-labs","context:metalsrc":"local:metalsrc","frontend.caps":"moby.buildkit.frontend.contexts+forward","sharedkey:localdir:metalsrc":"metalsrc:db3d05e7ef638029","source":"docker/dockerfile:1.7-labs"},"locals":[{"name":"context"},{"name":"dockerfile"},{"name":"metalsrc"}],"root":{"configSource":{"path":"Dockerfile"},"request":{"args":{"context:metalsrc":"local:metalsrc","frontend.caps":"moby.buildkit.frontend.contexts+forward","sharedkey:localdir:metalsrc":"metalsrc:db3d05e7ef638029","vcs:localdir:context":"build/ornith-1.5-35b-a3b/ctx","vcs:localdir:dockerfile":"build/ornith-1.5-35b-a3b/ctx","vcs:localdir:metalsrc":"build/ornith-1.5-35b-a3b/ctx/metalsrc","vcs:revision":"d9ed49c49330298e207d9532d173c9d6a09ff81a","vcs:source":"https://github.com/rnabeelTT/tt-metal.git"}}},"compatibilityVersion":20}},"internalParameters":{"builderPlatform":"linux/amd64"}},"runDetails":{"builder":{"id":""},"metadata":{"invocationId":"7heng4ysdkp5lgn3s9tju2jvc","startedOn":"2026-10-28T16:16:04.119507082Z","finishedOn":"2026-10-28T16:23:47.663171762Z","buildkit_metadata":{"vcs":{"localdir:context":"build/ornith-1.5-35b-a3b/ctx","localdir:dockerfile":"build/ornith-1.5-35b-a3b/ctx","localdir:metalsrc":"build/ornith-1.5-35b-a3b/ctx/metalsrc","revision":"d9ed49c49330298e207d9532d173c9d6a09ff81a","source":"https://github.com/rnabeelTT/tt-metal.git"}},"buildkit_completeness":{"request":false,"resolvedDependencies":false}}}}}

image/blobs/sha256/7aaaeb94261a67a4f8106192d8f0d2d11f23d59f004821d3e123fbe348e28315

@@ -0,0 +1,19 @@
+{
+  "schemaVersion": 2,
+  "mediaType": "application/vnd.oci.image.manifest.v1+json",
+  "config": {
+    "mediaType": "application/vnd.oci.image.config.v1+json",
+    "digest": "sha256:ad25ec64ccc9394cfafc7d49bc4eb00efd33b0efdec082ebd3e2cf727b245e5a",
+    "size": 167
+  },
+  "layers": [
+    {
+      "mediaType": "application/vnd.in-toto+json",
+      "digest": "sha256:78602e7aa61c680b9692967f3000ddef670cfbebf5346c9e346aa2cacd939bbe",
+      "size": 2468,
+      "annotations": {
+        "in-toto.io/predicate-type": "https://slsa.dev/provenance/v1"
+      }
+    }
+  ]
+}

image/blobs/sha256/7ff93218a1f461f0ff7de29caaae299027fab21dccd93fd62d14156e127755d6

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ff93218a1f461f0ff7de29caaae299027fab21dccd93fd62d14156e127755d6
+size 38849281

image/blobs/sha256/940f6ae42c997504c9b9e88c251ac61e84464c9df9f7891766696355cbfe0da9

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:940f6ae42c997504c9b9e88c251ac61e84464c9df9f7891766696355cbfe0da9
+size 138994710

image/blobs/sha256/97c8a3d25d09f696e6e83b7473d7e14b2ea600392714355b93223d98e52696d8

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97c8a3d25d09f696e6e83b7473d7e14b2ea600392714355b93223d98e52696d8
+size 65895425

image/blobs/sha256/9f0546abcdb8140fc1aac8e20141b19039f4e4a73f1805cbea62d51936ce4649

@@ -0,0 +1,28 @@
+{
+  "schemaVersion": 2,
+  "mediaType": "application/vnd.oci.image.index.v1+json",
+  "manifests": [
+    {
+      "mediaType": "application/vnd.oci.image.manifest.v1+json",
+      "digest": "sha256:1cdaabe0a2d653efb168fd2a169808b75b553a8fd092af096a5e7b17af2610bd",
+      "size": 4693,
+      "platform": {
+        "architecture": "amd64",
+        "os": "linux"
+      }
+    },
+    {
+      "mediaType": "application/vnd.oci.image.manifest.v1+json",
+      "digest": "sha256:7aaaeb94261a67a4f8106192d8f0d2d11f23d59f004821d3e123fbe348e28315",
+      "size": 564,
+      "annotations": {
+        "vnd.docker.reference.digest": "sha256:1cdaabe0a2d653efb168fd2a169808b75b553a8fd092af096a5e7b17af2610bd",
+        "vnd.docker.reference.type": "attestation-manifest"
+      },
+      "platform": {
+        "architecture": "unknown",
+        "os": "unknown"
+      }
+    }
+  ]
+}

image/blobs/sha256/a69718c502a0ae46d1f411856c7b1386309f93266af2300428077847bd558ee3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a69718c502a0ae46d1f411856c7b1386309f93266af2300428077847bd558ee3
+size 1729216

image/blobs/sha256/ad25ec64ccc9394cfafc7d49bc4eb00efd33b0efdec082ebd3e2cf727b245e5a

@@ -0,0 +1 @@
+{"architecture":"unknown","os":"unknown","config":{},"rootfs":{"type":"layers","diff_ids":["sha256:78602e7aa61c680b9692967f3000ddef670cfbebf5346c9e346aa2cacd939bbe"]}}

image/blobs/sha256/b6cf338c8b67f26a22cf5cf521de5384f082fc6f8b6277fc3a825af6f5a35180

@@ -0,0 +1 @@
+{"architecture":"amd64","config":{"User":"tt","Env":["PATH=/opt/tt-venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","VENV=/opt/tt-venv","VIRTUAL_ENV=/opt/tt-venv","TT_METAL_RUNTIME_ROOT=/opt/tt-metal","TT_METAL_HOME=/opt/tt-metal","PYTHONPATH=/opt/tt-metal","LD_LIBRARY_PATH=/opt/tt-metal/build/lib:/opt/openmpi-v5.0.7-ulfm/lib","EXTRA_MODELS_DIR=/opt/tt-metal/models/autoports/ornith_ai_ornith_1_5_35b_a3b/vllm_bundle","TT_VLLM_BUILTIN_MODELS=0","TT_MODEL_KIND=vllm-plugin","HF_HOME=/hf","TT_METAL_CACHE=/cache","HOME=/home/tt","USER=tt","LOGNAME=tt"],"Entrypoint":["/usr/local/bin/entrypoint.sh"],"Cmd":["/usr/local/bin/serve-default.sh"],"WorkingDir":"/home/tt/work","Labels":{"org.opencontainers.image.revision":"d9ed49c49330298e207d9532d173c9d6a09ff81a","org.opencontainers.image.version":"22.04","org.tenstorrent.tt-model":"ornith-1.5-35b-a3b","org.tenstorrent.tt-model.arch":"blackhole","org.tenstorrent.tt-model.kind":"vllm-plugin","org.tenstorrent.tt-model.plugin":"e37ccd87d88f87628ffb63e24bcd3d2f16350936","org.tenstorrent.tt-model.profiles":"p300x2,p150x4,p300,p150x2,p150,p100","org.tenstorrent.tt-model.repo":"raahemnabeel/ornith-1.5-35b-a3b","org.tenstorrent.tt-model.tt-metal":"v0.79.0-dev20260909-320-gd9ed49c493","org.tenstorrent.tt-model.weights":"ornith-ai/Ornith-1.5-35B-A3B"},"ArgsEscaped":true},"created":"2026-10-28T16:22:31.266664396Z","history":[{"created":"2026-10-25T20:10:42.179339716Z","created_by":"/bin/sh -c #(nop)  ARG RELEASE","empty_layer":true},{"created":"2026-10-25T20:10:42.215422845Z","created_by":"/bin/sh -c #(nop)  ARG LAUNCHPAD_BUILD_ARCH","empty_layer":true},{"created":"2026-10-25T20:10:42.244053327Z","created_by":"/bin/sh -c #(nop)  LABEL org.opencontainers.image.version=22.04","empty_layer":true},{"created":"2026-10-25T20:10:44.32185261Z","created_by":"/bin/sh -c #(nop) ADD file:ba52f9eec2a5a7de74e9a460017725370755c2d4814dd746bde7507601befb53 in / "},{"created":"2026-10-25T20:10:44.698249596Z","created_by":"/bin/sh -c #(nop)  CMD [\"/bin/bash\"]","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"ARG OMPI_DIR=/opt/openmpi-v5.0.7-ulfm","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"ARG EXTRA_MODELS_DIR=","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"ARG TT_MODEL_KIND","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"ARG MODEL_NAME","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"ARG MODEL_REPO","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"ARG MODEL_WEIGHTS","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"ARG MODEL_ARCH","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"ARG MODEL_PROFILES","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"ARG MODEL_TT_METAL_SHA","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"ARG MODEL_TT_METAL_DESCRIBE","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"ARG MODEL_PLUGIN_SHA","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:16:12.54575175Z","created_by":"RUN |11 OMPI_DIR=/opt/openmpi-v5.0.7-ulfm EXTRA_MODELS_DIR=/opt/tt-metal/models/autoports/ornith_ai_ornith_1_5_35b_a3b/vllm_bundle TT_MODEL_KIND=vllm-plugin MODEL_NAME=ornith-1.5-35b-a3b MODEL_REPO=raahemnabeel/ornith-1.5-35b-a3b MODEL_WEIGHTS=ornith-ai/Ornith-1.5-35B-A3B MODEL_ARCH=blackhole MODEL_PROFILES=p300x2,p150x4,p300,p150x2,p150,p100 MODEL_TT_METAL_SHA=d9ed49c49330298e207d9532d173c9d6a09ff81a MODEL_TT_METAL_DESCRIBE=v0.79.0-dev20260909-320-gd9ed49c493 MODEL_PLUGIN_SHA=e37ccd87d88f87628ffb63e24bcd3d2f16350936 /bin/sh -c apt-get update \u0026\u0026 apt-get install -y --no-install-recommends     libhwloc15 libnuma1 libatomic1 libudev1 libcap2 zlib1g     libmpc3 libmpfr6 libgmp10 libzstd1     libevent-core-2.1-7 libevent-pthreads-2.1-7     libgl1 libsndfile1 ca-certificates     \u0026\u0026 apt-get clean \u0026\u0026 rm -rf /var/lib/apt/lists/* # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:16:12.675351736Z","created_by":"RUN |11 OMPI_DIR=/opt/openmpi-v5.0.7-ulfm EXTRA_MODELS_DIR=/opt/tt-metal/models/autoports/ornith_ai_ornith_1_5_35b_a3b/vllm_bundle TT_MODEL_KIND=vllm-plugin MODEL_NAME=ornith-1.5-35b-a3b MODEL_REPO=raahemnabeel/ornith-1.5-35b-a3b MODEL_WEIGHTS=ornith-ai/Ornith-1.5-35B-A3B MODEL_ARCH=blackhole MODEL_PROFILES=p300x2,p150x4,p300,p150x2,p150,p100 MODEL_TT_METAL_SHA=d9ed49c49330298e207d9532d173c9d6a09ff81a MODEL_TT_METAL_DESCRIBE=v0.79.0-dev20260909-320-gd9ed49c493 MODEL_PLUGIN_SHA=e37ccd87d88f87628ffb63e24bcd3d2f16350936 /bin/sh -c existing=\"$(getent passwd 1000 | cut -d: -f1)\"     \u0026\u0026 if [ -n \"$existing\" ]; then          userdel -r \"$existing\" 2\u003e/dev/null || userdel \"$existing\";        fi     \u0026\u0026 useradd --uid 1000 --create-home --home-dir /home/tt --shell /bin/bash tt     \u0026\u0026 mkdir -p /home/tt/work/logs /cache /opt/tt-metal     \u0026\u0026 chown -R tt:tt /home/tt /cache /opt/tt-metal     \u0026\u0026 chmod 1777 /home/tt /home/tt/work /home/tt/work/logs /cache # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:25.15275685Z","created_by":"COPY /opt/openmpi-v5.0.7-ulfm /opt/openmpi-v5.0.7-ulfm # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:27.431724383Z","created_by":"COPY /opt/tenstorrent /opt/tenstorrent # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:35.945461051Z","created_by":"COPY /usr/local/share/uv /usr/local/share/uv # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:49.473373285Z","created_by":"COPY /opt/tt-venv /opt/tt-venv # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:49.715817431Z","created_by":"COPY /opt/vllm /opt/vllm # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:50.892368595Z","created_by":"COPY --chown=tt:tt /opt/tt-metal/runtime /opt/tt-metal/runtime # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:51.236731676Z","created_by":"COPY --chown=tt:tt /opt/tt-metal/build_Release /opt/tt-metal/build_Release # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:51.581260374Z","created_by":"COPY --chown=tt:tt /opt/tt-metal/build /opt/tt-metal/build # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:52.275297945Z","created_by":"COPY --chown=tt:tt /opt/tt-metal/tt_metal /opt/tt-metal/tt_metal # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:52.759717281Z","created_by":"COPY --chown=tt:tt /opt/tt-metal/ttnn /opt/tt-metal/ttnn # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:52.865578259Z","created_by":"COPY --chown=tt:tt /opt/tt-metal/tools /opt/tt-metal/tools # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:52.914706602Z","created_by":"COPY --chown=tt:tt /opt/tt-metal/setup.py /opt/tt-metal/pyproject.toml /opt/tt-metal/ # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:53.033887314Z","created_by":"COPY --chown=tt:tt code/ /opt/tt-metal/ # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:59.553526511Z","created_by":"RUN |11 OMPI_DIR=/opt/openmpi-v5.0.7-ulfm EXTRA_MODELS_DIR=/opt/tt-metal/models/autoports/ornith_ai_ornith_1_5_35b_a3b/vllm_bundle TT_MODEL_KIND=vllm-plugin MODEL_NAME=ornith-1.5-35b-a3b MODEL_REPO=raahemnabeel/ornith-1.5-35b-a3b MODEL_WEIGHTS=ornith-ai/Ornith-1.5-35B-A3B MODEL_ARCH=blackhole MODEL_PROFILES=p300x2,p150x4,p300,p150x2,p150,p100 MODEL_TT_METAL_SHA=d9ed49c49330298e207d9532d173c9d6a09ff81a MODEL_TT_METAL_DESCRIBE=v0.79.0-dev20260909-320-gd9ed49c493 MODEL_PLUGIN_SHA=e37ccd87d88f87628ffb63e24bcd3d2f16350936 /bin/sh -c find /opt/tt-metal \\( -type f -o -type d \\)       \\( ! -perm -o+r -o \\( -type d ! -perm -o+x \\) \\) -exec chmod a+rX {} + # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:59.628874319Z","created_by":"COPY --chmod=0755 entrypoint.sh /usr/local/bin/entrypoint.sh # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"COPY --chmod=0755 serve-default.sh /usr/local/bin/serve-default.sh # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV VENV=/opt/tt-venv","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV VIRTUAL_ENV=/opt/tt-venv","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV PATH=/opt/tt-venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV TT_METAL_RUNTIME_ROOT=/opt/tt-metal","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV TT_METAL_HOME=/opt/tt-metal","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV PYTHONPATH=/opt/tt-metal","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV LD_LIBRARY_PATH=/opt/tt-metal/build/lib:/opt/openmpi-v5.0.7-ulfm/lib","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV EXTRA_MODELS_DIR=/opt/tt-metal/models/autoports/ornith_ai_ornith_1_5_35b_a3b/vllm_bundle","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ARG TT_VLLM_BUILTIN_MODELS=","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV TT_VLLM_BUILTIN_MODELS=0","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV TT_MODEL_KIND=vllm-plugin","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV HF_HOME=/hf","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV TT_METAL_CACHE=/cache","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV HOME=/home/tt","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV USER=tt","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"ENV LOGNAME=tt","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.694079565Z","created_by":"USER tt","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:21:59.737497696Z","created_by":"WORKDIR /home/tt/work","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:21:59.803366981Z","created_by":"COPY --chmod=0755 verify.sh /ctx/verify.sh # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:22:31.149697853Z","created_by":"RUN |12 OMPI_DIR=/opt/openmpi-v5.0.7-ulfm EXTRA_MODELS_DIR=/opt/tt-metal/models/autoports/ornith_ai_ornith_1_5_35b_a3b/vllm_bundle TT_MODEL_KIND=vllm-plugin MODEL_NAME=ornith-1.5-35b-a3b MODEL_REPO=raahemnabeel/ornith-1.5-35b-a3b MODEL_WEIGHTS=ornith-ai/Ornith-1.5-35B-A3B MODEL_ARCH=blackhole MODEL_PROFILES=p300x2,p150x4,p300,p150x2,p150,p100 MODEL_TT_METAL_SHA=d9ed49c49330298e207d9532d173c9d6a09ff81a MODEL_TT_METAL_DESCRIBE=v0.79.0-dev20260909-320-gd9ed49c493 MODEL_PLUGIN_SHA=e37ccd87d88f87628ffb63e24bcd3d2f16350936 TT_VLLM_BUILTIN_MODELS=0 /bin/sh -c bash /ctx/verify.sh # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:22:31.149697853Z","created_by":"USER root","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:22:31.266664396Z","created_by":"RUN |12 OMPI_DIR=/opt/openmpi-v5.0.7-ulfm EXTRA_MODELS_DIR=/opt/tt-metal/models/autoports/ornith_ai_ornith_1_5_35b_a3b/vllm_bundle TT_MODEL_KIND=vllm-plugin MODEL_NAME=ornith-1.5-35b-a3b MODEL_REPO=raahemnabeel/ornith-1.5-35b-a3b MODEL_WEIGHTS=ornith-ai/Ornith-1.5-35B-A3B MODEL_ARCH=blackhole MODEL_PROFILES=p300x2,p150x4,p300,p150x2,p150,p100 MODEL_TT_METAL_SHA=d9ed49c49330298e207d9532d173c9d6a09ff81a MODEL_TT_METAL_DESCRIBE=v0.79.0-dev20260909-320-gd9ed49c493 MODEL_PLUGIN_SHA=e37ccd87d88f87628ffb63e24bcd3d2f16350936 TT_VLLM_BUILTIN_MODELS=0 /bin/sh -c chmod -R a+rwX /home/tt # buildkit","comment":"buildkit.dockerfile.v0"},{"created":"2026-10-28T16:22:31.266664396Z","created_by":"USER tt","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:22:31.266664396Z","created_by":"LABEL org.tenstorrent.tt-model=ornith-1.5-35b-a3b org.tenstorrent.tt-model.repo=raahemnabeel/ornith-1.5-35b-a3b org.tenstorrent.tt-model.weights=ornith-ai/Ornith-1.5-35B-A3B org.tenstorrent.tt-model.arch=blackhole org.tenstorrent.tt-model.kind=vllm-plugin org.tenstorrent.tt-model.profiles=p300x2,p150x4,p300,p150x2,p150,p100 org.opencontainers.image.revision=d9ed49c49330298e207d9532d173c9d6a09ff81a org.tenstorrent.tt-model.tt-metal=v0.79.0-dev20260909-320-gd9ed49c493 org.tenstorrent.tt-model.plugin=e37ccd87d88f87628ffb63e24bcd3d2f16350936","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:22:31.266664396Z","created_by":"ENTRYPOINT [\"/usr/local/bin/entrypoint.sh\"]","comment":"buildkit.dockerfile.v0","empty_layer":true},{"created":"2026-10-28T16:22:31.266664396Z","created_by":"CMD [\"/usr/local/bin/serve-default.sh\"]","comment":"buildkit.dockerfile.v0","empty_layer":true}],"os":"linux","rootfs":{"type":"layers","diff_ids":["sha256:297a592b66593a3dddc39e7970a718cc1cbb04c8a9b904009b33eb623dafc588","sha256:dec9bce6abcf77da06683318f35ad70c0d6c37855d5212ad453a895856d241ab","sha256:faf9e2e4571a867af077c05d27a54020b8d3261e2e31ddf2dbd530da76eae154","sha256:da5bc5532f7f7c6c590d1b48a0dcc610c331d0d66ca472fe3f1688781e1c75c2","sha256:cfd63e541c78170d235661b023ac3a574294307f56ecb8ab6635bb0d396cebae","sha256:fdab0f8e1745e63f8df58b3ee5abb01085f8b49c46d68dcee5d4aae0666d8d0f","sha256:d6a458b123271e58a8011afe32b52e92297573f2fea9b54e0fcea5217e036703","sha256:a0dd0c51f479b013f1d307a0b63a1e0938ad228be311974ee554d2e30f162232","sha256:fb1508e9b57ec9f667be54d4bcf56d75f72330c0a7ada61cfa92558df5b78508","sha256:6d79d70a865f891cfc646178a8bad7774775e5834b10cba101caac77498b9682","sha256:b953c199591b551730e2c5cf3b77e495f4509f7156c149346e1608cfae0c5a80","sha256:c308103610f0aaf1e56e0980b80d614367c9c4426a10e1ef73587c7f5290857e","sha256:f8192ce7924818715750d540f7336538efca850ee6ca0d582b0d19e5e425326c","sha256:0799cf0b87f0c696fd85c83cff16c039aac6fd42fc5c9f60f14cd9034de03ee0","sha256:49ac6c8a3e38992aee81329fef8a5556cb3fa9a4661a1c46d72a4f684042a24a","sha256:3ba1ed3abfe881c03f83e788842369fd136826c1c272d8ef61c5fb3ec4ab4800","sha256:e100386bc2db085cd3b6422aa4ca8b993d6ecff91ed3e9ae767462ccad13c0fc","sha256:af39938446b86f8b29b67c63bac761c36d3cac623cd07a9aca51e465ea708787","sha256:ac2e307e8d575b230df6705603c627f9270711936cca63ad54a2f7d85529a0f0","sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef","sha256:1ce55df43f01ba1e36c506174fac04b2cf3bb3d32dc38ff839c32cccb8ad4d68","sha256:b4faf192c724affaedcfbc5347414009de92c96b17449367b623bc83616652e9","sha256:c8a5f03a87454a5acdc326ad8d862c24e2b4f441dc89ae4f504e12bafd215273"]}}

image/blobs/sha256/bcdb3d1b52e2726654acc1fe4fb0358e26fecd2e4ddfb3425696e98bc0cb1a51

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcdb3d1b52e2726654acc1fe4fb0358e26fecd2e4ddfb3425696e98bc0cb1a51
+size 26673453

image/blobs/sha256/c080bfa67ae4d74a8da698e7749cd6d68552e3080cb7118f91c2033161097d43

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c080bfa67ae4d74a8da698e7749cd6d68552e3080cb7118f91c2033161097d43
+size 69478212

image/blobs/sha256/ebde235144ede87b0d3ac60b1ddf31e48d09fdebcfa2ec2f0063b94a62de5b83

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebde235144ede87b0d3ac60b1ddf31e48d09fdebcfa2ec2f0063b94a62de5b83
+size 3609277

image/blobs/sha256/f99c891cf7810365d8f2f0f08d964da3b6a207fc911cdfb161c1b61b1c7f7c75

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f99c891cf7810365d8f2f0f08d964da3b6a207fc911cdfb161c1b61b1c7f7c75
+size 38849127

image/index.json

@@ -1 +1 @@
-{"schemaVersion":2,"mediaType":"application/vnd.oci.image.index.v1+json","manifests":[{"mediaType":"application/vnd.oci.image.index.v1+json","digest":"sha256:fe0482ed6aa240d749ca05250027a25cddf1e7172f2b6ea1b0bf03da94a9323a","size":856,"annotations":{"io.containerd.image.name":"docker.io/tt-model/ornith-1.5-35b-a3b:fe0482ed6aa2","org.opencontainers.image.ref.name":"fe0482ed6aa2"}}]}
+{"schemaVersion":2,"mediaType":"application/vnd.oci.image.index.v1+json","manifests":[{"mediaType":"application/vnd.oci.image.index.v1+json","digest":"sha256:9f0546abcdb8140fc1aac8e20141b19039f4e4a73f1805cbea62d51936ce4649","size":856,"annotations":{"io.containerd.image.name":"docker.io/tt-model/ornith-1.5-35b-a3b:9f0546abcdb8","org.opencontainers.image.ref.name":"9f0546abcdb8"}}]}

image/manifest.json

@@ -1 +1 @@
-[{"Config":"blobs/sha256/c401e8efe49f5f70f9d0f181fddc96a2e03d62964cccb4186ce59f8fd475ccc4","RepoTags":["tt-model/ornith-1.5-35b-a3b:fe0482ed6aa2"],"Layers":["blobs/sha256/0626d3cc316c9fea3d4891e2ec19b8f371fe22ffb915622f6072a2ee2ca9017f","blobs/sha256/9034c2f0a8726ba1ff6d7f7ae25697e56e85727344be61f682052f69dc89ee3c","blobs/sha256/e9a88b89633244b6b3eab9b1f9b6fafd8a52d7c068d5cf7b3e45370ffd139c30","blobs/sha256/137117efb1ba4fb3e8504bc1a172bb5cedbea7793b4fbe6ad0a5d973d653d833","blobs/sha256/0f907a13c1408088f97fbf69cdf1c75754905e3e125ab8cdeba117f577b8c75f","blobs/sha256/c89021dfdc17c56095df3fa75bd153f2905a5fdf013021f7b6b69df4e7be6796","blobs/sha256/37bc78f8cba50f66d9083d23e700b721aa4d5c0765bf106803d1496a3412dda4","blobs/sha256/c0f66d611b4c975e3d25c393a44d4b7a0a87ebeede6344b16506abb96deea93d","blobs/sha256/094d61fecec11d7c60840ec970ef5cdf7bdedde6b6e91572065ea23e7f750f64","blobs/sha256/f1e1d8efd066a8614ec11d3cdaaa344ec94ea6a837dd6fc97fab6c891d7bfc92","blobs/sha256/8acd8190b760d16cc00ef8c190dc6a0f29d52eca0e1875ae011ca2c4421e1f1b","blobs/sha256/b878d11d72566cfc4d9f0c062acca16a7d2e03118e4b02fb70e252a6b60c05d8","blobs/sha256/45a23477acf82e026a14afd6438fc6582feafd74de87187dc246513bd2cf32fa","blobs/sha256/fa321c822076f2819f538dfe916ccc0dbf6ebf757f0cb7b49a21b9147a5763a5","blobs/sha256/5848f404674809715e4eef48912b5ded3f00ca6bd69582d6808859cbd6da1bc8","blobs/sha256/f75bb4510cf8bf2f78d6dd2fd74dff9c4c99c23fd1c4753cda908446f04dcef0","blobs/sha256/6b6624019b527abcd3fff6e3742cb081a9df0c3e51f2e321466c0581838e26a9","blobs/sha256/c628da7f9340f1cf3027c36c860e81c728470b26fe505c2efd34a22e54f69bac","blobs/sha256/2ca865ce447d4cce43a07ac8c5760f3ab3674574400f17907b46893d26dcc6d9","blobs/sha256/4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1","blobs/sha256/b1e1772b358bab0a429fb0fccbe37d2514e8afdb949d730f7b92005eb6712a41","blobs/sha256/b9f07a7c247d8665afaf3354379c38806296377d38352191e77cb861c7841bc1","blobs/sha256/0060469fccd790368b51ff06d566eba1e833e3ee2cf166d65a369d8709e8a29c"]}]
+[{"Config":"blobs/sha256/b6cf338c8b67f26a22cf5cf521de5384f082fc6f8b6277fc3a825af6f5a35180","RepoTags":["tt-model/ornith-1.5-35b-a3b:9f0546abcdb8"],"Layers":["blobs/sha256/0626d3cc316c9fea3d4891e2ec19b8f371fe22ffb915622f6072a2ee2ca9017f","blobs/sha256/c080bfa67ae4d74a8da698e7749cd6d68552e3080cb7118f91c2033161097d43","blobs/sha256/2ae30a9fd16710d03b00efdfc79603fd5281ea25e0b5d104dbd43f690481bca5","blobs/sha256/4c5662049740585a4856802bfe87058375a729d038f90304d86be187c0f22f32","blobs/sha256/940f6ae42c997504c9b9e88c251ac61e84464c9df9f7891766696355cbfe0da9","blobs/sha256/97c8a3d25d09f696e6e83b7473d7e14b2ea600392714355b93223d98e52696d8","blobs/sha256/0ab7ba472d2c55edefa1cf5f7f3e6531b8053eeff029c88ff48dc2fce8052c9a","blobs/sha256/8abd8ce75f0ca026a80af1fbc547e289505ad0d37b453b2816951524afda8d3d","blobs/sha256/60a24d7bdb99b37e3524309c9a45b649f0869a088137c6396e33bcf88a837f3d","blobs/sha256/7ff93218a1f461f0ff7de29caaae299027fab21dccd93fd62d14156e127755d6","blobs/sha256/f99c891cf7810365d8f2f0f08d964da3b6a207fc911cdfb161c1b61b1c7f7c75","blobs/sha256/4bd0ed151188eb03b1c73f6f1a45c5e91ca14ea4ebb07c0e3a3570ca3cb7cb62","blobs/sha256/2fb2fadb0dd1bdb50f6dc1675cc4de2f6a15a625f5db9771c9cd3584ba5a524c","blobs/sha256/a69718c502a0ae46d1f411856c7b1386309f93266af2300428077847bd558ee3","blobs/sha256/e1d70dd4872043b59108086de8c84903998e542c9cfba4679ff1f22f4089ee96","blobs/sha256/ebde235144ede87b0d3ac60b1ddf31e48d09fdebcfa2ec2f0063b94a62de5b83","blobs/sha256/bcdb3d1b52e2726654acc1fe4fb0358e26fecd2e4ddfb3425696e98bc0cb1a51","blobs/sha256/b34bde9dfb9910f28026e11d228d3b1239a3d5a30b65137811c6e17a787dad5a","blobs/sha256/58e4beb1037631cccf62c362c01c579f402d94671765a32741c4a917197f8232","blobs/sha256/4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1","blobs/sha256/7f1fcf02808f34fbdae045185153d7c65ee5144f9f13b55118b6bb4134209519","blobs/sha256/0a2d1d3ac4dd4a9b97b4b36d3b9c33fd7000be09be485d76a275cde6b4746e59","blobs/sha256/ef6a9fc8dfbb5f0d324a39b73fd5f7ae958a1e69256c47bac74a34cb694feaa8"]}]

requirements.lock

@@ -52,7 +52,7 @@ googleapis-common-protos==1.76.0
 graphviz==0.21
 grpcio==1.85.0
 h11==0.16.0
-hf-xet==1.7.0
+hf-xet==1.7.1
 httpcore==1.0.9
 httpcore2==2.16.0
 httptools==0.9.0
@@ -83,7 +83,7 @@ mcp==2.3.0
 mcp-types==2.3.0
 mdit-py-plugins==0.6.1
 mdurl==0.1.2
-mistral_common==1.12.0
+mistral_common==1.13.0
 ml_dtypes==0.5.4
 model-hosting-container-standards==0.1.16
 mpmath==1.3.0
@@ -128,7 +128,7 @@ packaging==26.3
 pandas==3.0.6
 partial-json-parser==0.2.1.1.post7
 pillow==13.0.0
-platformdirs==4.13.6
+platformdirs==4.13.7
 pre_commit==4.7.0
 prometheus-fastapi-instrumentator==8.1.0
 prometheus_client==0.26.0
@@ -141,7 +141,7 @@ pycountry==26.2.16
 pycparser==3.12
 pydantic==2.14.1
 pydantic-extra-types==2.11.1
-pydantic-settings==2.16.1
+pydantic-settings==2.16.2
 pydantic_core==2.50.1
 pyelftools==0.33
 Pygments==2.21.0
@@ -150,7 +150,7 @@ pyluwen==0.10.0
 pynvvideocodec==2.0.4
 pyparsing==3.3.3
 python-dateutil==3.0.0
-python-discovery==1.6.5
+python-discovery==1.6.7
 python-dotenv==1.2.4
 python-json-logger==4.2.0
 python-multipart==0.0.32
@@ -201,11 +201,11 @@ tt-umd==0.9.12
 typer==0.27.3
 typing-inspection==0.4.4
 typing_extensions==4.16.0
-tzdata==2026.5
+tzdata==2026.6
 urllib3==2.8.0
 uvicorn==0.54.0
 uvloop==0.23.0
-virtualenv==21.15.0
+virtualenv==21.15.1
 watchfiles==1.3.0
 websockets==17.3
 wheel==0.48.0

tt_kernel_manifest.json

@@ -1,12 +1,12 @@
 {
   "schema_version": "5.1",
   "name": "ornith-1.5-35b-a3b",
-  "tt_metal_version": "0.65.2.dev10115",
+  "tt_metal_version": "0.65.2.dev10123",
   "arch": "blackhole",
   "device_count": 2,
   "producer": {
     "tt_kernel_version": "0.1.0",
-    "created_at": "2026-10-27T19:11:37.867315+00:00",
+    "created_at": "2026-10-28T16:24:18.987830+00:00",
     "hostname": "qb2-120-p06t08"
   },
   "weights": {
@@ -28,8 +28,8 @@
     "image": {
       "registry": "hf",
       "repository": "ornith-1.5-35b-a3b",
-      "tag": "tt-model/ornith-1.5-35b-a3b:fe0482ed6aa2",
-      "digest": "sha256:fe0482ed6aa240d749ca05250027a25cddf1e7172f2b6ea1b0bf03da94a9323a"
+      "tag": "tt-model/ornith-1.5-35b-a3b:9f0546abcdb8",
+      "digest": "sha256:9f0546abcdb8140fc1aac8e20141b19039f4e4a73f1805cbea62d51936ce4649"
     },
     "kind": "vllm-plugin",
     "runtime": {
@@ -38,7 +38,7 @@
       },
       "plugin": {
         "path": "/home/raahem/vllm-tt-plugin-pub",
-        "sha": "556404a17669d52e8bfb32c26107f535541bfbd2",
+        "sha": "e37ccd87d88f87628ffb63e24bcd3d2f16350936",
         "dirty": false
       },
       "extra_models_dir": "models/autoports/ornith_ai_ornith_1_5_35b_a3b/vllm_bundle",
@@ -72,6 +72,7 @@
       },
       "args": [
         "--enable-auto-tool-choice",
+        "--enable-prefix-caching",
         "--default-chat-template-kwargs",
         "{\"enable_thinking\": true}",
         "--limit-mm-per-prompt",
@@ -188,6 +189,7 @@
         },
         "args": [
           "--enable-auto-tool-choice",
+          "--enable-prefix-caching",
           "--default-chat-template-kwargs",
           "{\"enable_thinking\": true}",
           "--limit-mm-per-prompt",
@@ -218,6 +220,7 @@
         },
         "args": [
           "--enable-auto-tool-choice",
+          "--enable-prefix-caching",
           "--default-chat-template-kwargs",
           "{\"enable_thinking\": true}",
           "--limit-mm-per-prompt",
@@ -240,40 +243,40 @@
       "import models.autoports.ornith_ai_ornith_1_5_35b_a3b.tt.decode_buckets as b; assert b.widths(32) == [1, 2, 4, 8, 16, 32]"
     ],
     "built": {
-      "image": "tt-model/ornith-1.5-35b-a3b:fe0482ed6aa2",
+      "image": "tt-model/ornith-1.5-35b-a3b:9f0546abcdb8",
       "repo": "raahemnabeel/ornith-1.5-35b-a3b",
       "tt_model_version": "0.1.0",
-      "created_at": "2026-10-27T19:07:07+00:00",
+      "created_at": "2026-10-28T16:16:04+00:00",
       "tt_metal": {
-        "sha": "fe3ddb815062a45f5153885cc5e0e69484c4687c",
-        "describe": "v0.79.0-dev20260909-312-gfe3ddb8150",
+        "sha": "d9ed49c49330298e207d9532d173c9d6a09ff81a",
+        "describe": "v0.79.0-dev20260909-320-gd9ed49c493",
         "dirty": false,
-        "scm_version": "0.65.2.dev10115",
+        "scm_version": "0.65.2.dev10123",
         "mode": "local",
         "remote": "https://github.com/rnabeelTT/tt-metal.git",
         "branch": "ornith_fusion",
         "pushed": true
       },
-      "code_sha256": "159a72b50ffa0e558a99fd214bc1fd4b93ee8ce10e73f7e75f26ac94251133b9",
+      "code_sha256": "aead6d7885da18440e9e5afc4c6d4426389c310cf33a34bd4fb3d0da5cc67bce",
       "plugin": {
-        "sha": "556404a17669d52e8bfb32c26107f535541bfbd2",
+        "sha": "e37ccd87d88f87628ffb63e24bcd3d2f16350936",
         "path": "/home/raahem/vllm-tt-plugin-pub",
         "dirty": false
       },
-      "image_digest": "sha256:fe0482ed6aa240d749ca05250027a25cddf1e7172f2b6ea1b0bf03da94a9323a"
+      "image_digest": "sha256:9f0546abcdb8140fc1aac8e20141b19039f4e4a73f1805cbea62d51936ce4649"
     },
     "card": {
       "description": "Ornith-1.5-35B-A3B (35B total / 3B active MoE: 256 experts, top-8 + shared expert, 40 layers of Gated DeltaNet + gated attention, vision tower, MTP head) on Blackhole through vLLM and the Tenstorrent plugin, with a fused decode step: the v1 bundle (raahemnabeel/ornith-1.5-35b-a3b-v1) plus fused MoE, Gated DeltaNet, attention, layer-boundary and sampler decode ops for 1-32 concurrent users. Same weights, profiles and serving recipe; on four dies single-user decode 45 -> 220 tok/s and 32-user output 346 -> 591 tok/s, prefill unchanged or faster.\n",
       "attribution": null,
       "prerequisites": null,
-      "quickstart": "### What changed since v1\nThis is the v1 bundle ([raahemnabeel/ornith-1.5-35b-a3b-v1](https://huggingface.co/raahemnabeel/ornith-1.5-35b-a3b-v1)) with its decode step\nrebuilt around fused kernels. Same checkpoint, same vLLM plugin build, same profiles and serving recipe; prefill is the\nv1's (TTFT equal or lower). The changes, in the order they were made (each measured, kept only if it won):\n\n1. **Router.** One 1D matmul (was DRAM-sharded on 8 cores); top-8 + softmax + shared-gate sigmoid on the RISC cores in\n   one op instead of a 7-op chain.\n2. **Fused MoE decode.** Routed + shared experts in three ops (route, gate|up + SwiGLU, down) that stream each selected\n   expert's weights in place, for every decode width from 1 to 32 rows (several users' experts as one union, each weight\n   column read once for all rows). On four dies the gate and up halves of an expert column are split across a pair of\n   cores, with readers placed off the congested NOC rows.\n3. **Replicated-residual layer boundary.** The decode residual stays replicated on every die: each mixer / MoE output is\n   one all-reduce + add + local RMSNorm.\n4. **Gated DeltaNet decode mixer** as two programs (causal conv; gates + norms + recurrent state update + gated norm),\n   with each head's value dimension split across up to four cores.\n5. **Attention decode ops** around the QKV matmul, cache update and SDPA decode (q / k norm + RoPE + gate in one prep op,\n   output gating in one post op).\n6. **Fabric layer boundary** (four dies, 1-4 users): the all-reduce, residual add and RMSNorm run as one op over the\n   fabric, and the next layer's input projection (or the MoE router) runs inside the same program, streaming its weights\n   while the boundary computes.\n7. **Sampler top-k over the live rows only** (1-8 users), instead of all 32 slot rows.\n\nMTP speculative decoding is off on the multi-die profiles: its verify pass runs the unfused path, and the fused plain\ndecode step is faster on real text (four dies: 218 vs 86 tok/s).\n\nNumerics: the routed and shared expert weights are stored rounded to 5 significant bits (exactly what the LoFi expert\nmatmul reads), activations / routing weights rounded to the 7 bits the matmul's other operand reads; the fused boundary\nand DeltaNet ops sum in a different order than the op chain they replace. Output quality is checked below.\n\nOn one die (`p150`, `p100`) the full expert width does not fit the fused kernels' bank layout, so those profiles keep the\nv1 MoE and gain only the fused router top-k (1-row steps), with MTP on as in v1.\n\n### Use with your client\nPoint an OpenAI-compatible client at `http://127.0.0.1:20000` (or the port `tt-model serve` printed) with model\n`ornith-ai/Ornith-1.5-35B-A3B`. Thinking is ON by default (reasoning in the message's `reasoning` field, answer in\n`content`); turn it off per request with `\"chat_template_kwargs\": {\"enable_thinking\": false}`. Give thinking requests a\nlarge `max_tokens`. Tool calls use the `qwen3_xml` parser. Images (up to 4 per request, resized to at most 4 MP) as\n`image_url` parts on the multi-die profiles.\n\n### Profiles\n`tt-model serve raahemnabeel/ornith-1.5-35b-a3b --profile <name>`:\n- `p300` (default) / `p150x2` - two dies, bfp8 routed experts, text + images (no fabric boundary: its receive buffers do\n  not fit next to the two-die prefill).\n- `p300x2` / `p150x4` - four dies, bfp8 routed experts, text + images: the fastest profile.\n- `p150` / `p100` - one die, bfp4 routed experts, text only (v1 MoE, MTP on).\n\nEvery profile: 32 concurrent sequences, 262144-token context.\n",
+      "quickstart": "### What changed since v1\nThis is the v1 bundle ([raahemnabeel/ornith-1.5-35b-a3b-v1](https://huggingface.co/raahemnabeel/ornith-1.5-35b-a3b-v1)) with its decode step\nrebuilt around fused kernels. Same checkpoint, same vLLM plugin build, same profiles and serving recipe; prefill is the\nv1's (TTFT equal or lower). The changes, in the order they were made (each measured, kept only if it won):\n\n1. **Router.** One 1D matmul (was DRAM-sharded on 8 cores); top-8 + softmax + shared-gate sigmoid on the RISC cores in\n   one op instead of a 7-op chain.\n2. **Fused MoE decode.** Routed + shared experts in three ops (route, gate|up + SwiGLU, down) that stream each selected\n   expert's weights in place, for every decode width from 1 to 32 rows (several users' experts as one union, each weight\n   column read once for all rows). On four dies the gate and up halves of an expert column are split across a pair of\n   cores, with readers placed off the congested NOC rows.\n3. **Replicated-residual layer boundary.** The decode residual stays replicated on every die: each mixer / MoE output is\n   one all-reduce + add + local RMSNorm.\n4. **Gated DeltaNet decode mixer** as two programs (causal conv; gates + norms + recurrent state update + gated norm),\n   with each head's value dimension split across up to four cores.\n5. **Attention decode ops** around the QKV matmul, cache update and SDPA decode (q / k norm + RoPE + gate in one prep op,\n   output gating in one post op).\n6. **Fabric layer boundary** (four dies, 1-4 users): the all-reduce, residual add and RMSNorm run as one op over the\n   fabric, and the next layer's input projection (or the MoE router) runs inside the same program, streaming its weights\n   while the boundary computes.\n7. **Sampler top-k over the live rows only** (1-8 users), instead of all 32 slot rows.\n\nMTP speculative decoding is off on the multi-die profiles: its verify pass runs the unfused path, and the fused plain\ndecode step is faster on real text (four dies: 218 vs 86 tok/s).\n\nNumerics: the routed and shared expert weights are stored rounded to 5 significant bits (exactly what the LoFi expert\nmatmul reads), activations / routing weights rounded to the 7 bits the matmul's other operand reads; the fused boundary\nand DeltaNet ops sum in a different order than the op chain they replace. Output quality is checked below.\n\nOn one die (`p150`, `p100`) the full expert width does not fit the fused kernels' bank layout, so those profiles keep the\nv1 MoE and gain only the fused router top-k (1-row steps), with MTP on as in v1.\n\n### Use with your client\nPoint an OpenAI-compatible client at `http://127.0.0.1:20000` (or the port `tt-model serve` printed) with model\n`ornith-ai/Ornith-1.5-35B-A3B`. Thinking is ON by default (reasoning in the message's `reasoning` field, answer in\n`content`); turn it off per request with `\"chat_template_kwargs\": {\"enable_thinking\": false}`. Give thinking requests a\nlarge `max_tokens`. Tool calls use the `qwen3_xml` parser. Images (up to 4 per request, resized to at most 4 MP) as\n`image_url` parts on the multi-die profiles.\n\n### Prefix caching\nOn by default (multi-die profiles). When a prompt starts with the same 2,048 or more tokens as a recent one (a document sent\nagain, the next turn of a chat, a long shared system prompt), the server resumes from the last 2,048-token boundary\nthe two share instead of prefilling from the start: vLLM reuses the attention KV blocks, and the server restores the\nGated DeltaNet state it saved at that boundary (32 snapshots on four dies, 16 on two; each snapshot 22.5 MiB per die on four dies, 0.7 GiB per die in all, taken from the KV pool, least recently used evicted). A prompt seen for the first time\nruns exactly the prefill it ran without caching. Turn it off with `tt-model serve ... -- --no-enable-prefix-caching`.\n\n### Profiles\n`tt-model serve raahemnabeel/ornith-1.5-35b-a3b --profile <name>`:\n- `p300` (default) / `p150x2` - two dies, bfp8 routed experts, text + images (no fabric boundary: its receive buffers do\n  not fit next to the two-die prefill).\n- `p300x2` / `p150x4` - four dies, bfp8 routed experts, text + images: the fastest profile.\n- `p150` / `p100` - one die, bfp4 routed experts, text only (v1 MoE, MTP on).\n\nEvery profile: 32 concurrent sequences, 262144-token context.\n",
       "architecture": null,
       "status": null,
       "intended_use": null,
       "out_of_scope_use": null,
       "usage": null,
-      "performance": "### Accuracy\nBenchmark evals through the served API (p300 profile, thinking off, one sample per question, seed 42, temperature 0.6 /\ntop_p 0.95 / top_k 20, lm-evaluation-harness), the same settings and box as the v1 bundle's run:\n\n| Eval | v1 | current |\n| --- | ---: | ---: |\n| GPQA-Diamond (40 questions) | 0.775 | 0.675 |\n| IFEval prompt-level strict (100) | 0.76 | 0.82 |\n| LiveCodeBench v6 (50) | 0.36 | 0.40 |\n\nAll within the error bars of these subset sizes (GPQA +-0.07, IFEval +-0.04, LiveCodeBench +-0.07): no measurable\nquality change. The whole GPQA-Diamond set (198 questions, same settings) scores 0.707 +-0.032 on this bundle; the\nsame build scored 0.75 on the first 40 questions in that run (0.675 in the subset run above), the spread one sample\nper question gives at n = 40. Layer-level PCC against Hugging Face (real weights): decode >= 0.9999 at 1 user,\nper-user minimum >= 0.9999 at 32 users.\n\nTeacher forcing against the Hugging Face bf16 model (v1 bundle; 100 greedy tokens, top-1 / top-5 / top-100):\n\n| profile | code chat prompt | 2048-token book prompt |\n| --- | --- | --- |\n| 4 dies (bfp8 experts) | 0.97 / 1.00 / 1.00 | 0.89 / 0.98 / 1.00 |\n| 2 dies (bfp8 experts) | 0.96 / 1.00 / 1.00 | 0.91 / 0.97 / 1.00 |\n| 1 die (bfp4 experts) | 0.96 / 1.00 / 1.00 | 0.84 / 0.94 / 0.99 |\n\n### Performance\nMeasured 2026-10-25 on one QuietBox 2 (4 Blackhole p300c dies; p300 and p150 rows on a subset of it), both bundles\nserved exactly as shipped (`tt-model serve <bundle> --profile <p>`), one server at a time: random-token prompts of\nexactly the input length, output pinned with ignore_eos, streamed; users are concurrent requests in a closed loop;\nTTFT = time to first streamed token; TPOT = (E2EL - TTFT) / (output - 1); one unrecorded warm-up per row. v1\nruns MTP speculative decoding while one request is live (its single-user rows include it); this bundle runs plain\ndecode on the multi-die profiles. The real-text row (12 chat prompts, 384 output tokens, temperature 0) shows\nsingle-user speed on predictable text, where MTP helps v1 most. Each cell is `v1 -> current`.\n\n| Profile | ISL | OSL | Concurrency | N | TTFT (ms) | Prefill tok/s/u | Decode tok/s/u | E2EL (s) |\n| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| p300x2 | 128 | 128 | 1 | 4 | 118 | 1,083 | 220.0 | 0.70 |\n| p300x2 | 1024 | 256 | 32 | 128 | 7,265 | 141 | 42.1 | 13.40 |\n| p300 | 128 | 128 | 1 | 4 | 133 | 962 | 126.3 | 1.14 |\n| p300 | 1024 | 256 | 32 | 128 | 8,628 | 119 | 25.4 | 18.69 |\n| p150 | 1024 | 256 | 32 | 128 | 11,879 | 86 | 8.4 | 42.06 |\n\n#### p300x2\n\nFour-die rows measured on the previous revision, which differs only in the sampler top-k's index width (16 -> 32 bits, for two dies). This revision, same setup: 1 user 4.57 ms TPOT, real text 217.4 tok/s, 8 users 9.71 ms, 32 users 1024/256 24.2 ms.\n\nLatency, batch 1 (single user): v1 -> current\n\n| Input tokens | Output tokens | TTFT (ms) | Prefill tok/s | TPOT (ms) | Decode tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 128 | 128 | 179 -> 118 | 717 -> 1,083 | 22.4 -> 4.5 | 44.5 -> **220.0** (+394 %) | 3.04 -> 0.70 |\n| 1,024 | 256 | 287 -> 223 | 3,572 -> 4,584 | 20.3 -> 4.7 | 49.2 -> **214.5** (+336 %) | 5.48 -> 1.41 |\n| 4,096 | 256 | 728 -> 646 | 5,625 -> 6,340 | 22.0 -> 4.8 | 45.4 -> **207.9** (+358 %) | 6.34 -> 1.87 |\n| 16,384 | 256 | 2,760 -> 2,636 | 5,936 -> 6,216 | 24.4 -> 5.0 | 41.0 -> **200.0** (+388 %) | 8.98 -> 3.91 |\n| 32,768 | 256 | 5,716 -> 5,495 | 5,733 -> 5,963 | 25.4 -> 5.3 | 39.3 -> **190.1** (+384 %) | 12.20 -> 6.84 |\n| 65,536 | 256 | 12,607 -> 12,177 | 5,198 -> 5,382 | 23.3 -> 5.8 | 43.0 -> **173.4** (+304 %) | 18.54 -> 13.65 |\n| 130,816 | 256 | 30,102 -> 28,766 | 4,346 -> 4,548 | 26.1 -> 6.6 | 38.4 -> **150.5** (+292 %) | 36.75 -> 30.46 |\n\nReal text, single user (12 chat prompts x 384 output tokens, temperature 0; median per prompt): 58.5 -> **217.9 tok/s** (+272 %).\n\nConcurrency, 128 input / 128 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 340 -> 332 | 24.8 -> 6.7 | 40.3 -> 149.6 | 73.3 -> **215.8** (+194 %) | 3.49 -> 1.19 |\n| 4 | 16 | 680 -> 660 | 28.4 -> 7.2 | 35.2 -> 139.3 | 118.9 -> **323.7** (+172 %) | 4.30 -> 1.59 |\n| 8 | 32 | 1,135 -> 1,125 | 37.0 -> 9.4 | 27.0 -> 106.0 | 175.0 -> **440.3** (+152 %) | 5.85 -> 2.32 |\n| 16 | 64 | 2,112 -> 2,052 | 55.1 -> 13.7 | 18.2 -> 73.2 | 225.0 -> **541.5** (+141 %) | 9.11 -> 3.77 |\n| 32 | 128 | 3,975 -> 3,927 | 62.0 -> 24.0 | 16.1 -> 41.7 | 345.8 -> **591.1** (+71 %) | 11.84 -> 6.96 |\n\nConcurrency, 1024 input / 256 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 561 -> 547 | 24.3 -> 6.4 | 41.2 -> 157.3 | 75.8 -> **235.3** (+210 %) | 6.75 -> 2.17 |\n| 4 | 16 | 1,119 -> 1,086 | 28.4 -> 7.2 | 35.2 -> 138.5 | 122.5 -> **349.9** (+186 %) | 8.36 -> 2.91 |\n| 8 | 32 | 1,996 -> 1,986 | 37.0 -> 9.6 | 27.0 -> 103.8 | 178.9 -> **464.0** (+159 %) | 11.44 -> 4.42 |\n| 16 | 64 | 3,814 -> 3,718 | 55.3 -> 13.5 | 18.1 -> 74.0 | 228.6 -> **570.1** (+149 %) | 17.93 -> 7.18 |\n| 32 | 128 | 7,347 -> 7,265 | 61.8 -> 23.8 | 16.2 -> 42.1 | 353.8 -> **614.0** (+74 %) | 23.12 -> 13.40 |\n\n#### p300\n\nLatency, batch 1 (single user): v1 -> current\n\n| Input tokens | Output tokens | TTFT (ms) | Prefill tok/s | TPOT (ms) | Decode tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 128 | 128 | 202 -> 133 | 634 -> 962 | 23.3 -> 7.9 | 42.9 -> **126.3** (+194 %) | 3.16 -> 1.14 |\n| 1,024 | 256 | 346 -> 270 | 2,960 -> 3,787 | 25.0 -> 8.0 | 40.0 -> **124.4** (+211 %) | 6.72 -> 2.32 |\n| 4,096 | 256 | 891 -> 807 | 4,595 -> 5,077 | 23.3 -> 8.2 | 42.9 -> **122.1** (+185 %) | 6.84 -> 2.90 |\n| 16,384 | 256 | 3,413 -> 3,267 | 4,801 -> 5,015 | 32.9 -> 8.4 | 30.4 -> **119.2** (+292 %) | 11.79 -> 5.41 |\n| 32,768 | 256 | 7,059 -> 6,801 | 4,642 -> 4,818 | 22.6 -> 8.6 | 44.3 -> **115.8** (+162 %) | 12.82 -> 9.00 |\n| 65,536 | 256 | 15,296 -> 14,765 | 4,284 -> 4,439 | 21.5 -> 9.1 | 46.5 -> **109.6** (+136 %) | 20.78 -> 17.09 |\n| 130,816 | 256 | 35,579 -> 34,008 | 3,677 -> 3,847 | 37.4 -> 10.1 | 26.8 -> **99.3** (+271 %) | 45.11 -> 36.58 |\n\nReal text, single user (12 chat prompts x 384 output tokens, temperature 0; median per prompt): 52.6 -> **125.6 tok/s** (+139 %).\n\nConcurrency, 128 input / 128 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 355 -> 352 | 28.5 -> 10.4 | 35.1 -> 96.2 | 64.4 -> **153.2** (+138 %) | 3.97 -> 1.67 |\n| 4 | 16 | 704 -> 689 | 34.5 -> 12.6 | 28.9 -> 79.3 | 100.5 -> **225.5** (+124 %) | 5.09 -> 2.28 |\n| 8 | 32 | 1,225 -> 1,239 | 47.8 -> 17.4 | 20.9 -> 57.6 | 140.0 -> **299.7** (+114 %) | 7.30 -> 3.43 |\n| 16 | 64 | 2,264 -> 2,288 | 75.3 -> 24.7 | 13.3 -> 40.5 | 172.8 -> **378.2** (+119 %) | 11.85 -> 5.40 |\n| 32 | 128 | 4,315 -> 4,226 | 84.8 -> 40.6 | 11.8 -> 24.6 | 270.8 -> **437.8** (+62 %) | 15.07 -> 9.36 |\n\nConcurrency, 1024 input / 256 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 642 -> 623 | 28.0 -> 10.1 | 35.7 -> 99.0 | 65.7 -> **159.5** (+143 %) | 7.78 -> 3.21 |\n| 4 | 16 | 1,286 -> 1,237 | 34.5 -> 12.7 | 29.0 -> 78.9 | 101.7 -> **232.6** (+129 %) | 10.07 -> 4.42 |\n| 8 | 32 | 2,358 -> 2,288 | 47.8 -> 17.6 | 20.9 -> 56.9 | 140.6 -> **303.5** (+116 %) | 14.56 -> 6.75 |\n| 16 | 64 | 4,498 -> 4,408 | 75.6 -> 24.2 | 13.2 -> 41.2 | 172.2 -> **385.6** (+124 %) | 23.77 -> 10.61 |\n| 32 | 128 | 8,802 -> 8,628 | 84.0 -> 39.3 | 11.9 -> 25.4 | 271.1 -> **436.4** (+61 %) | 30.16 -> 18.69 |\n\n#### p150\n\nOne die keeps the v1 decode path (the fused MoE kernel needs at most two up|gate columns per DRAM bank, i.e. two or more dies, and the replicated boundary needs a mesh): p150 decode is the v1 MoE plus the fused router top-k on 1-row steps, MTP on, and the tables below show parity. The single-user random-token latency table is left out: with adaptive MTP on random tokens it does not repeat on one die (15.9, 52.4 and 27.0 ms TPOT at 128/128 in three runs of v1); the v1 card's p150 latency rows still describe this bundle.\n\nReal text, single user (12 chat prompts x 384 output tokens, temperature 0; median per prompt): 42.6 -> **44.1 tok/s** (+4 %).\n\nConcurrency, 128 input / 128 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 412 -> 414 | 36.8 -> 36.8 | 27.2 -> 27.2 | 50.4 -> **50.4** (+0 %) | 5.08 -> 5.08 |\n| 4 | 16 | 814 -> 816 | 45.6 -> 45.6 | 21.9 -> 21.9 | 77.4 -> **77.4** (-0 %) | 6.61 -> 6.61 |\n| 8 | 32 | 1,455 -> 1,445 | 63.9 -> 63.9 | 15.6 -> 15.6 | 106.9 -> **107.0** (+0 %) | 9.58 -> 9.57 |\n| 16 | 64 | 2,715 -> 2,711 | 100.4 -> 100.4 | 10.0 -> 10.0 | 132.4 -> **132.4** (+0 %) | 15.47 -> 15.47 |\n| 32 | 128 | 5,244 -> 5,219 | 120.2 -> 120.2 | 8.3 -> 8.3 | 200.0 -> **200.0** (+0 %) | 20.51 -> 20.47 |\n\nConcurrency, 1024 input / 256 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 829 -> 829 | 36.3 -> 36.3 | 27.5 -> 27.5 | 50.7 -> **50.7** (+0 %) | 10.09 -> 10.09 |\n| 4 | 16 | 1,658 -> 1,642 | 45.6 -> 45.6 | 21.9 -> 21.9 | 77.1 -> **77.2** (+0 %) | 13.29 -> 13.27 |\n| 8 | 32 | 3,108 -> 3,108 | 64.1 -> 64.1 | 15.6 -> 15.6 | 105.3 -> **105.3** (+0 %) | 19.46 -> 19.46 |\n| 16 | 64 | 6,033 -> 6,032 | 100.9 -> 100.9 | 9.9 -> 9.9 | 128.9 -> **128.9** (-0 %) | 31.77 -> 31.77 |\n| 32 | 128 | 11,885 -> 11,879 | 119.4 -> 118.3 | 8.4 -> 8.4 | 193.0 -> **194.7** (+1 %) | 42.33 -> 42.06 |\n",
-      "limitations": "- MTP speculative decoding is off on the multi-die profiles: the MTP verify pass still runs the unfused rows path, and\n  with it on, single-user real-text decode is slower than the fused plain step (four dies: 86 vs 218 tok/s). A fused\n  multi-token verify is the way back to MTP.\n- Two dies (`p300`, `p150x2`) run without the fabric layer boundary: its persistent receive buffers do not fit next to\n  the two-die prefill's circular buffers.\n- One die (`p150`, `p100`) keeps the v1 MoE: the fused kernels read 1 or 2 up|gate columns per DRAM bank and\n  expert, one die holds 4 (and p100 has 7 banks).\n- The p150x4 and p150x2 profiles run the same meshes as p300x2 / p300 but were not run on p150 boards.\n- The device sampler supports `top_k` up to 32.\n- Benchmark evals ran on subsets (40 / 100 / 50 questions, plus the full 198-question GPQA-Diamond for this build) on the\n  p300 profile only.\n",
+      "performance": "### Accuracy\nBenchmark evals through the served API (p300 profile, thinking off, one sample per question, seed 42, temperature 0.6 /\ntop_p 0.95 / top_k 20, lm-evaluation-harness), the same settings and box as the v1 bundle's run:\n\n| Eval | v1 | current |\n| --- | ---: | ---: |\n| GPQA-Diamond (40 questions) | 0.775 | 0.675 |\n| IFEval prompt-level strict (100) | 0.76 | 0.82 |\n| LiveCodeBench v6 (50) | 0.36 | 0.40 |\n\nAll within the error bars of these subset sizes (GPQA +-0.07, IFEval +-0.04, LiveCodeBench +-0.07): no measurable\nquality change. The whole GPQA-Diamond set (198 questions, same settings) scores 0.707 +-0.032 on this bundle; the\nsame build scored 0.75 on the first 40 questions in that run (0.675 in the subset run above), the spread one sample\nper question gives at n = 40. Layer-level PCC against Hugging Face (real weights): decode >= 0.9999 at 1 user,\nper-user minimum >= 0.9999 at 32 users.\n\nTeacher forcing against the Hugging Face bf16 model (v1 bundle; 100 greedy tokens, top-1 / top-5 / top-100):\n\n| profile | code chat prompt | 2048-token book prompt |\n| --- | --- | --- |\n| 4 dies (bfp8 experts) | 0.97 / 1.00 / 1.00 | 0.89 / 0.98 / 1.00 |\n| 2 dies (bfp8 experts) | 0.96 / 1.00 / 1.00 | 0.91 / 0.97 / 1.00 |\n| 1 die (bfp4 experts) | 0.96 / 1.00 / 1.00 | 0.84 / 0.94 / 0.99 |\n\n### Performance\nMeasured 2026-10-25 on one QuietBox 2 (4 Blackhole p300c dies; p300 and p150 rows on a subset of it), both bundles\nserved exactly as shipped (`tt-model serve <bundle> --profile <p>`), one server at a time: random-token prompts of\nexactly the input length, output pinned with ignore_eos, streamed; users are concurrent requests in a closed loop;\nTTFT = time to first streamed token; TPOT = (E2EL - TTFT) / (output - 1); one unrecorded warm-up per row. v1\nruns MTP speculative decoding while one request is live (its single-user rows include it); this bundle runs plain\ndecode on the multi-die profiles. The real-text row (12 chat prompts, 384 output tokens, temperature 0) shows\nsingle-user speed on predictable text, where MTP helps v1 most. Each cell is `v1 -> current`.\n\n| Profile | ISL | OSL | Concurrency | N | TTFT (ms) | Prefill tok/s/u | Decode tok/s/u | E2EL (s) |\n| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| p300x2 | 128 | 128 | 1 | 4 | 118 | 1,083 | 220.0 | 0.70 |\n| p300x2 | 1024 | 256 | 32 | 128 | 7,265 | 141 | 42.1 | 13.40 |\n| p300 | 128 | 128 | 1 | 4 | 133 | 962 | 126.3 | 1.14 |\n| p300 | 1024 | 256 | 32 | 128 | 8,628 | 119 | 25.4 | 18.69 |\n| p150 | 1024 | 256 | 32 | 128 | 11,879 | 86 | 8.4 | 42.06 |\n\n#### p300x2\n\nFour-die rows measured on the previous revision, which differs only in the sampler top-k's index width (16 -> 32 bits, for two dies). This revision, same setup: 1 user 4.57 ms TPOT, real text 217.4 tok/s, 8 users 9.71 ms, 32 users 1024/256 24.2 ms.\n\nLatency, batch 1 (single user): v1 -> current\n\n| Input tokens | Output tokens | TTFT (ms) | Prefill tok/s | TPOT (ms) | Decode tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 128 | 128 | 179 -> 118 | 717 -> 1,083 | 22.4 -> 4.5 | 44.5 -> **220.0** (+394 %) | 3.04 -> 0.70 |\n| 1,024 | 256 | 287 -> 223 | 3,572 -> 4,584 | 20.3 -> 4.7 | 49.2 -> **214.5** (+336 %) | 5.48 -> 1.41 |\n| 4,096 | 256 | 728 -> 646 | 5,625 -> 6,340 | 22.0 -> 4.8 | 45.4 -> **207.9** (+358 %) | 6.34 -> 1.87 |\n| 16,384 | 256 | 2,760 -> 2,636 | 5,936 -> 6,216 | 24.4 -> 5.0 | 41.0 -> **200.0** (+388 %) | 8.98 -> 3.91 |\n| 32,768 | 256 | 5,716 -> 5,495 | 5,733 -> 5,963 | 25.4 -> 5.3 | 39.3 -> **190.1** (+384 %) | 12.20 -> 6.84 |\n| 65,536 | 256 | 12,607 -> 12,177 | 5,198 -> 5,382 | 23.3 -> 5.8 | 43.0 -> **173.4** (+304 %) | 18.54 -> 13.65 |\n| 130,816 | 256 | 30,102 -> 28,766 | 4,346 -> 4,548 | 26.1 -> 6.6 | 38.4 -> **150.5** (+292 %) | 36.75 -> 30.46 |\n\nReal text, single user (12 chat prompts x 384 output tokens, temperature 0; median per prompt): 58.5 -> **217.9 tok/s** (+272 %).\n\nConcurrency, 128 input / 128 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 340 -> 332 | 24.8 -> 6.7 | 40.3 -> 149.6 | 73.3 -> **215.8** (+194 %) | 3.49 -> 1.19 |\n| 4 | 16 | 680 -> 660 | 28.4 -> 7.2 | 35.2 -> 139.3 | 118.9 -> **323.7** (+172 %) | 4.30 -> 1.59 |\n| 8 | 32 | 1,135 -> 1,125 | 37.0 -> 9.4 | 27.0 -> 106.0 | 175.0 -> **440.3** (+152 %) | 5.85 -> 2.32 |\n| 16 | 64 | 2,112 -> 2,052 | 55.1 -> 13.7 | 18.2 -> 73.2 | 225.0 -> **541.5** (+141 %) | 9.11 -> 3.77 |\n| 32 | 128 | 3,975 -> 3,927 | 62.0 -> 24.0 | 16.1 -> 41.7 | 345.8 -> **591.1** (+71 %) | 11.84 -> 6.96 |\n\nConcurrency, 1024 input / 256 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 561 -> 547 | 24.3 -> 6.4 | 41.2 -> 157.3 | 75.8 -> **235.3** (+210 %) | 6.75 -> 2.17 |\n| 4 | 16 | 1,119 -> 1,086 | 28.4 -> 7.2 | 35.2 -> 138.5 | 122.5 -> **349.9** (+186 %) | 8.36 -> 2.91 |\n| 8 | 32 | 1,996 -> 1,986 | 37.0 -> 9.6 | 27.0 -> 103.8 | 178.9 -> **464.0** (+159 %) | 11.44 -> 4.42 |\n| 16 | 64 | 3,814 -> 3,718 | 55.3 -> 13.5 | 18.1 -> 74.0 | 228.6 -> **570.1** (+149 %) | 17.93 -> 7.18 |\n| 32 | 128 | 7,347 -> 7,265 | 61.8 -> 23.8 | 16.2 -> 42.1 | 353.8 -> **614.0** (+74 %) | 23.12 -> 13.40 |\n\n#### p300\n\nLatency, batch 1 (single user): v1 -> current\n\n| Input tokens | Output tokens | TTFT (ms) | Prefill tok/s | TPOT (ms) | Decode tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 128 | 128 | 202 -> 133 | 634 -> 962 | 23.3 -> 7.9 | 42.9 -> **126.3** (+194 %) | 3.16 -> 1.14 |\n| 1,024 | 256 | 346 -> 270 | 2,960 -> 3,787 | 25.0 -> 8.0 | 40.0 -> **124.4** (+211 %) | 6.72 -> 2.32 |\n| 4,096 | 256 | 891 -> 807 | 4,595 -> 5,077 | 23.3 -> 8.2 | 42.9 -> **122.1** (+185 %) | 6.84 -> 2.90 |\n| 16,384 | 256 | 3,413 -> 3,267 | 4,801 -> 5,015 | 32.9 -> 8.4 | 30.4 -> **119.2** (+292 %) | 11.79 -> 5.41 |\n| 32,768 | 256 | 7,059 -> 6,801 | 4,642 -> 4,818 | 22.6 -> 8.6 | 44.3 -> **115.8** (+162 %) | 12.82 -> 9.00 |\n| 65,536 | 256 | 15,296 -> 14,765 | 4,284 -> 4,439 | 21.5 -> 9.1 | 46.5 -> **109.6** (+136 %) | 20.78 -> 17.09 |\n| 130,816 | 256 | 35,579 -> 34,008 | 3,677 -> 3,847 | 37.4 -> 10.1 | 26.8 -> **99.3** (+271 %) | 45.11 -> 36.58 |\n\nReal text, single user (12 chat prompts x 384 output tokens, temperature 0; median per prompt): 52.6 -> **125.6 tok/s** (+139 %).\n\nConcurrency, 128 input / 128 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 355 -> 352 | 28.5 -> 10.4 | 35.1 -> 96.2 | 64.4 -> **153.2** (+138 %) | 3.97 -> 1.67 |\n| 4 | 16 | 704 -> 689 | 34.5 -> 12.6 | 28.9 -> 79.3 | 100.5 -> **225.5** (+124 %) | 5.09 -> 2.28 |\n| 8 | 32 | 1,225 -> 1,239 | 47.8 -> 17.4 | 20.9 -> 57.6 | 140.0 -> **299.7** (+114 %) | 7.30 -> 3.43 |\n| 16 | 64 | 2,264 -> 2,288 | 75.3 -> 24.7 | 13.3 -> 40.5 | 172.8 -> **378.2** (+119 %) | 11.85 -> 5.40 |\n| 32 | 128 | 4,315 -> 4,226 | 84.8 -> 40.6 | 11.8 -> 24.6 | 270.8 -> **437.8** (+62 %) | 15.07 -> 9.36 |\n\nConcurrency, 1024 input / 256 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 642 -> 623 | 28.0 -> 10.1 | 35.7 -> 99.0 | 65.7 -> **159.5** (+143 %) | 7.78 -> 3.21 |\n| 4 | 16 | 1,286 -> 1,237 | 34.5 -> 12.7 | 29.0 -> 78.9 | 101.7 -> **232.6** (+129 %) | 10.07 -> 4.42 |\n| 8 | 32 | 2,358 -> 2,288 | 47.8 -> 17.6 | 20.9 -> 56.9 | 140.6 -> **303.5** (+116 %) | 14.56 -> 6.75 |\n| 16 | 64 | 4,498 -> 4,408 | 75.6 -> 24.2 | 13.2 -> 41.2 | 172.2 -> **385.6** (+124 %) | 23.77 -> 10.61 |\n| 32 | 128 | 8,802 -> 8,628 | 84.0 -> 39.3 | 11.9 -> 25.4 | 271.1 -> **436.4** (+61 %) | 30.16 -> 18.69 |\n\n#### p150\n\nOne die keeps the v1 decode path (the fused MoE kernel needs at most two up|gate columns per DRAM bank, i.e. two or more dies, and the replicated boundary needs a mesh): p150 decode is the v1 MoE plus the fused router top-k on 1-row steps, MTP on, and the tables below show parity. The single-user random-token latency table is left out: with adaptive MTP on random tokens it does not repeat on one die (15.9, 52.4 and 27.0 ms TPOT at 128/128 in three runs of v1); the v1 card's p150 latency rows still describe this bundle.\n\nReal text, single user (12 chat prompts x 384 output tokens, temperature 0; median per prompt): 42.6 -> **44.1 tok/s** (+4 %).\n\nConcurrency, 128 input / 128 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 412 -> 414 | 36.8 -> 36.8 | 27.2 -> 27.2 | 50.4 -> **50.4** (+0 %) | 5.08 -> 5.08 |\n| 4 | 16 | 814 -> 816 | 45.6 -> 45.6 | 21.9 -> 21.9 | 77.4 -> **77.4** (-0 %) | 6.61 -> 6.61 |\n| 8 | 32 | 1,455 -> 1,445 | 63.9 -> 63.9 | 15.6 -> 15.6 | 106.9 -> **107.0** (+0 %) | 9.58 -> 9.57 |\n| 16 | 64 | 2,715 -> 2,711 | 100.4 -> 100.4 | 10.0 -> 10.0 | 132.4 -> **132.4** (+0 %) | 15.47 -> 15.47 |\n| 32 | 128 | 5,244 -> 5,219 | 120.2 -> 120.2 | 8.3 -> 8.3 | 200.0 -> **200.0** (+0 %) | 20.51 -> 20.47 |\n\nConcurrency, 1024 input / 256 output tokens: v1 -> current\n\n| Users | N | TTFT (ms) | TPOT (ms) | Decode tok/s/user | Total out tok/s | E2EL (s) |\n| ---: | ---: | ---: | ---: | ---: | ---: | ---: |\n| 2 | 8 | 829 -> 829 | 36.3 -> 36.3 | 27.5 -> 27.5 | 50.7 -> **50.7** (+0 %) | 10.09 -> 10.09 |\n| 4 | 16 | 1,658 -> 1,642 | 45.6 -> 45.6 | 21.9 -> 21.9 | 77.1 -> **77.2** (+0 %) | 13.29 -> 13.27 |\n| 8 | 32 | 3,108 -> 3,108 | 64.1 -> 64.1 | 15.6 -> 15.6 | 105.3 -> **105.3** (+0 %) | 19.46 -> 19.46 |\n| 16 | 64 | 6,033 -> 6,032 | 100.9 -> 100.9 | 9.9 -> 9.9 | 128.9 -> **128.9** (-0 %) | 31.77 -> 31.77 |\n| 32 | 128 | 11,885 -> 11,879 | 119.4 -> 118.3 | 8.4 -> 8.4 | 193.0 -> **194.7** (+1 %) | 42.33 -> 42.06 |\n\n### Prefix caching\nMeasured 2026-10-28 on p300x2, one user, temperature 0: the same build served with caching off and on. TTFT of a\nprompt (Python source from this repo) sent a second time, and of each turn of a chat that opens with an 8k-token code\ndocument and resends the conversation so far (as every chat client does):\n\n| Request | caching off | caching on |\n| --- | ---: | ---: |\n| 7,652-token prompt, sent again | 1.30 s | 0.37 s (3.5x) |\n| 24,224-token prompt, sent again | 4.02 s | 0.42 s (9.6x) |\n| chat turns 2-6 (7.9k-8.5k tokens) | 1.29-1.48 s | 0.21-0.51 s |\n| chat with thinking on, turns 2-4 (4.1k tokens) | 0.75-0.76 s | 0.14-0.44 s |\n\nPrompts seen for the first time are unchanged: no-hit rows at 128 / 2,048 / 8,192 input tokens (1 user) and 128/128 at\n32 users, TTFT and TPOT within 1 % (1 user 128/128: TTFT 120 / 120 ms, TPOT 4.55 / 4.58 ms). Two dies (p300): a 27,176-token prompt sent again 5.78 -> 0.37 s, chat turns 2-4 0.45-0.55 s (turn 1 1.62 s). Greedy output with caching on matched caching off on every resent prompt and on\n9 of 10 chat turns; the one that differs diverges at a near-tie token 71 characters in (the reused KV blocks were computed in a different prefill pass).\n",
+      "limitations": "- MTP speculative decoding is off on the multi-die profiles: the MTP verify pass still runs the unfused rows path, and\n  with it on, single-user real-text decode is slower than the fused plain step (four dies: 86 vs 218 tok/s). A fused\n  multi-token verify is the way back to MTP.\n- Two dies (`p300`, `p150x2`) run without the fabric layer boundary: its persistent receive buffers do not fit next to\n  the two-die prefill's circular buffers.\n- One die (`p150`, `p100`) keeps the v1 MoE: the fused kernels read 1 or 2 up|gate columns per DRAM bank and\n  expert, one die holds 4 (and p100 has 7 banks).\n- Prefix caching resumes only at 2,048-token boundaries: a prompt sharing fewer than 2,048 tokens with an earlier one\n  gets no benefit, and the shared tokens past the last boundary are prefilled again. Requests with images are not\n  cached. The one-die profiles (`p150`, `p100`) keep MTP on and serve without prefix caching.\n- The p150x4 and p150x2 profiles run the same meshes as p300x2 / p300 but were not run on p150 boards.\n- The device sampler supports `top_k` up to 32.\n- Benchmark evals ran on subsets (40 / 100 / 50 questions, plus the full 198-question GPQA-Diamond for this build) on the\n  p300 profile only.\n",
       "risks": null,
       "licensing": null,
       "related": "The v1 bundle this builds on: [raahemnabeel/ornith-1.5-35b-a3b-v1](https://huggingface.co/raahemnabeel/ornith-1.5-35b-a3b-v1). Engineering log with every measurement: `models/autoports/ornith_ai_ornith_1_5_35b_a3b/fusion/HANDOFF.md` on the `ornith_fusion` branch of github.com/rnabeelTT/tt-metal.\n",