| <svg xmlns="http://www.w3.org/2000/svg" width="1500" height="1810" viewBox="0 0 1500 1810" role="img"><title>Decision decoder family — actual causal backbone and candidate readout</title><desc>One shared architecture, two independently trained Qwen3.5 text checkpoints. Numerical candidate readout, with no generated explanation tokens.</desc> |
| <defs><marker id="arrow" markerWidth="12" markerHeight="10" refX="11" refY="5" orient="auto" markerUnits="userSpaceOnUse"><path d="M0 0 L12 5 L0 10 L3 5 Z" fill="#202C36"/></marker></defs> |
| <style>text{font-family:Arial,Helvetica,sans-serif;fill:#202C36;font-weight:400}.math{font-family:Georgia,'Times New Roman',serif;font-style:italic}.serif{font-family:Georgia,'Times New Roman',serif}</style> |
| <text x="355" y="52.0" font-size="31" text-anchor="middle" class="serif">(a) Causal decision backbone</text> |
| <path d="M112,75 L635,75" fill="none" stroke="#A36A47" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round"/> |
| <text x="355" y="110.0" font-size="27" text-anchor="middle" class="">K probabilities · no token generation</text> |
| <g data-box="110.0 155 490 62"> |
| <rect x="110.0" y="155" width="490" height="62" rx="7" fill="#DFE9DE" stroke="#202C36" stroke-width="2.2"/> |
| <text x="355" y="195.18" font-size="27" text-anchor="middle" class="">Softmax over valid candidates</text> |
| </g> |
| <path d="M355,154 L355,125" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="145.0 265 420 82"> |
| <rect x="145.0" y="265" width="420" height="82" rx="7" fill="#E5E0EA" stroke="#202C36" stroke-width="2.2"/> |
| <text x="355" y="299.385" font-size="27" text-anchor="middle" class="">Shared candidate scorer</text> |
| <text x="355" y="330.97499999999997" font-size="27" text-anchor="middle" class="">expanded in (b)</text> |
| </g> |
| <path d="M355,264 L355,218" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="120.0 405 470 90"> |
| <rect x="120.0" y="405" width="470" height="90" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/> |
| <text x="355" y="443.385" font-size="27" text-anchor="middle" class="">Gather K candidate endpoints</text> |
| <text x="355" y="474.97499999999997" font-size="27" text-anchor="middle" class="">and the final query token</text> |
| </g> |
| <path d="M355,404 L355,348" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <text x="355" y="550.0" font-size="30" text-anchor="middle" class="math">H ∈ Rᴸˣᵈ</text> |
| <path d="M355,526 L355,496" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="180.0 595 350 60"> |
| <rect x="180.0" y="595" width="350" height="60" rx="7" fill="#F4EDD7" stroke="#202C36" stroke-width="2.2"/> |
| <text x="355" y="634.18" font-size="27" text-anchor="middle" class="">Final RMSNorm</text> |
| </g> |
| <path d="M355,594 L355,563" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <rect x="100" y="725" width="555" height="645" rx="23" fill="#F7F5EF" stroke="#202C36" stroke-width="2.2"/> |
| <text x="76" y="1056.5" font-size="30" text-anchor="end" class="math">N×</text> |
| <circle cx="355" cy="785" r="18" fill="#FFFFFF" stroke="#202C36" stroke-width="2.1"/> |
| <text x="355" y="793.99" font-size="29" text-anchor="middle" class="">+</text> |
| <path d="M355,766 L355,656" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="145.0 837 420 130"> |
| <rect x="145.0" y="837" width="420" height="130" rx="7" fill="#DCE8E8" stroke="#202C36" stroke-width="2.2"/> |
| <text x="355" y="879.5899999999999" font-size="27" text-anchor="middle" class="">SwiGLU FFN</text> |
| <text x="355" y="911.18" font-size="27" text-anchor="middle" class="">u = Wᵤ x; g = SiLU(Wg x)</text> |
| <text x="355" y="942.7699999999999" font-size="27" text-anchor="middle" class="">Wdown (g ⊙ u)</text> |
| </g> |
| <path d="M355,836 L355,804" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="180.0 1008 350 58"> |
| <rect x="180.0" y="1008" width="350" height="58" rx="7" fill="#F4EDD7" stroke="#202C36" stroke-width="2.2"/> |
| <text x="355" y="1046.18" font-size="27" text-anchor="middle" class="">RMSNorm</text> |
| </g> |
| <path d="M355,1007 L355,968" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <circle cx="355" cy="1117" r="18" fill="#FFFFFF" stroke="#202C36" stroke-width="2.1"/> |
| <text x="355" y="1125.99" font-size="29" text-anchor="middle" class="">+</text> |
| <path d="M355,1098 L355,1067" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="110.0 1161 490 98"> |
| <rect x="110.0" y="1161" width="490" height="98" rx="7" fill="#F0DDC2" stroke="#202C36" stroke-width="2.2"/> |
| <text x="355" y="1190.0800000000002" font-size="24" text-anchor="middle" class="">Gated DeltaNet · 3 of 4 layers</text> |
| <text x="355" y="1218.16" font-size="24" text-anchor="middle" class="">Gated causal GQA · 1 of 4</text> |
| <text x="355" y="1246.2400000000002" font-size="24" text-anchor="middle" class="">Q/K RoPE in GQA only</text> |
| </g> |
| <path d="M355,1160 L355,1136" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="180.0 1284 350 58"> |
| <rect x="180.0" y="1284" width="350" height="58" rx="7" fill="#F4EDD7" stroke="#202C36" stroke-width="2.2"/> |
| <text x="355" y="1322.18" font-size="27" text-anchor="middle" class="">RMSNorm</text> |
| </g> |
| <path d="M355,1283 L355,1260" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <path d="M355,1356 L610,1356 L610,1117 L374,1117" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <path d="M355,1082 L610,1082 L610,785 L374,785" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <circle cx="355" cy="1356" r="3.5" fill="#202C36"/> |
| <circle cx="355" cy="1082" r="3.5" fill="#202C36"/> |
| <g data-box="180.0 1410 350 58"> |
| <rect x="180.0" y="1410" width="350" height="58" rx="7" fill="#F0DDD4" stroke="#202C36" stroke-width="2.2"/> |
| <text x="355" y="1448.18" font-size="27" text-anchor="middle" class="">Token embeddings</text> |
| </g> |
| <path d="M355,1409 L355,1343" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="110.0 1510 490 84"> |
| <rect x="110.0" y="1510" width="490" height="84" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/> |
| <text x="355" y="1545.385" font-size="27" text-anchor="middle" class="">State · question · K candidates</text> |
| <text x="355" y="1576.975" font-size="27" text-anchor="middle" class="">global query suffix</text> |
| </g> |
| <path d="M355,1509 L355,1469" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <text x="1100" y="52.0" font-size="31" text-anchor="middle" class="serif">(b) Dynamic candidate scorer</text> |
| <path d="M790,75 L1410,75" fill="none" stroke="#A36A47" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round"/> |
| <text x="1100" y="110.0" font-size="31" text-anchor="middle" class="math">pᵢ = softmaxᵢ(sᵢ / T)</text> |
| <g data-box="850.0 155 500 82"> |
| <rect x="850.0" y="155" width="500" height="82" rx="7" fill="#DFE9DE" stroke="#202C36" stroke-width="2.2"/> |
| <text x="1100" y="189.38500000000002" font-size="27" text-anchor="middle" class="">Mask padded candidates</text> |
| <text x="1100" y="220.97500000000002" font-size="27" text-anchor="middle" class="">softmax over K valid scores</text> |
| </g> |
| <path d="M1100,154 L1100,125" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <circle cx="1100" cy="315" r="18" fill="#FFFFFF" stroke="#202C36" stroke-width="2.1"/> |
| <text x="1100" y="323.99" font-size="29" text-anchor="middle" class="">+</text> |
| <path d="M1100,296 L1100,238" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <text x="1142" y="285.0" font-size="29" text-anchor="start" class="math">sᵢ</text> |
| <g data-box="742.5 490 325 90"> |
| <rect x="742.5" y="490" width="325" height="90" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/> |
| <text x="905" y="528.385" font-size="27" text-anchor="middle" class="">Scaled dot product</text> |
| <text x="905" y="559.975" font-size="27" text-anchor="middle" class="">kᵢᵀ v / √256</text> |
| </g> |
| <g data-box="1142.5 490 305 90"> |
| <rect x="1142.5" y="490" width="305" height="90" rx="7" fill="#E5E0EA" stroke="#202C36" stroke-width="2.2"/> |
| <text x="1295" y="544.18" font-size="27" text-anchor="middle" class="">Linear → scalar</text> |
| </g> |
| <path d="M905,489 L905,315 L1081,315" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <path d="M1295,489 L1295,315 L1119,315" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="1165.0 640 260 58"> |
| <rect x="1165.0" y="640" width="260" height="58" rx="7" fill="#E5DCE6" stroke="#202C36" stroke-width="2.2"/> |
| <text x="1295" y="678.18" font-size="27" text-anchor="middle" class="">GELU</text> |
| </g> |
| <path d="M1295,639 L1295,581" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="742.5 790 325 90"> |
| <rect x="742.5" y="790" width="325" height="90" rx="7" fill="#E5E0EA" stroke="#202C36" stroke-width="2.2"/> |
| <text x="905" y="828.385" font-size="27" text-anchor="middle" class="">kᵢ = Wk cᵢ</text> |
| <text x="905" y="859.975" font-size="27" text-anchor="middle" class="">v = Wq q</text> |
| </g> |
| <path d="M905,789 L905,581" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="1122.5 790 345 90"> |
| <rect x="1122.5" y="790" width="345" height="90" rx="7" fill="#E5E0EA" stroke="#202C36" stroke-width="2.2"/> |
| <text x="1295" y="828.385" font-size="27" text-anchor="middle" class="">Ac cᵢ + Aq q + b</text> |
| <text x="1295" y="859.975" font-size="27" text-anchor="middle" class="">256-dimensional</text> |
| </g> |
| <path d="M1295,789 L1295,699" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <path d="M1100,990 L905,990 L905,881" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <path d="M1100,990 L1295,990 L1295,881" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <circle cx="1100" cy="990" r="3.5" fill="#202C36"/> |
| <g data-box="847.5 1040 505 88"> |
| <rect x="847.5" y="1040" width="505" height="88" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/> |
| <text x="1100" y="1077.385" font-size="27" text-anchor="middle" class="">Normalized pair (cᵢ, q)</text> |
| <text x="1100" y="1108.975" font-size="27" text-anchor="middle" class="">broadcast q to every candidate</text> |
| </g> |
| <path d="M1100,1039 L1100,991" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="755.0 1230 300 62"> |
| <rect x="755.0" y="1230" width="300" height="62" rx="7" fill="#F4EDD7" stroke="#202C36" stroke-width="2.2"/> |
| <text x="905" y="1270.18" font-size="27" text-anchor="middle" class="">LayerNorm c</text> |
| </g> |
| <g data-box="1145.0 1230 300 62"> |
| <rect x="1145.0" y="1230" width="300" height="62" rx="7" fill="#F4EDD7" stroke="#202C36" stroke-width="2.2"/> |
| <text x="1295" y="1270.18" font-size="27" text-anchor="middle" class="">LayerNorm q</text> |
| </g> |
| <path d="M905,1229 L905,1180 L995,1180 L995,1129" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <path d="M1295,1229 L1295,1180 L1205,1180 L1205,1129" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <g data-box="742.5 1400 325 90"> |
| <rect x="742.5" y="1400" width="325" height="90" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/> |
| <text x="905" y="1438.385" font-size="27" text-anchor="middle" class="">hᵢ</text> |
| <text x="905" y="1469.975" font-size="27" text-anchor="middle" class="">candidate endpoint</text> |
| </g> |
| <g data-box="1142.5 1400 305 90"> |
| <rect x="1142.5" y="1400" width="305" height="90" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/> |
| <text x="1295" y="1438.385" font-size="27" text-anchor="middle" class="">hq</text> |
| <text x="1295" y="1469.975" font-size="27" text-anchor="middle" class="">final query token</text> |
| </g> |
| <path d="M905,1399 L905,1293" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <path d="M1295,1399 L1295,1293" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/> |
| <text x="1100" y="1560.0" font-size="27" text-anchor="middle" class="">Same weights for all K candidates</text> |
| <text x="1100" y="1591.59" font-size="27" text-anchor="middle" class="">Head and softmax computed in FP32</text> |
| <path d="M110,1626 L1438,1626" fill="none" stroke="#A36A47" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round"/> |
| <text x="395" y="1668.0" font-size="29" text-anchor="middle" class="serif">Decision-1.0-Sol (2B)</text> |
| <text x="395" y="1704.0" font-size="25" text-anchor="middle" class="">N = 24 · d = 2048 · FFN = 6144</text> |
| <text x="395" y="1738.0" font-size="23" text-anchor="middle" class="">18 DeltaNet + 6 GQA layers · Q/KV heads 8/2</text> |
| <text x="1112" y="1668.0" font-size="29" text-anchor="middle" class="serif">Decision-1.0-Nox (4B)</text> |
| <text x="1112" y="1704.0" font-size="25" text-anchor="middle" class="">N = 32 · d = 2560 · FFN = 9216</text> |
| <text x="1112" y="1738.0" font-size="23" text-anchor="middle" class="">24 DeltaNet + 8 GQA layers · Q/KV heads 16/4</text> |
| <text x="750" y="1784.0" font-size="25" text-anchor="middle" class="">Same computation; independently trained weights · K = 2…255</text> |
| </svg> |
|
|
