File size: 14,718 Bytes

43d6004

<svg xmlns="http://www.w3.org/2000/svg" width="1500" height="1810" viewBox="0 0 1500 1810" role="img"><title>Decision decoder family — actual causal backbone and candidate readout</title><desc>One shared architecture, two independently trained Qwen3.5 text checkpoints. Numerical candidate readout, with no generated explanation tokens.</desc>
<defs><marker id="arrow" markerWidth="12" markerHeight="10" refX="11" refY="5" orient="auto" markerUnits="userSpaceOnUse"><path d="M0 0 L12 5 L0 10 L3 5 Z" fill="#202C36"/></marker></defs>
        <style>text{font-family:Arial,Helvetica,sans-serif;fill:#202C36;font-weight:400}.math{font-family:Georgia,'Times New Roman',serif;font-style:italic}.serif{font-family:Georgia,'Times New Roman',serif}</style>
<text x="355" y="52.0" font-size="31" text-anchor="middle" class="serif">(a) Causal decision backbone</text>
<path d="M112,75 L635,75" fill="none" stroke="#A36A47" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round"/>
<text x="355" y="110.0" font-size="27" text-anchor="middle" class="">K probabilities · no token generation</text>
<g data-box="110.0 155 490 62">
<rect x="110.0" y="155" width="490" height="62" rx="7" fill="#DFE9DE" stroke="#202C36" stroke-width="2.2"/>
<text x="355" y="195.18" font-size="27" text-anchor="middle" class="">Softmax over valid candidates</text>
</g>
<path d="M355,154 L355,125" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="145.0 265 420 82">
<rect x="145.0" y="265" width="420" height="82" rx="7" fill="#E5E0EA" stroke="#202C36" stroke-width="2.2"/>
<text x="355" y="299.385" font-size="27" text-anchor="middle" class="">Shared candidate scorer</text>
<text x="355" y="330.97499999999997" font-size="27" text-anchor="middle" class="">expanded in (b)</text>
</g>
<path d="M355,264 L355,218" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="120.0 405 470 90">
<rect x="120.0" y="405" width="470" height="90" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/>
<text x="355" y="443.385" font-size="27" text-anchor="middle" class="">Gather K candidate endpoints</text>
<text x="355" y="474.97499999999997" font-size="27" text-anchor="middle" class="">and the final query token</text>
</g>
<path d="M355,404 L355,348" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<text x="355" y="550.0" font-size="30" text-anchor="middle" class="math">H ∈ Rᴸˣᵈ</text>
<path d="M355,526 L355,496" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="180.0 595 350 60">
<rect x="180.0" y="595" width="350" height="60" rx="7" fill="#F4EDD7" stroke="#202C36" stroke-width="2.2"/>
<text x="355" y="634.18" font-size="27" text-anchor="middle" class="">Final RMSNorm</text>
</g>
<path d="M355,594 L355,563" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<rect x="100" y="725" width="555" height="645" rx="23" fill="#F7F5EF" stroke="#202C36" stroke-width="2.2"/>
<text x="76" y="1056.5" font-size="30" text-anchor="end" class="math">N×</text>
<circle cx="355" cy="785" r="18" fill="#FFFFFF" stroke="#202C36" stroke-width="2.1"/>
<text x="355" y="793.99" font-size="29" text-anchor="middle" class="">+</text>
<path d="M355,766 L355,656" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="145.0 837 420 130">
<rect x="145.0" y="837" width="420" height="130" rx="7" fill="#DCE8E8" stroke="#202C36" stroke-width="2.2"/>
<text x="355" y="879.5899999999999" font-size="27" text-anchor="middle" class="">SwiGLU FFN</text>
<text x="355" y="911.18" font-size="27" text-anchor="middle" class="">u = Wᵤ x; g = SiLU(Wg x)</text>
<text x="355" y="942.7699999999999" font-size="27" text-anchor="middle" class="">Wdown (g ⊙ u)</text>
</g>
<path d="M355,836 L355,804" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="180.0 1008 350 58">
<rect x="180.0" y="1008" width="350" height="58" rx="7" fill="#F4EDD7" stroke="#202C36" stroke-width="2.2"/>
<text x="355" y="1046.18" font-size="27" text-anchor="middle" class="">RMSNorm</text>
</g>
<path d="M355,1007 L355,968" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<circle cx="355" cy="1117" r="18" fill="#FFFFFF" stroke="#202C36" stroke-width="2.1"/>
<text x="355" y="1125.99" font-size="29" text-anchor="middle" class="">+</text>
<path d="M355,1098 L355,1067" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="110.0 1161 490 98">
<rect x="110.0" y="1161" width="490" height="98" rx="7" fill="#F0DDC2" stroke="#202C36" stroke-width="2.2"/>
<text x="355" y="1190.0800000000002" font-size="24" text-anchor="middle" class="">Gated DeltaNet · 3 of 4 layers</text>
<text x="355" y="1218.16" font-size="24" text-anchor="middle" class="">Gated causal GQA · 1 of 4</text>
<text x="355" y="1246.2400000000002" font-size="24" text-anchor="middle" class="">Q/K RoPE in GQA only</text>
</g>
<path d="M355,1160 L355,1136" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="180.0 1284 350 58">
<rect x="180.0" y="1284" width="350" height="58" rx="7" fill="#F4EDD7" stroke="#202C36" stroke-width="2.2"/>
<text x="355" y="1322.18" font-size="27" text-anchor="middle" class="">RMSNorm</text>
</g>
<path d="M355,1283 L355,1260" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<path d="M355,1356 L610,1356 L610,1117 L374,1117" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<path d="M355,1082 L610,1082 L610,785 L374,785" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<circle cx="355" cy="1356" r="3.5" fill="#202C36"/>
<circle cx="355" cy="1082" r="3.5" fill="#202C36"/>
<g data-box="180.0 1410 350 58">
<rect x="180.0" y="1410" width="350" height="58" rx="7" fill="#F0DDD4" stroke="#202C36" stroke-width="2.2"/>
<text x="355" y="1448.18" font-size="27" text-anchor="middle" class="">Token embeddings</text>
</g>
<path d="M355,1409 L355,1343" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="110.0 1510 490 84">
<rect x="110.0" y="1510" width="490" height="84" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/>
<text x="355" y="1545.385" font-size="27" text-anchor="middle" class="">State · question · K candidates</text>
<text x="355" y="1576.975" font-size="27" text-anchor="middle" class="">global query suffix</text>
</g>
<path d="M355,1509 L355,1469" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<text x="1100" y="52.0" font-size="31" text-anchor="middle" class="serif">(b) Dynamic candidate scorer</text>
<path d="M790,75 L1410,75" fill="none" stroke="#A36A47" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round"/>
<text x="1100" y="110.0" font-size="31" text-anchor="middle" class="math">pᵢ = softmaxᵢ(sᵢ / T)</text>
<g data-box="850.0 155 500 82">
<rect x="850.0" y="155" width="500" height="82" rx="7" fill="#DFE9DE" stroke="#202C36" stroke-width="2.2"/>
<text x="1100" y="189.38500000000002" font-size="27" text-anchor="middle" class="">Mask padded candidates</text>
<text x="1100" y="220.97500000000002" font-size="27" text-anchor="middle" class="">softmax over K valid scores</text>
</g>
<path d="M1100,154 L1100,125" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<circle cx="1100" cy="315" r="18" fill="#FFFFFF" stroke="#202C36" stroke-width="2.1"/>
<text x="1100" y="323.99" font-size="29" text-anchor="middle" class="">+</text>
<path d="M1100,296 L1100,238" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<text x="1142" y="285.0" font-size="29" text-anchor="start" class="math">sᵢ</text>
<g data-box="742.5 490 325 90">
<rect x="742.5" y="490" width="325" height="90" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/>
<text x="905" y="528.385" font-size="27" text-anchor="middle" class="">Scaled dot product</text>
<text x="905" y="559.975" font-size="27" text-anchor="middle" class="">kᵢᵀ v / √256</text>
</g>
<g data-box="1142.5 490 305 90">
<rect x="1142.5" y="490" width="305" height="90" rx="7" fill="#E5E0EA" stroke="#202C36" stroke-width="2.2"/>
<text x="1295" y="544.18" font-size="27" text-anchor="middle" class="">Linear → scalar</text>
</g>
<path d="M905,489 L905,315 L1081,315" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<path d="M1295,489 L1295,315 L1119,315" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="1165.0 640 260 58">
<rect x="1165.0" y="640" width="260" height="58" rx="7" fill="#E5DCE6" stroke="#202C36" stroke-width="2.2"/>
<text x="1295" y="678.18" font-size="27" text-anchor="middle" class="">GELU</text>
</g>
<path d="M1295,639 L1295,581" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="742.5 790 325 90">
<rect x="742.5" y="790" width="325" height="90" rx="7" fill="#E5E0EA" stroke="#202C36" stroke-width="2.2"/>
<text x="905" y="828.385" font-size="27" text-anchor="middle" class="">kᵢ = Wk cᵢ</text>
<text x="905" y="859.975" font-size="27" text-anchor="middle" class="">v = Wq q</text>
</g>
<path d="M905,789 L905,581" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="1122.5 790 345 90">
<rect x="1122.5" y="790" width="345" height="90" rx="7" fill="#E5E0EA" stroke="#202C36" stroke-width="2.2"/>
<text x="1295" y="828.385" font-size="27" text-anchor="middle" class="">Ac cᵢ + Aq q + b</text>
<text x="1295" y="859.975" font-size="27" text-anchor="middle" class="">256-dimensional</text>
</g>
<path d="M1295,789 L1295,699" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<path d="M1100,990 L905,990 L905,881" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<path d="M1100,990 L1295,990 L1295,881" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<circle cx="1100" cy="990" r="3.5" fill="#202C36"/>
<g data-box="847.5 1040 505 88">
<rect x="847.5" y="1040" width="505" height="88" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/>
<text x="1100" y="1077.385" font-size="27" text-anchor="middle" class="">Normalized pair (cᵢ, q)</text>
<text x="1100" y="1108.975" font-size="27" text-anchor="middle" class="">broadcast q to every candidate</text>
</g>
<path d="M1100,1039 L1100,991" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="755.0 1230 300 62">
<rect x="755.0" y="1230" width="300" height="62" rx="7" fill="#F4EDD7" stroke="#202C36" stroke-width="2.2"/>
<text x="905" y="1270.18" font-size="27" text-anchor="middle" class="">LayerNorm c</text>
</g>
<g data-box="1145.0 1230 300 62">
<rect x="1145.0" y="1230" width="300" height="62" rx="7" fill="#F4EDD7" stroke="#202C36" stroke-width="2.2"/>
<text x="1295" y="1270.18" font-size="27" text-anchor="middle" class="">LayerNorm q</text>
</g>
<path d="M905,1229 L905,1180 L995,1180 L995,1129" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<path d="M1295,1229 L1295,1180 L1205,1180 L1205,1129" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<g data-box="742.5 1400 325 90">
<rect x="742.5" y="1400" width="325" height="90" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/>
<text x="905" y="1438.385" font-size="27" text-anchor="middle" class="">hᵢ</text>
<text x="905" y="1469.975" font-size="27" text-anchor="middle" class="">candidate endpoint</text>
</g>
<g data-box="1142.5 1400 305 90">
<rect x="1142.5" y="1400" width="305" height="90" rx="7" fill="#EBEEE8" stroke="#202C36" stroke-width="2.2"/>
<text x="1295" y="1438.385" font-size="27" text-anchor="middle" class="">hq</text>
<text x="1295" y="1469.975" font-size="27" text-anchor="middle" class="">final query token</text>
</g>
<path d="M905,1399 L905,1293" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<path d="M1295,1399 L1295,1293" fill="none" stroke="#202C36" stroke-width="2.3" stroke-linecap="round" stroke-linejoin="round" marker-end="url(#arrow)"/>
<text x="1100" y="1560.0" font-size="27" text-anchor="middle" class="">Same weights for all K candidates</text>
<text x="1100" y="1591.59" font-size="27" text-anchor="middle" class="">Head and softmax computed in FP32</text>
<path d="M110,1626 L1438,1626" fill="none" stroke="#A36A47" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round"/>
<text x="395" y="1668.0" font-size="29" text-anchor="middle" class="serif">Decision-1.0-Sol (2B)</text>
<text x="395" y="1704.0" font-size="25" text-anchor="middle" class="">N = 24 · d = 2048 · FFN = 6144</text>
<text x="395" y="1738.0" font-size="23" text-anchor="middle" class="">18 DeltaNet + 6 GQA layers · Q/KV heads 8/2</text>
<text x="1112" y="1668.0" font-size="29" text-anchor="middle" class="serif">Decision-1.0-Nox (4B)</text>
<text x="1112" y="1704.0" font-size="25" text-anchor="middle" class="">N = 32 · d = 2560 · FFN = 9216</text>
<text x="1112" y="1738.0" font-size="23" text-anchor="middle" class="">24 DeltaNet + 8 GQA layers · Q/KV heads 16/4</text>
<text x="750" y="1784.0" font-size="25" text-anchor="middle" class="">Same computation; independently trained weights · K = 2…255</text>
</svg>