xThr45hx commited on
Commit
3a4253b
Β·
verified Β·
1 Parent(s): 3dfef62

Docs: replace in-joke test prompt with md5 reproducibility check

Browse files
Files changed (2) hide show
  1. AOT_RECIPE.md +9 -8
  2. README.md +6 -5
AOT_RECIPE.md CHANGED
@@ -1,7 +1,7 @@
1
  # AOT recipe β€” Gemma3-1B-IT int4 + Hadamard β†’ Google Tensor NPU
2
 
3
- The exact recipe behind every `.litertlm` in this repo. The G4 build made this way gives a
4
- **word-for-word identical** reply to the published G4 build on the golden prompt below.
5
 
6
  Executable version with guardrails: [`recipe/gemma3_int4hr_recipe.py`](./recipe/gemma3_int4hr_recipe.py).
7
 
@@ -70,12 +70,13 @@ Put the compiled prefill_decode into a `.litertlm` together with the embedder, a
70
  metadata sections, using `litert-lm-builder toml --path model.toml output --path <out>.litertlm`.
71
  Check the result with `litert-lm-peek --litertlm_file <out>.litertlm`.
72
 
73
- ## Golden test (does your build match?)
74
- Prompt: `IT WAS ALWAYS "LMS" BARE!`
75
- The verified G4 build replies, starting: *"Okay, I understand you're feeling frustrated and struggling
76
- with something. It sounds like you've been experiencing difficulties with a particular learning
77
- management system (LMS)…"* The context counter is ~289 tokens after the reply.
78
- A rebuild that doesn't reproduce this has drifted.
 
79
 
80
  ## Per-SoC notes
81
  | SoC | Rank-2 | Result |
 
1
  # AOT recipe β€” Gemma3-1B-IT int4 + Hadamard β†’ Google Tensor NPU
2
 
3
+ The exact recipe behind every `.litertlm` in this repo. A from-scratch G4 rebuild made this way is
4
+ **bit-identical** to the published G4 build (see "Does your build match?" below).
5
 
6
  Executable version with guardrails: [`recipe/gemma3_int4hr_recipe.py`](./recipe/gemma3_int4hr_recipe.py).
7
 
 
70
  metadata sections, using `litert-lm-builder toml --path model.toml output --path <out>.litertlm`.
71
  Check the result with `litert-lm-peek --litertlm_file <out>.litertlm`.
72
 
73
+ ## Does your build match?
74
+ With the pinned toolchain, the compiled G4 prefill_decode (step 5 output, rank-2) should be
75
+ **560,536,512 B, md5 `f1102df2fdc182d7738bd495ad8f9cf0`**, bit-identical to the one in the published
76
+ G4 bundle. A different md5 means something in your pipeline drifted (most often: quantizing export_hf's
77
+ default int8 copy instead of the float `model.tflite`).
78
+ On device, the rebuild and the published bundle also give word-for-word identical replies to the same
79
+ prompt (greedy decoding).
80
 
81
  ## Per-SoC notes
82
  | SoC | Rank-2 | Result |
README.md CHANGED
@@ -45,21 +45,21 @@ extra_gated_button_content: Acknowledge license
45
  | SoC | Pixel | File | Status |
46
  |---|---|---|---|
47
  | **Tensor G4** | Pixel 9 / 9 Pro / 9 Pro XL / 9 Pro Fold | `Gemma3-1B-IT_int4hadamard_aot_ekv4096_G4.litertlm` | βœ… **VERIFIED on device** |
48
- | **Tensor G3** | Pixel 8 / 8 Pro / 8a | `Gemma3-1B-IT_int4hadamard_aot_ekv4096_G3.litertlm` | 🟑 **Same program as the verified G4 build; passes the golden test on G4 hardware β€” awaiting a Pixel 8 report** |
49
  | Tensor G5 | Pixel 10 series | β€” | πŸ”§ in progress (current recipe doesn't compile yet for G5) |
50
  | Tensor G6 | Pixel 11 series | β€” | πŸ”§ in progress (current recipe doesn't compile yet for G6; will be verified on device) |
51
 
52
  > [!WARNING]
53
  > **Tensor G3 is not yet confirmed on a Pixel 8.** Strong evidence it works: the G3 and G4 compiles
54
  > are the **same program**. They differ in only 19 bytes (a device-type tag and its hash), and the G3
55
- > bundle **runs on a Pixel 9 (G4) and passes the golden test word-for-word**. **If you have a Pixel 8 /
56
  > 8 Pro / 8a, please test it and comment on this repo** (Community tab) with: device, whether it loads
57
  > on the NPU, tokens/s, and a sample multi-sentence reply. That's what moves it to βœ….
58
 
59
  ### Cross-generation DGC compatibility (tested on a Pixel 9 Pro XL / Tensor G4)
60
  - **G3 ↔ G4:** the G3 and G4 builds compile to the same program. They differ only in a device-type
61
  field (`03` vs `04`) in the embedded compiler options, plus its 8-byte hash. The G3 bundle loads
62
- and passes the golden test word-for-word on G4, so the G4 runtime does not enforce the tag.
63
  - **G5 β†’ G4:** in an earlier test, a G5-compiled Gemma4-E2B loaded on G4 and its **prefill** executed
64
  on the G4 NPU, but **decode did not**. G5 compiles use dual-cluster optimizations that G4 hardware
65
  lacks, so newer-generation DGCs are only partly forward-loadable.
@@ -123,8 +123,9 @@ Full step-by-step in **[`AOT_RECIPE.md`](./AOT_RECIPE.md)**; executable version
123
  **LMS only, no sharding flag.**
124
  6. **Toolchain:** `ai_edge_litert 2.2.0` + the Google Tensor beta compiler (BuildID `87ea807a…`).
125
 
126
- Coherence check: the prompt `IT WAS ALWAYS "LMS" BARE!` gives a word-for-word identical reply on the
127
- published G4 build and on a from-scratch rebuild with this recipe.
 
128
 
129
  ### Quantization notes
130
 
 
45
  | SoC | Pixel | File | Status |
46
  |---|---|---|---|
47
  | **Tensor G4** | Pixel 9 / 9 Pro / 9 Pro XL / 9 Pro Fold | `Gemma3-1B-IT_int4hadamard_aot_ekv4096_G4.litertlm` | βœ… **VERIFIED on device** |
48
+ | **Tensor G3** | Pixel 8 / 8 Pro / 8a | `Gemma3-1B-IT_int4hadamard_aot_ekv4096_G3.litertlm` | 🟑 **Same program as the verified G4 build; replies word-for-word identically to the G4 build on G4 hardware β€” awaiting a Pixel 8 report** |
49
  | Tensor G5 | Pixel 10 series | β€” | πŸ”§ in progress (current recipe doesn't compile yet for G5) |
50
  | Tensor G6 | Pixel 11 series | β€” | πŸ”§ in progress (current recipe doesn't compile yet for G6; will be verified on device) |
51
 
52
  > [!WARNING]
53
  > **Tensor G3 is not yet confirmed on a Pixel 8.** Strong evidence it works: the G3 and G4 compiles
54
  > are the **same program**. They differ in only 19 bytes (a device-type tag and its hash), and the G3
55
+ > bundle **runs on a Pixel 9 (G4) and gives word-for-word identical replies to the G4 build**. **If you have a Pixel 8 /
56
  > 8 Pro / 8a, please test it and comment on this repo** (Community tab) with: device, whether it loads
57
  > on the NPU, tokens/s, and a sample multi-sentence reply. That's what moves it to βœ….
58
 
59
  ### Cross-generation DGC compatibility (tested on a Pixel 9 Pro XL / Tensor G4)
60
  - **G3 ↔ G4:** the G3 and G4 builds compile to the same program. They differ only in a device-type
61
  field (`03` vs `04`) in the embedded compiler options, plus its 8-byte hash. The G3 bundle loads
62
+ and gives word-for-word identical replies to the G4 build on G4 hardware, so the G4 runtime does not enforce the tag.
63
  - **G5 β†’ G4:** in an earlier test, a G5-compiled Gemma4-E2B loaded on G4 and its **prefill** executed
64
  on the G4 NPU, but **decode did not**. G5 compiles use dual-cluster optimizations that G4 hardware
65
  lacks, so newer-generation DGCs are only partly forward-loadable.
 
123
  **LMS only, no sharding flag.**
124
  6. **Toolchain:** `ai_edge_litert 2.2.0` + the Google Tensor beta compiler (BuildID `87ea807a…`).
125
 
126
+ Reproducibility check: a from-scratch rebuild with this recipe produces a compiled prefill_decode that is
127
+ **bit-identical** to the published G4 build (md5 `f1102df2…`), and gives word-for-word identical replies
128
+ on device.
129
 
130
  ### Quantization notes
131