ryoji-info commited on
Commit
00ed811
·
verified ·
1 Parent(s): cdadf1d

Card, manifest, script and bridge config for the backbone-at-root layout

Browse files
MANIFEST.md CHANGED
@@ -1,12 +1,11 @@
1
  # MANIFEST — Hugging Face repo `ryoji-info/Qwen3.5-9B-PsiLM`
2
 
3
  Everything in this directory (`release/qwen3.5-9b-psilm/`) is uploaded as-is to the
4
- model repo; nothing outside it is needed. The backbone is **not** in this repo: the
5
- script downloads `ryoji-info/Qwen3.5-9B-nvfp4-mlx` at first run (the Qwen3.5 9B text
6
- tower, NVFP4, reassembled from Ollama's `qwen3.5:9b-mlx` by `eval/ollama_to_mlx.py`
7
- and to be uploaded from `/Users/rxiii/Documents/huggingface/qwen3.5-9b-mlx` by
8
- `results/hf_export/upload_backbone_large.py`; the upload was paused on 2026-09-11 at the
9
- maintainer's request, so until it completes the script needs `--backbone <local dir>`).
10
 
11
  ## Files to upload (and where each one came from)
12
 
@@ -18,11 +17,18 @@ maintainer's request, so until it completes the script needs `--backbone <local
18
  | `MANIFEST.md` | this file (harmless to upload; drop it if you prefer) | — | — |
19
  | `psilm-banner.png` | `assets/psilm-banner.png` (1600 px wide) | 1,352,073 B | `4cd65f32fe4aab5b49b66681a841b0128628e8d94c75494ae84f941b49c48089` |
20
  | `bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/bridges.safetensors` | `results/hf_export/bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/bridges.safetensors` (= `results/stage2_qwen35/bridges.npz`, step 11,500 — the end of the no-harm phase; learned-pointer tensors dropped; 28.39M params in 41 fp32 tensors) | 113,579,339 B | `a45e3b4aa1bebff75c672d64fbd665668802e4774d1d59777df89386e4c80762` |
21
- | `bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/config.json` | `results/hf_export/bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/config.json` (`eval/export_bridges.py --backbone-name ryoji-info/Qwen3.5-9B-nvfp4-mlx`; coupling 13/26 of 32; per-chunk held-out scores, n=16 except the first coupled chunk at n=48) | 1,522 B | `bd3176aa668a2f94efb6e9e61c2344b057314662c0ecac2445dc8bed531dd17b` |
22
  | `physics/fno_burgers_singlemode.safetensors` | `results/hf_export/physics/fno_burgers_singlemode.safetensors` (= `results/stage2/fno.pt`; identical to the file in the Gemma release) | 552,076 B | `7bb0076c85cdcf2505a9079c05964e3eb77ac4a776eccf34216953f3c37bfcdd` |
23
 
 
 
 
 
 
 
 
24
  The copies were made with `cp` on 2026-09-11 and the hashes match the sources
25
- (`shasum -a 256`). Not uploaded: `__pycache__/`, `.DS_Store`.
26
 
27
  ## Records the card cites
28
 
@@ -30,7 +36,7 @@ The copies were made with `cp` on 2026-09-11 and the hashes match the sources
30
  |---|---|
31
  | held-out 100% / MAE 0.0147, oracle 98.3% / 0.100, backbone 3.3% / 0.570, zero 1.7% | `results/stage2_qwen35/final_eval.json` (kernel path) |
32
  | same 60 answers on the training numerics (58 identical, 2 within 0.01) | `results/stage2_qwen35/final_eval_ops.json` (`--ops-path`) |
33
- | parity 0.0, padded-row parity 0.0, ops-vs-kernel 1.4e-2 | `results/qwen35/setup.log` (`eval/mlx_qwen35_setup.py`) |
34
  | coupling-depth probes (16.0 GB at 26, 19.1 GB at 24, 23.5 GB at 20) | `results/qwen35/probes.txt` |
35
  | training trajectory, gate on negatives 0.001 / physics 0.94 | `results/stage2_qwen35/train_log.jsonl`, `supervisor.log`, `results/qwen35/coupled.log` |
36
  | guard-rail table | `results/bench/guardrail_qwen35_summary.json` (fill in when the run completes) |
 
1
  # MANIFEST — Hugging Face repo `ryoji-info/Qwen3.5-9B-PsiLM`
2
 
3
  Everything in this directory (`release/qwen3.5-9b-psilm/`) is uploaded as-is to the
4
+ model repo, and the backbone is uploaded to the **root of the same repo** from
5
+ `/Users/rxiii/Documents/huggingface/qwen3.5-9b-mlx` (the Qwen3.5 9B text tower, NVFP4,
6
+ reassembled from Ollama's `qwen3.5:9b-mlx` by `eval/ollama_to_mlx.py`), so the repo is
7
+ both the PsiLM release and a loadable `mlx-lm` model. `psilm_infer.py` uses the directory
8
+ it sits in as the backbone when `config.json` is beside it, else the Hub id.
 
9
 
10
  ## Files to upload (and where each one came from)
11
 
 
17
  | `MANIFEST.md` | this file (harmless to upload; drop it if you prefer) | — | — |
18
  | `psilm-banner.png` | `assets/psilm-banner.png` (1600 px wide) | 1,352,073 B | `4cd65f32fe4aab5b49b66681a841b0128628e8d94c75494ae84f941b49c48089` |
19
  | `bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/bridges.safetensors` | `results/hf_export/bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/bridges.safetensors` (= `results/stage2_qwen35/bridges.npz`, step 11,500 — the end of the no-harm phase; learned-pointer tensors dropped; 28.39M params in 41 fp32 tensors) | 113,579,339 B | `a45e3b4aa1bebff75c672d64fbd665668802e4774d1d59777df89386e4c80762` |
20
+ | `bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/config.json` | `results/hf_export/bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/config.json` (`eval/export_bridges.py --backbone-name ryoji-info/Qwen3.5-9B-PsiLM`; coupling 13/26 of 32; per-chunk held-out scores, n=16 except the first coupled chunk at n=48) | 1,522 B | `bd3176aa668a2f94efb6e9e61c2344b057314662c0ecac2445dc8bed531dd17b` |
21
  | `physics/fno_burgers_singlemode.safetensors` | `results/hf_export/physics/fno_burgers_singlemode.safetensors` (= `results/stage2/fno.pt`; identical to the file in the Gemma release) | 552,076 B | `7bb0076c85cdcf2505a9079c05964e3eb77ac4a776eccf34216953f3c37bfcdd` |
22
 
23
+ | `config.json` | `/Users/rxiii/Documents/huggingface/qwen3.5-9b-mlx/config.json` (the converted backbone; not in the GitHub checkout — weights never enter git) | 2,189 B | `31426fc674d7e58150cac520a4e4cb320fc051d697e3bafb82640801b43ddb30` |
24
+ | `model.safetensors` | `/Users/rxiii/Documents/huggingface/qwen3.5-9b-mlx/model.safetensors` (the converted backbone; not in the GitHub checkout — weights never enter git) | 7,971,382,795 B | `89c1eb2f0901a5e3fd2bcd43fffbea15752fed62fcecd5a046777a1e7682eba7` |
25
+ | `tokenizer.json` | `/Users/rxiii/Documents/huggingface/qwen3.5-9b-mlx/tokenizer.json` (the converted backbone; not in the GitHub checkout — weights never enter git) | 12,807,982 B | `5f9e4d4901a92b997e463c1f46055088b6cca5ca61a6522d1b9f64c4bb81cb42` |
26
+ | `tokenizer_config.json` | `/Users/rxiii/Documents/huggingface/qwen3.5-9b-mlx/tokenizer_config.json` (the converted backbone; not in the GitHub checkout — weights never enter git) | 16,710 B | `316230d6a809701f4db5ea8f8fc862bc3a6f3229c937c174e674ff3ca0a64ac8` |
27
+ | `vocab.json` | `/Users/rxiii/Documents/huggingface/qwen3.5-9b-mlx/vocab.json` (the converted backbone; not in the GitHub checkout — weights never enter git) | 6,722,759 B | `ce99b4cb2983d118806ce0a8b777a35b093e2000a503ebde25853284c9dfa003` |
28
+ | `LICENSE` | `/Users/rxiii/Documents/huggingface/qwen3.5-9b-mlx/LICENSE` (the converted backbone; not in the GitHub checkout — weights never enter git) | 11,357 B | `5f3a3c817e78f5b8a4ad2d2c458a3e4b2cce470d6c12642c4eeb12cb8a9bf51d` |
29
+
30
  The copies were made with `cp` on 2026-09-11 and the hashes match the sources
31
+ (`shasum -a 256`). Not uploaded: `__pycache__/`, `.DS_Store`, the release smoke-run log (kept at `results/qwen35/release_smoke_run.log`), and the converted directory's `preprocessor_config.json` / `video_preprocessor_config.json` (vision-side files the text tower does not use).
32
 
33
  ## Records the card cites
34
 
 
36
  |---|---|
37
  | held-out 100% / MAE 0.0147, oracle 98.3% / 0.100, backbone 3.3% / 0.570, zero 1.7% | `results/stage2_qwen35/final_eval.json` (kernel path) |
38
  | same 60 answers on the training numerics (58 identical, 2 within 0.01) | `results/stage2_qwen35/final_eval_ops.json` (`--ops-path`) |
39
+ | parity 0.0, padded-row parity 0.0, ops-vs-kernel 1.4e-2 | `results/qwen35/setup_summary.txt` (`eval/mlx_qwen35_setup.py`) |
40
  | coupling-depth probes (16.0 GB at 26, 19.1 GB at 24, 23.5 GB at 20) | `results/qwen35/probes.txt` |
41
  | training trajectory, gate on negatives 0.001 / physics 0.94 | `results/stage2_qwen35/train_log.jsonl`, `supervisor.log`, `results/qwen35/coupled.log` |
42
  | guard-rail table | `results/bench/guardrail_qwen35_summary.json` (fill in when the run completes) |
README.md CHANGED
@@ -1,7 +1,8 @@
1
  ---
2
  license: apache-2.0
3
- base_model: ryoji-info/Qwen3.5-9B-nvfp4-mlx
4
  library_name: mlx
 
5
  tags:
6
  - mlx
7
  - psilm
@@ -24,7 +25,7 @@ PsiLM (ΨLM) runs a **frozen language model** and a **frozen physics model** *to
24
 
25
  | part | what | trained? | where it comes from |
26
  |---|---|---|---|
27
- | language model | Qwen3.5 9B text tower, NVFP4 (32 layers: 24 Gated DeltaNet linear-attention layers carrying a recurrent state, 8 full-attention; hidden 4096) | frozen | downloaded from [`ryoji-info/Qwen3.5-9B-nvfp4-mlx`](https://huggingface.co/ryoji-info/Qwen3.5-9B-nvfp4-mlx) at first run — Ollama's `qwen3.5:9b-mlx` release reassembled for `mlx-lm`, the exact weights the bridges were trained against |
28
  | physics model | a 1D Burgers Fourier Neural Operator, 70K parameters (`physics/fno_burgers_singlemode.safetensors`) | frozen | this repo (also in [`ryoji-info/PsiLM-physics`](https://huggingface.co/ryoji-info/PsiLM-physics)) |
29
  | bridges | forward readout + value-token channel + gated injection, 28.4M parameters (`bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/`) | **trained** | this repo (also in [`ryoji-info/PsiLM-bridges`](https://huggingface.co/ryoji-info/PsiLM-bridges)) |
30
 
@@ -41,7 +42,7 @@ with *a* in [0.5, 1.5], *φ* in [0, 6.28], *x₀* in [0, 0.99], two decimals eac
41
  Apple Silicon Mac (the NVFP4 backbone is 8.0 GB on disk; MLX ≥ 0.32.2 reads NVFP4 natively; training of the bridges peaked at 16 GB on a 24 GB M2, inference needs less), Python 3.11+.
42
 
43
  ```bash
44
- # 1. get this repo (≈114 MB of bridges + 0.5 MB of physics model)
45
  huggingface-cli download ryoji-info/Qwen3.5-9B-PsiLM --local-dir Qwen3.5-9B-PsiLM
46
  cd Qwen3.5-9B-PsiLM
47
 
@@ -52,7 +53,7 @@ pip install -r requirements.txt
52
  python psilm_infer.py
53
  ```
54
 
55
- The first run downloads the backbone (8.0 GB) into the Hugging Face cache. The script answers the default question (*a* = 1.28, *φ* = 0.5, *x₀* = 0.76) three ways and prints timing:
56
 
57
  ```
58
  [1] PsiLM (coupled) : 'u at x = 0.76 equals -0.26.'
@@ -64,7 +65,7 @@ The first run downloads the backbone (8.0 GB) into the Hugging Face cache. The s
64
  PsiLM match (|-0.26--0.25| <= 0.05); backbone alone off
65
  ```
66
 
67
- (Recorded on 2026-09-11 with the backbone given as a local directory, `--backbone /path/to/qwen3.5-9b-mlx --hf-tokenizer /path/to/qwen3.5-9b-mlx`; the coupled arm's 20 s is the release script's uncached staged decode, the guard-rail's KV-cached decoder answers the same questions in 2.35 s.)
68
 
69
  Other questions and options:
70
 
@@ -75,7 +76,7 @@ python psilm_infer.py --question-only # print the exact prompt,
75
  python psilm_infer.py --help # --bridges DIR, --physics FILE, --backbone ID, ...
76
  ```
77
 
78
- The backbone can also be a local directory: `python psilm_infer.py --backbone /path/to/qwen3.5-9b-mlx` (for instance one you rebuilt yourself with `eval/ollama_to_mlx.py qwen3.5:9b-mlx --out /path/to/qwen3.5-9b-mlx` from an `ollama pull`). Without `pip install`-ing the package, a clone of the GitHub repository works: `PSILM_REPO=/path/to/PsiLM python psilm_infer.py`.
79
 
80
  Loading the pieces yourself, in Python:
81
 
@@ -86,7 +87,7 @@ from psilm.mlx.bridges import PsiBridgesMLX
86
  from psilm.mlx.fno import load_fno_safetensors
87
  from psilm.mlx.model import PsiLMMLX
88
 
89
- model, stock, tok = load_backbone_any("ryoji-info/Qwen3.5-9B-nvfp4-mlx")
90
  d = "bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective"
91
  cfg = json.load(open(f"{d}/config.json"))
92
  bridges = PsiBridgesMLX(**cfg["construct"]) # d_model 4096, channel "value", inj_cap 0.2, readout_norm "dim"
@@ -146,7 +147,29 @@ Training: 11,500 steps on one Apple M2 (24 GB) — 2,000 readout-only warm-up st
146
 
147
  The prompt runs through the first 13 of Qwen3.5's 32 layers. The **forward bridge** reads the queried position *x₀* by pooling the hidden states over its tokens (a deterministic span pointer computed by the QA builder, plus a 100-bin classifier) and the initial-condition parameters with a learned pool, after a calibrated per-dimension standardization, and emits (*a*, sin *φ*, cos *φ*) and *x₀*; from these it builds the initial condition on a 128-point grid. The frozen **FNO** evolves it to t = 0.5. A learned periodic lookup kernel reads the field at *x₀*, and the **value-token channel** turns that single number into eight soft tokens through Fourier features. At layer 26 a **gated cross-attention** injects them into the residual stream, capped at 20% of the stream's RMS; the gate is a small MLP on the residual stream, trained to open on physics prompts and close elsewhere. Layers 26–32 and the answer are Qwen3.5's own — four of those six layers are Gated DeltaNet, so the injected value travels onward mostly through recurrent state rather than attention. Nothing in the bridges knows what kind of layer they read from or write into. Details, the coupling-depth measurement and the failure analyses that produced this design are in the paper (`paper/psilm.pdf` in the repository, Section 9.8 for this backbone).
148
 
149
- Two things the adapter (`psilm/mlx/qwen35_loader.py`) does that the attention backbones never needed: it hands each Gated DeltaNet layer a boolean key-validity mask derived from the staged forward's additive causal mask (the recurrence would otherwise zero every real token), and it switches only the layers above the injection onto `mlx-lm`'s differentiable recurrent scan during training, because the Metal kernel has no backward rule. The staged forward reproduces the stock model exactly (`results/qwen35/setup.log`: maximum logit difference 0.0, also at every real position of a right-padded row).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
150
 
151
  ## Is the answer really coming through the channel?
152
 
@@ -156,7 +179,7 @@ Two controls on the earlier backbones, and the second is decisive. **Zeroing the
156
 
157
  - **One task family.** The bridges read exactly the three quantities of the trained question and the FNO solves exactly one equation family; a different PDE, boundary condition, viscosity or final time is out of scope, and the gate closing on non-physics text does not mean it can recognize *other* physics. Free-text initial conditions are not supported.
158
  - **The pointer is task-supplied.** Which tokens hold *x₀* is computed by the QA builder from the prompt (`QABuilder.x0_span`), not learned from the words. `psilm_infer.py` builds the prompt itself for that reason; a paraphrased question is not the trained input.
159
- - **This exact backbone.** The bridges were trained through these NVFP4 weights and expect them: `ryoji-info/Qwen3.5-9B-nvfp4-mlx`, or a directory rebuilt from Ollama's `qwen3.5:9b-mlx` with `eval/ollama_to_mlx.py`. No other Qwen3.5-9B quantization on the Hub is bit-compatible (they also quantize the embeddings, the output head and the recurrence's gate projections, which this one keeps in bf16); the script refuses a backbone of another width or depth. MLX's quantized kernels are not guaranteed bit-identical across `mlx` versions or Apple chips, so a given question can land a hundredth away from the recorded run; the numbers above are from mlx 0.32.2 / mlx-lm 0.31.3 on an M2.
160
  - **Two scan paths.** Training used `mlx-lm`'s pure-MLX recurrent scan above the injection (the Metal kernel has no backward rule); inference uses the kernel. The two differ by 1.4e-2 in maximum relative logit difference with identical argmax; on the held-out set they give the same 60 answers to two decimals on 58 items and differ by at most 0.01 on the other two.
161
  - **Thinking is off.** The backbone is driven with `enable_thinking=False`, and its chat template then opens every assistant turn with an empty `<think></think>` block; answer positions and token budgets count from after it. With thinking on, nothing here has been measured.
162
  - **Inputs with two decimals, inside the training ranges.** The readout was trained on numbers formatted like the training set; `psilm_infer.py` rounds inputs to two decimals and warns outside [0.5, 1.5] × [0, 6.28].
@@ -182,7 +205,6 @@ README.md this card
182
 
183
  ## Related
184
 
185
- - The backbone these bridges expect: [ryoji-info/Qwen3.5-9B-nvfp4-mlx](https://huggingface.co/ryoji-info/Qwen3.5-9B-nvfp4-mlx)
186
  - Code, training scripts, evaluation records and the paper: [github.com/ryoji-info/PsiLM](https://github.com/ryoji-info/PsiLM)
187
  - Bridges for every backbone (Qwen2.5-0.5B, Qwen3-1.7B, Qwen3-8B, Gemma 4 12B, Qwen3.5 9B): [ryoji-info/PsiLM-bridges](https://huggingface.co/ryoji-info/PsiLM-bridges)
188
  - The frozen physics models: [ryoji-info/PsiLM-physics](https://huggingface.co/ryoji-info/PsiLM-physics)
@@ -206,7 +228,7 @@ PsiLM is independent research, run on a single Apple M2. If it is useful to you,
206
 
207
  ## License
208
 
209
- The bridges, the FNO and the code in this repository are released under **Apache-2.0**. The backbone is not part of this repository: `psilm_infer.py` downloads it from `ryoji-info/Qwen3.5-9B-nvfp4-mlx`, a repackaging of the Qwen Team's Apache-2.0 Qwen3.5 release ([Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)) as shipped by Ollama.
210
 
211
  ## AI generation disclosure
212
 
 
1
  ---
2
  license: apache-2.0
3
+ base_model: Qwen/Qwen3.5-9B
4
  library_name: mlx
5
+ pipeline_tag: text-generation
6
  tags:
7
  - mlx
8
  - psilm
 
25
 
26
  | part | what | trained? | where it comes from |
27
  |---|---|---|---|
28
+ | language model | Qwen3.5 9B text tower, NVFP4 (32 layers: 24 Gated DeltaNet linear-attention layers carrying a recurrent state, 8 full-attention; hidden 4096) | frozen | **this repo, at its root** (`config.json`, `model.safetensors`, the tokenizer): Ollama's `qwen3.5:9b-mlx` release reassembled for `mlx-lm`, the exact weights the bridges were trained against — see [The backbone in this repo](#the-backbone-in-this-repo) |
29
  | physics model | a 1D Burgers Fourier Neural Operator, 70K parameters (`physics/fno_burgers_singlemode.safetensors`) | frozen | this repo (also in [`ryoji-info/PsiLM-physics`](https://huggingface.co/ryoji-info/PsiLM-physics)) |
30
  | bridges | forward readout + value-token channel + gated injection, 28.4M parameters (`bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/`) | **trained** | this repo (also in [`ryoji-info/PsiLM-bridges`](https://huggingface.co/ryoji-info/PsiLM-bridges)) |
31
 
 
42
  Apple Silicon Mac (the NVFP4 backbone is 8.0 GB on disk; MLX ≥ 0.32.2 reads NVFP4 natively; training of the bridges peaked at 16 GB on a 24 GB M2, inference needs less), Python 3.11+.
43
 
44
  ```bash
45
+ # 1. get this repo (8.0 GB of backbone + ≈114 MB of bridges + 0.5 MB of physics model)
46
  huggingface-cli download ryoji-info/Qwen3.5-9B-PsiLM --local-dir Qwen3.5-9B-PsiLM
47
  cd Qwen3.5-9B-PsiLM
48
 
 
53
  python psilm_infer.py
54
  ```
55
 
56
+ Nothing else is downloaded: the backbone is the directory you just fetched, and the script uses it by default (run from elsewhere, `--backbone ryoji-info/Qwen3.5-9B-PsiLM` loads the same files through `mlx-lm` into the Hugging Face cache). The script answers the default question (*a* = 1.28, *φ* = 0.5, *x₀* = 0.76) three ways and prints timing:
57
 
58
  ```
59
  [1] PsiLM (coupled) : 'u at x = 0.76 equals -0.26.'
 
65
  PsiLM match (|-0.26--0.25| <= 0.05); backbone alone off
66
  ```
67
 
68
+ (Recorded on 2026-09-11 against the same weights as a local directory; the coupled arm's 20 s is the release script's uncached staged decode, the guard-rail's KV-cached decoder answers the same questions in 2.35 s.)
69
 
70
  Other questions and options:
71
 
 
76
  python psilm_infer.py --help # --bridges DIR, --physics FILE, --backbone ID, ...
77
  ```
78
 
79
+ Any other copy of the same weights works too: `python psilm_infer.py --backbone /path/to/qwen3.5-9b-mlx` (for instance one you rebuilt yourself with `eval/ollama_to_mlx.py qwen3.5:9b-mlx --out /path/to/qwen3.5-9b-mlx` from an `ollama pull`). Without `pip install`-ing the package, a clone of the GitHub repository works: `PSILM_REPO=/path/to/PsiLM python psilm_infer.py`.
80
 
81
  Loading the pieces yourself, in Python:
82
 
 
87
  from psilm.mlx.fno import load_fno_safetensors
88
  from psilm.mlx.model import PsiLMMLX
89
 
90
+ model, stock, tok = load_backbone_any(".") # or "ryoji-info/Qwen3.5-9B-PsiLM"
91
  d = "bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective"
92
  cfg = json.load(open(f"{d}/config.json"))
93
  bridges = PsiBridgesMLX(**cfg["construct"]) # d_model 4096, channel "value", inj_cap 0.2, readout_norm "dim"
 
147
 
148
  The prompt runs through the first 13 of Qwen3.5's 32 layers. The **forward bridge** reads the queried position *x₀* by pooling the hidden states over its tokens (a deterministic span pointer computed by the QA builder, plus a 100-bin classifier) and the initial-condition parameters with a learned pool, after a calibrated per-dimension standardization, and emits (*a*, sin *φ*, cos *φ*) and *x₀*; from these it builds the initial condition on a 128-point grid. The frozen **FNO** evolves it to t = 0.5. A learned periodic lookup kernel reads the field at *x₀*, and the **value-token channel** turns that single number into eight soft tokens through Fourier features. At layer 26 a **gated cross-attention** injects them into the residual stream, capped at 20% of the stream's RMS; the gate is a small MLP on the residual stream, trained to open on physics prompts and close elsewhere. Layers 26–32 and the answer are Qwen3.5's own — four of those six layers are Gated DeltaNet, so the injected value travels onward mostly through recurrent state rather than attention. Nothing in the bridges knows what kind of layer they read from or write into. Details, the coupling-depth measurement and the failure analyses that produced this design are in the paper (`paper/psilm.pdf` in the repository, Section 9.8 for this backbone).
149
 
150
+ Two things the adapter (`psilm/mlx/qwen35_loader.py`) does that the attention backbones never needed: it hands each Gated DeltaNet layer a boolean key-validity mask derived from the staged forward's additive causal mask (the recurrence would otherwise zero every real token), and it switches only the layers above the injection onto `mlx-lm`'s differentiable recurrent scan during training, because the Metal kernel has no backward rule. The staged forward reproduces the stock model exactly (`results/qwen35/setup_summary.txt`: maximum logit difference 0.0, also at every real position of a right-padded row).
151
+
152
+ ## The backbone in this repo
153
+
154
+ The root of this repository is a loadable `mlx-lm` model: **the exact Qwen3.5 9B weights the bridges were trained against**, so that no separate download or conversion is needed. It is not a new quantization. It is Ollama's `qwen3.5:9b-mlx` release — NVFP4, 4-bit floats with one 8-bit (E4M3) scale per 16-value block, on every wide linear projection of the decoder — reassembled from Ollama's per-tensor blob store into a Hugging Face-style directory by [`eval/ollama_to_mlx.py`](https://github.com/ryoji-info/PsiLM/blob/main/eval/ollama_to_mlx.py), with the vision tower removed. No Hub NVFP4 build of Qwen3.5-9B is bit-compatible with it (the Hub builds also quantize the embeddings, the output head and the recurrence's gate projections, which this release keeps in bf16), which is why it is republished here rather than referenced.
155
+
156
+ | tensors | dtype | what |
157
+ |---|---|---|
158
+ | 200 + 200 | uint32 weights + uint8 scales (NVFP4, group 16) | every wide linear projection: q/k/v/o, Gated DeltaNet in_proj_qkv / in_proj_z / out_proj, MLP gate/up/down |
159
+ | 203 | bf16 | embeddings (248,320 × 4,096, untied), output head, all norms, the recurrence's conv1d, dt_bias and its two 32-wide gate projections (in_proj_a, in_proj_b) |
160
+ | 24 | fp32 | the recurrence's decay parameters (A_log) |
161
+
162
+ 627 tensors, 7.97 GB on disk, 8.95B parameters. `config.json` carries the `quantization` stanza (`{"group_size": 16, "bits": 4, "mode": "nvfp4"}`) that the Ollama release leaves implicit in tensor metadata; `vision_config` is removed along with the 333 vision tensors. As a plain language model:
163
+
164
+ ```python
165
+ from mlx_lm import load, generate
166
+ model, tokenizer = load("ryoji-info/Qwen3.5-9B-PsiLM") # mlx>=0.32.2, mlx-lm>=0.31.3
167
+ prompt = tokenizer.apply_chat_template([{"role": "user", "content": "What is Burgers' equation?"}],
168
+ add_generation_prompt=True, enable_thinking=False)
169
+ print(generate(model, tokenizer, prompt=prompt, max_tokens=64))
170
+ ```
171
+
172
+ Provenance and checks: `ollama pull qwen3.5:9b-mlx` (manifest `registry.ollama.ai/library/qwen3.5/9b-mlx`), upstream [`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B), Apache-2.0. The converter renames Ollama's `<w>.weight.scale` to the `<w>.scales` that `mlx.nn.QuantizedLinear` expects, writes the quantization stanza, drops `vision_tower.*` and copies the licence — byte-for-byte the same weights. PsiLM's staged forward reproduces this model's stock forward exactly (`results/qwen35/setup_summary.txt` in the GitHub repository: maximum logit difference 0.0 at batch 1, and 0.0 at every real position of a right-padded row).
173
 
174
  ## Is the answer really coming through the channel?
175
 
 
179
 
180
  - **One task family.** The bridges read exactly the three quantities of the trained question and the FNO solves exactly one equation family; a different PDE, boundary condition, viscosity or final time is out of scope, and the gate closing on non-physics text does not mean it can recognize *other* physics. Free-text initial conditions are not supported.
181
  - **The pointer is task-supplied.** Which tokens hold *x₀* is computed by the QA builder from the prompt (`QABuilder.x0_span`), not learned from the words. `psilm_infer.py` builds the prompt itself for that reason; a paraphrased question is not the trained input.
182
+ - **This exact backbone.** The bridges were trained through these NVFP4 weights and expect them: the ones at the root of this repo, or a directory rebuilt from Ollama's `qwen3.5:9b-mlx` with `eval/ollama_to_mlx.py`. No other Qwen3.5-9B quantization on the Hub is bit-compatible (they also quantize the embeddings, the output head and the recurrence's gate projections, which this one keeps in bf16); the script refuses a backbone of another width or depth. MLX's quantized kernels are not guaranteed bit-identical across `mlx` versions or Apple chips, so a given question can land a hundredth away from the recorded run; the numbers above are from mlx 0.32.2 / mlx-lm 0.31.3 on an M2.
183
  - **Two scan paths.** Training used `mlx-lm`'s pure-MLX recurrent scan above the injection (the Metal kernel has no backward rule); inference uses the kernel. The two differ by 1.4e-2 in maximum relative logit difference with identical argmax; on the held-out set they give the same 60 answers to two decimals on 58 items and differ by at most 0.01 on the other two.
184
  - **Thinking is off.** The backbone is driven with `enable_thinking=False`, and its chat template then opens every assistant turn with an empty `<think></think>` block; answer positions and token budgets count from after it. With thinking on, nothing here has been measured.
185
  - **Inputs with two decimals, inside the training ranges.** The readout was trained on numbers formatted like the training set; `psilm_infer.py` rounds inputs to two decimals and warns outside [0.5, 1.5] × [0, 6.28].
 
205
 
206
  ## Related
207
 
 
208
  - Code, training scripts, evaluation records and the paper: [github.com/ryoji-info/PsiLM](https://github.com/ryoji-info/PsiLM)
209
  - Bridges for every backbone (Qwen2.5-0.5B, Qwen3-1.7B, Qwen3-8B, Gemma 4 12B, Qwen3.5 9B): [ryoji-info/PsiLM-bridges](https://huggingface.co/ryoji-info/PsiLM-bridges)
210
  - The frozen physics models: [ryoji-info/PsiLM-physics](https://huggingface.co/ryoji-info/PsiLM-physics)
 
228
 
229
  ## License
230
 
231
+ The bridges, the FNO and the code in this repository are released under **Apache-2.0**. The backbone at the root is a repackaging of the Qwen Team's Apache-2.0 Qwen3.5 release ([Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)) as shipped by Ollama; its licence text is in `LICENSE`.
232
 
233
  ## AI generation disclosure
234
 
bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/config.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
- "backbone": "ryoji-info/Qwen3.5-9B-nvfp4-mlx",
3
- "hf_tokenizer": "ryoji-info/Qwen3.5-9B-nvfp4-mlx",
4
  "task": "1d",
5
  "loader": "psilm.mlx.gemma_loader.load_backbone_any",
6
  "physics": {
 
1
  {
2
+ "backbone": "ryoji-info/Qwen3.5-9B-PsiLM",
3
+ "hf_tokenizer": "ryoji-info/Qwen3.5-9B-PsiLM",
4
  "task": "1d",
5
  "loader": "psilm.mlx.gemma_loader.load_backbone_any",
6
  "physics": {
psilm_infer.py CHANGED
@@ -6,9 +6,9 @@
6
  python psilm_infer.py --question-only # print the prompt, load nothing
7
  python psilm_infer.py --no-baseline # skip the (slow) backbone-alone arm
8
 
9
- Loads a frozen NVFP4 language model (default ``ryoji-info/Qwen3.5-9B-nvfp4-mlx``: the
10
- Qwen3.5 9B text tower, 24 Gated DeltaNet + 8 attention layers, exactly the weights the
11
- bridges were trained against),
12
  the trained PsiLM bridges for one task (``bridges/<name>/{bridges.safetensors,config.json}``)
13
  and that task's frozen physics model, then answers one field-value question three ways:
14
 
@@ -54,7 +54,11 @@ import time
54
  from pathlib import Path
55
 
56
  HERE = Path(__file__).resolve().parent
57
- DEFAULT_BACKBONE = "ryoji-info/Qwen3.5-9B-nvfp4-mlx"
 
 
 
 
58
  TASKS = {
59
  "1d": {"bridges": "qwen3.5-9b-nvfp4-mlx-1d-value-selective",
60
  "physics": "fno_burgers_singlemode.safetensors",
@@ -553,7 +557,7 @@ def run_1d(args):
553
  n_layers = len(model.model.layers)
554
  d_model = int(model.args.hidden_size)
555
  bridges, cfg, coupling, rep = load_bridges(args.bridges, d_model, n_layers, args)
556
- hf_id = args.hf_tokenizer or cfg.get("hf_tokenizer") or args.backbone
557
  hf_tok = AutoTokenizer.from_pretrained(hf_id)
558
  fno = load_physics(args.physics)
559
  psi = PsiLMMLX(model, tok, fno, bridges, l_fwd=coupling.get("l_fwd"), l_rev=coupling.get("l_rev"))
@@ -723,7 +727,7 @@ def run_multimode(args):
723
  n_layers = len(model.model.layers)
724
  d_model = int(model.args.hidden_size)
725
  bridges, cfg, coupling, rep = load_bridges_task(args.bridges, "multimode", d_model, n_layers, args)
726
- hf_id = args.hf_tokenizer or cfg.get("hf_tokenizer") or args.backbone
727
  hf_tok = AutoTokenizer.from_pretrained(hf_id)
728
  fno = load_physics(args.physics)
729
  psi = PsiLMMLXMulti(model, tok, fno, bridges, l_fwd=coupling["l_fwd"], l_rev=coupling["l_rev"])
@@ -798,7 +802,7 @@ def run_2d(args):
798
  n_layers = len(model.model.layers)
799
  d_model = int(model.args.hidden_size)
800
  bridges, cfg, coupling, rep = load_bridges_task(args.bridges, "2d", d_model, n_layers, args)
801
- hf_id = args.hf_tokenizer or cfg.get("hf_tokenizer") or args.backbone
802
  hf_tok = AutoTokenizer.from_pretrained(hf_id)
803
  base_path, base_src = resolve_dpot_base(args.dpot_base)
804
  phys = load_physics_2d(args.physics, base_path, args.phys_device)
 
6
  python psilm_infer.py --question-only # print the prompt, load nothing
7
  python psilm_infer.py --no-baseline # skip the (slow) backbone-alone arm
8
 
9
+ Loads a frozen NVFP4 language model (the Qwen3.5 9B text tower, 24 Gated DeltaNet + 8
10
+ attention layers, exactly the weights the bridges were trained against; it ships at the
11
+ root of this repository, so the default is this directory, else ``ryoji-info/Qwen3.5-9B-PsiLM``),
12
  the trained PsiLM bridges for one task (``bridges/<name>/{bridges.safetensors,config.json}``)
13
  and that task's frozen physics model, then answers one field-value question three ways:
14
 
 
54
  from pathlib import Path
55
 
56
  HERE = Path(__file__).resolve().parent
57
+ # The backbone ships at the root of this repo (config.json, model.safetensors, the
58
+ # tokenizer), so after `huggingface-cli download ... --local-dir` it is the directory
59
+ # this script sits in; when the script is run from elsewhere, the Hub id loads the
60
+ # same files through mlx-lm.
61
+ DEFAULT_BACKBONE = str(HERE) if (HERE / "config.json").is_file() else "ryoji-info/Qwen3.5-9B-PsiLM"
62
  TASKS = {
63
  "1d": {"bridges": "qwen3.5-9b-nvfp4-mlx-1d-value-selective",
64
  "physics": "fno_burgers_singlemode.safetensors",
 
557
  n_layers = len(model.model.layers)
558
  d_model = int(model.args.hidden_size)
559
  bridges, cfg, coupling, rep = load_bridges(args.bridges, d_model, n_layers, args)
560
+ hf_id = args.hf_tokenizer or (args.backbone if os.path.isfile(os.path.join(args.backbone, "tokenizer_config.json")) else None) or cfg.get("hf_tokenizer") or args.backbone
561
  hf_tok = AutoTokenizer.from_pretrained(hf_id)
562
  fno = load_physics(args.physics)
563
  psi = PsiLMMLX(model, tok, fno, bridges, l_fwd=coupling.get("l_fwd"), l_rev=coupling.get("l_rev"))
 
727
  n_layers = len(model.model.layers)
728
  d_model = int(model.args.hidden_size)
729
  bridges, cfg, coupling, rep = load_bridges_task(args.bridges, "multimode", d_model, n_layers, args)
730
+ hf_id = args.hf_tokenizer or (args.backbone if os.path.isfile(os.path.join(args.backbone, "tokenizer_config.json")) else None) or cfg.get("hf_tokenizer") or args.backbone
731
  hf_tok = AutoTokenizer.from_pretrained(hf_id)
732
  fno = load_physics(args.physics)
733
  psi = PsiLMMLXMulti(model, tok, fno, bridges, l_fwd=coupling["l_fwd"], l_rev=coupling["l_rev"])
 
802
  n_layers = len(model.model.layers)
803
  d_model = int(model.args.hidden_size)
804
  bridges, cfg, coupling, rep = load_bridges_task(args.bridges, "2d", d_model, n_layers, args)
805
+ hf_id = args.hf_tokenizer or (args.backbone if os.path.isfile(os.path.join(args.backbone, "tokenizer_config.json")) else None) or cfg.get("hf_tokenizer") or args.backbone
806
  hf_tok = AutoTokenizer.from_pretrained(hf_id)
807
  base_path, base_src = resolve_dpot_base(args.dpot_base)
808
  phys = load_physics_2d(args.physics, base_path, args.phys_device)