---
license: apache-2.0
base_model: Qwen/Qwen3.5-9B
library_name: mlx
pipeline_tag: text-generation
tags:
  - mlx
  - psilm
  - physics
  - qwen3.5
  - nvfp4
---

![PsiLM](psilm-banner.png)

# Qwen3.5-9B-PsiLM

**Qwen3.5 9B — a decoder that is mostly recurrent — coupled to a physics model through trained latent bridges, runnable on a Mac with one command.**

*Research generated by Claude (Anthropic) under the direction of Ryoji Furui; see the [AI generation disclosure](#ai-generation-disclosure).*

## What this is

PsiLM (ΨLM) runs a **frozen language model** and a **frozen physics model** *together* while one answer is produced. Nothing is fine-tuned and no text crosses the interface: small trainable **bridges** read the physical problem out of the language model's hidden states, the physics model computes, and its result flows back into the language model's residual stream as a few soft tokens through a gated cross-attention. This repository packages that system for a third backbone family, and the first one that is not an attention stack:

| part | what | trained? | where it comes from |
|---|---|---|---|
| language model | Qwen3.5 9B text tower, NVFP4 (32 layers: 24 Gated DeltaNet linear-attention layers carrying a recurrent state, 8 full-attention; hidden 4096) | frozen | **this repo, at its root** (`config.json`, `model.safetensors`, the tokenizer): Ollama's `qwen3.5:9b-mlx` release reassembled for `mlx-lm`, the exact weights the bridges were trained against — see [The backbone in this repo](#the-backbone-in-this-repo) |
| physics model | a 1D Burgers Fourier Neural Operator, 70K parameters (`physics/fno_burgers_singlemode.safetensors`) | frozen | this repo (also in [`ryoji-info/PsiLM-physics`](https://huggingface.co/ryoji-info/PsiLM-physics)) |
| bridges | forward readout + value-token channel + gated injection, 28.4M parameters (`bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/`) | **trained** | this repo (also in [`ryoji-info/PsiLM-bridges`](https://huggingface.co/ryoji-info/PsiLM-bridges)) |

**What it can answer today.** One family of questions, the one the bridges were trained on:

> A velocity field on the periodic domain [0,1) starts as u(x,0) = *a* · sin(2πx + *φ*). It evolves by Burgers' equation with viscosity 0.02 until t = 0.5. What is the value of u at x = *x₀*? Answer with a number rounded to 2 decimal places.

with *a* in [0.5, 1.5], *φ* in [0, 6.28], *x₀* in [0, 0.99], two decimals each. Qwen3.5 alone answers 3.3% of them (it derives for its whole 768-token budget and, forced to commit, is usually wrong). Coupled through the bridges it answers **100%** of the sixty held-out questions within ±0.05 (MAE 0.015) — above the **98.3%** it reaches when the true value is written into the prompt as text, the oracle ceiling. On everything else the gate stays shut and the model is Qwen3.5 (guard-rail below).

**What it is not.** Not a general physics assistant, not a fine-tuned Qwen3.5, and not a model that will know when a *different* PDE applies: the physics model solves exactly one equation family and the bridges read exactly three quantities (*a*, *φ*, *x₀*) from the text. It is a research artifact: a working, measured instance of latent coupling between a language model and a physics model — here through a residual stream that is carried by recurrent state for most of the depth above the injection — on consumer hardware.

## Run it

Apple Silicon Mac (the NVFP4 backbone is 8.0 GB on disk; MLX ≥ 0.32.2 reads NVFP4 natively; training of the bridges peaked at 16 GB on a 24 GB M2, inference needs less), Python 3.11+.

```bash
# 1. get this repo (8.0 GB of backbone + ≈114 MB of bridges + 0.5 MB of physics model)
huggingface-cli download ryoji-info/Qwen3.5-9B-PsiLM --local-dir Qwen3.5-9B-PsiLM
cd Qwen3.5-9B-PsiLM

# 2. dependencies (mlx, mlx-lm, transformers, torch, huggingface_hub + the psilm package from GitHub)
pip install -r requirements.txt

# 3. one command
python psilm_infer.py
```

Nothing else is downloaded: the backbone is the directory you just fetched, and the script uses it by default (run from elsewhere, `--backbone ryoji-info/Qwen3.5-9B-PsiLM` loads the same files through `mlx-lm` into the Hugging Face cache). The script answers the default question (*a* = 1.28, *φ* = 0.5, *x₀* = 0.76) three ways and prints timing:

```
[1] PsiLM (coupled)     : 'u at x = 0.76 equals -0.26.'
    value               : -0.26   (20.0 s)
[2] backbone alone      : "...The characteristic time for diffusion to smooth out a gradient of size $L$ is $L^2 / \nu$.\n\nAnswer:1.28\n\nWait"
    value               : 1.28   (64.3 s, 786 tokens, answer forced)
[3] physics model (FNO) : u(0.76) = -0.2522   (8 ms; the reference the coupled answer should match to +-0.05)
    spectral solver     : u(0.76) = -0.2517   (ground truth)
PsiLM match (|-0.26--0.25| <= 0.05); backbone alone off
```

(Recorded on 2026-09-11 against the same weights as a local directory; the coupled arm's 20 s is the release script's uncached staged decode, the guard-rail's KV-cached decoder answers the same questions in 2.35 s.)

Other questions and options:

```bash
python psilm_infer.py --a 0.9 --phi 2.1 --x0 0.33     # any (a, phi, x0) in the ranges above
python psilm_infer.py --no-baseline                   # skip the slow backbone-alone arm
python psilm_infer.py --question-only                 # print the exact prompt, load nothing
python psilm_infer.py --help                          # --bridges DIR, --physics FILE, --backbone ID, ...
```

Any other copy of the same weights works too: `python psilm_infer.py --backbone /path/to/qwen3.5-9b-mlx` (for instance one you rebuilt yourself with `eval/ollama_to_mlx.py qwen3.5:9b-mlx --out /path/to/qwen3.5-9b-mlx` from an `ollama pull`). Without `pip install`-ing the package, a clone of the GitHub repository works: `PSILM_REPO=/path/to/PsiLM python psilm_infer.py`.

Loading the pieces yourself, in Python:

```python
import json, mlx.core as mx
from psilm.mlx.gemma_loader import load_backbone_any     # dispatches on the config: Qwen3.5 -> psilm.mlx.qwen35_loader
from psilm.mlx.bridges import PsiBridgesMLX
from psilm.mlx.fno import load_fno_safetensors
from psilm.mlx.model import PsiLMMLX

model, stock, tok = load_backbone_any(".")   # or "ryoji-info/Qwen3.5-9B-PsiLM"
d = "bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective"
cfg = json.load(open(f"{d}/config.json"))
bridges = PsiBridgesMLX(**cfg["construct"])                # d_model 4096, channel "value", inj_cap 0.2, readout_norm "dim"
bridges.load_weights(f"{d}/bridges.safetensors", strict=False)   # the retired learned-pointer tensors are omitted
fno = load_fno_safetensors("physics/fno_burgers_singlemode.safetensors")
psi = PsiLMMLX(model, tok, fno, bridges, l_fwd=cfg["coupling"]["l_fwd"], l_rev=cfg["coupling"]["l_rev"])   # read @13, inject @26 of 32
# psi.generate(QABuilder(hf_tokenizer), {"a": 1.28, "phi": 0.5, "x0": 0.76}) -- see psilm_infer.py
```

## What it costs and what it buys

| component | parameters | on disk | trained? |
|---|---:|---:|---|
| Qwen3.5 9B text tower, NVFP4 (language model) | 8.95B | 8.0 GB | **frozen** |
| PsiLM bridges, one (backbone, task) pair | **28.4M** | 114 MB | **trained** |
| Burgers FNO, the physics hemisphere | 0.07M | 0.55 MB | frozen, pretrained |

The trained part is **0.32% of the backbone's parameters** and 1.4% of its checkpoint size. The 8.95B never move.

Measured on one Apple M2 (24 GB): **+0.32% parameters and +114 MB turn 3.3% into 100% on the physics task, at 5.7× lower latency** (2.35 vs 13.5 seconds per question on the guard-rail's physics set, 16.9 vs 160 generated tokens), with GSM8K, MMLU and BoolQ unchanged — the gate's σ is 0.81 on physics against 0.004–0.014 elsewhere, so the channel is shut when physics is irrelevant. The held-out backbone figure (3.3%) is its own text protocol: it spends the whole 768-token budget deriving, is forced to answer, and is usually wrong.

## Results for this backbone

Held-out evaluation, 60 questions, accuracy within ±0.05 (`results/stage2_qwen35/final_eval.json` in the [GitHub repository](https://github.com/ryoji-info/PsiLM)):

| arm | accuracy | MAE | note |
|---|---:|---:|---|
| Qwen3.5 9B alone, forced to answer | 3.3% | 0.570 | derives for the whole 768-token budget on every item, then commits to a wrong number |
| **PsiLM (this repo)** | **100%** | **0.0147** | bridges read the prompt, FNO computes, value returns in latent space; largest error 0.047 |
| oracle (true value written into the prompt) | 98.3% | 0.100 | the tool-loop ceiling; its one miss is a forced reply whose parser picked the phase out of the derivation (5.61 for a true −0.202) |
| always answer 0.00 | 1.7% | 0.308 | calibration |

Scored again on the numerics the bridges were trained with — the differentiable (pure-MLX) recurrent scan above the injection layer rather than the Metal kernel, `final_eval_ops.json` — PsiLM is again 100% at MAE 0.015, with 58 of the 60 answers identical to two decimals and the other two within 0.01.

Guard-rail: does the coupled model still do everything else? 100 questions per dataset, three arms — backbone alone / PsiLM / PsiLM with the injection zeroed — with the gate recorded per question (`results/bench/guardrail_qwen35_summary.json`):

| dataset (n=100) | backbone | **PsiLM** | zeroed | gate σ (PsiLM) | open on |
|---|---:|---:|---:|---:|---:|
| physics QA (this task) | 1% | **99%** | 10% | 0.81 | 100% |
| GSM8K, 384 tokens | 83% | **83%** | 83% | 0.004 | 0% |
| MMLU, 5 subjects, 256 tokens | 64% | **66%** | 64% | 0.014 | 0% |
| BoolQ | 90% | **89%** | 90% | 0.008 | 0% |

GSM8K is item-identical across the three arms (paired: 83 both, 0 either way). MMLU's +2 is two items gained and none lost (p = 0.5), from a parse rate of 0.86 against 0.82 — the brevity effect the paper's leaky-gate section describes; BoolQ's −1 is one item (p = 1). The backbone's 1% on physics is its 160-token nudge protocol on this bench; the zeroed arm's 10% is not physics: with the write zeroed every reply runs to the 32-token cap and the number scored from it is 0.5, which the question itself contains, and 10 of the 100 answers lie within ±0.05 of 0.5 (Gemma's zeroed arm does the same, on the same 10 items).

The gate selectivity comes from a **no-harm training arm**: 1,194 non-physics items from 597 questions — 400 from GSM8K's training split, each with and without the `Answer:` nudge line, and 197 from MMLU's validation split, each twice as it is (997 distinct prompts) — paired with the first 32 tokens of Qwen3.5's own greedy continuations, on which only the gate receives gradients, under a mean-gate penalty (`config.json` → `training`). In the training logs the gate closed on the negatives within the first no-harm chunk (0.001 at their answer positions, against 0.94 on physics prompts) and stayed shut (0.0003 at the end), while the physics rollouts went 93.8%, 100%, 100%.

Training: 11,500 steps on one Apple M2 (24 GB) — 2,000 readout-only warm-up steps at batch 8 (3.4 s/step), 8,000 coupled steps at batch 2 (lr 3e-4; 4.6–8.5 s/step at a 16 GB peak), then 1,500 no-harm steps at lr 1e-4 (batch 2; 7.7 s/step at a 35–38 GB peak, through swap — the negatives are long sequences and the recurrent scan's backward pass keeps its whole recurrence in memory). The recipes are `results/qwen35/phaseA_recipe.sh` and `results/qwen35/coupled_recipe.sh`; the coupling-depth probes that put the injection at layer 26 (a memory cliff between 16 and 19 GB, not a design preference) are in `results/qwen35/probes.txt`. The readout standardizes each hidden dimension with statistics from a 32-prompt calibration pass (`readout_norm: "dim"`; the two frozen vectors `fwd.dim_mu`, `fwd.dim_sigma` are in the checkpoint), the adjustment Gemma needed, simply left on here.

### Bridges in this repository

| directory | task | physics model | trained params | held-out | status |
|---|---|---|---|---:|---|
| `bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/` | 1D Burgers, single-mode initial conditions, value at *x₀* | `physics/fno_burgers_singlemode.safetensors` | 28.4M | **100%** @±0.05, MAE 0.0147 (n=60; oracle 98.3%); GSM8K 83/83 | **released** |

## How it works, in one paragraph

The prompt runs through the first 13 of Qwen3.5's 32 layers. The **forward bridge** reads the queried position *x₀* by pooling the hidden states over its tokens (a deterministic span pointer computed by the QA builder, plus a 100-bin classifier) and the initial-condition parameters with a learned pool, after a calibrated per-dimension standardization, and emits (*a*, sin *φ*, cos *φ*) and *x₀*; from these it builds the initial condition on a 128-point grid. The frozen **FNO** evolves it to t = 0.5. A learned periodic lookup kernel reads the field at *x₀*, and the **value-token channel** turns that single number into eight soft tokens through Fourier features. At layer 26 a **gated cross-attention** injects them into the residual stream, capped at 20% of the stream's RMS; the gate is a small MLP on the residual stream, trained to open on physics prompts and close elsewhere. Layers 26–32 and the answer are Qwen3.5's own — four of those six layers are Gated DeltaNet, so the injected value travels onward mostly through recurrent state rather than attention. Nothing in the bridges knows what kind of layer they read from or write into. Details, the coupling-depth measurement and the failure analyses that produced this design are in the paper (`paper/psilm.pdf` in the repository, Section 9.8 for this backbone).

Two things the adapter (`psilm/mlx/qwen35_loader.py`) does that the attention backbones never needed: it hands each Gated DeltaNet layer a boolean key-validity mask derived from the staged forward's additive causal mask (the recurrence would otherwise zero every real token), and it switches only the layers above the injection onto `mlx-lm`'s differentiable recurrent scan during training, because the Metal kernel has no backward rule. The staged forward reproduces the stock model exactly (`results/qwen35/setup_summary.txt`: maximum logit difference 0.0, also at every real position of a right-padded row).

## The backbone in this repo

The root of this repository is a loadable `mlx-lm` model: **the exact Qwen3.5 9B weights the bridges were trained against**, so that no separate download or conversion is needed. It is not a new quantization. It is Ollama's `qwen3.5:9b-mlx` release — NVFP4, 4-bit floats with one 8-bit (E4M3) scale per 16-value block, on every wide linear projection of the decoder — reassembled from Ollama's per-tensor blob store into a Hugging Face-style directory by [`eval/ollama_to_mlx.py`](https://github.com/ryoji-info/PsiLM/blob/main/eval/ollama_to_mlx.py), with the vision tower removed. No Hub NVFP4 build of Qwen3.5-9B is bit-compatible with it (the Hub builds also quantize the embeddings, the output head and the recurrence's gate projections, which this release keeps in bf16), which is why it is republished here rather than referenced.

| tensors | dtype | what |
|---|---|---|
| 200 + 200 | uint32 weights + uint8 scales (NVFP4, group 16) | every wide linear projection: q/k/v/o, Gated DeltaNet in_proj_qkv / in_proj_z / out_proj, MLP gate/up/down |
| 203 | bf16 | embeddings (248,320 × 4,096, untied), output head, all norms, the recurrence's conv1d, dt_bias and its two 32-wide gate projections (in_proj_a, in_proj_b) |
| 24 | fp32 | the recurrence's decay parameters (A_log) |

627 tensors, 7.97 GB on disk, 8.95B parameters. `config.json` carries the `quantization` stanza (`{"group_size": 16, "bits": 4, "mode": "nvfp4"}`) that the Ollama release leaves implicit in tensor metadata; `vision_config` is removed along with the 333 vision tensors. As a plain language model:

```python
from mlx_lm import load, generate
model, tokenizer = load("ryoji-info/Qwen3.5-9B-PsiLM")      # mlx>=0.32.2, mlx-lm>=0.31.3
prompt = tokenizer.apply_chat_template([{"role": "user", "content": "What is Burgers' equation?"}],
                                       add_generation_prompt=True, enable_thinking=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=64))
```

Provenance and checks: `ollama pull qwen3.5:9b-mlx` (manifest `registry.ollama.ai/library/qwen3.5/9b-mlx`), upstream [`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B), Apache-2.0. The converter renames Ollama's `<w>.weight.scale` to the `<w>.scales` that `mlx.nn.QuantizedLinear` expects, writes the quantization stanza, drops `vision_tower.*` and copies the licence — byte-for-byte the same weights. PsiLM's staged forward reproduces this model's stock forward exactly (`results/qwen35/setup_summary.txt` in the GitHub repository: maximum logit difference 0.0 at batch 1, and 0.0 at every real position of a right-padded row).

## Is the answer really coming through the channel?

Two controls on the earlier backbones, and the second is decisive. **Zeroing the injection** while running everything else — readout, FNO, value tokens, gate — removes the physics result (0% for Qwen3-8B; 10% for Gemma, whose zeroed replies all give 0.5, a number the question contains, which lies within ±0.05 of the answer on 10 of the 100). **Corrupting only the number** — feeding the value encoder another question's answer at matched magnitude, with prompt, readout, gate, reply length and parsing untouched — makes the frozen model report the corruption: the spoken answer lands within ±0.05 of the *injected* value on **99 of 100** held-out questions and within ±0.05 of the truth on 9, while the KL to the base model is unchanged. Run on non-physics prompts the same swap changes nothing, which separates what the channel does by its **presence** from what it does by its **content**. Records: `results/bench/leaky_8b_shuf_guardrail_summary.json` and §9.7 of the [paper](https://github.com/ryoji-info/PsiLM/blob/main/paper/psilm.pdf). The zeroed arm of this backbone's guard-rail (table above) repeats the first control here.

## Limitations

- **One task family.** The bridges read exactly the three quantities of the trained question and the FNO solves exactly one equation family; a different PDE, boundary condition, viscosity or final time is out of scope, and the gate closing on non-physics text does not mean it can recognize *other* physics. Free-text initial conditions are not supported.
- **The pointer is task-supplied.** Which tokens hold *x₀* is computed by the QA builder from the prompt (`QABuilder.x0_span`), not learned from the words. `psilm_infer.py` builds the prompt itself for that reason; a paraphrased question is not the trained input.
- **This exact backbone.** The bridges were trained through these NVFP4 weights and expect them: the ones at the root of this repo, or a directory rebuilt from Ollama's `qwen3.5:9b-mlx` with `eval/ollama_to_mlx.py`. No other Qwen3.5-9B quantization on the Hub is bit-compatible (they also quantize the embeddings, the output head and the recurrence's gate projections, which this one keeps in bf16); the script refuses a backbone of another width or depth. MLX's quantized kernels are not guaranteed bit-identical across `mlx` versions or Apple chips, so a given question can land a hundredth away from the recorded run; the numbers above are from mlx 0.32.2 / mlx-lm 0.31.3 on an M2.
- **Two scan paths.** Training used `mlx-lm`'s pure-MLX recurrent scan above the injection (the Metal kernel has no backward rule); inference uses the kernel. The two differ by 1.4e-2 in maximum relative logit difference with identical argmax; on the held-out set they give the same 60 answers to two decimals on 58 items and differ by at most 0.01 on the other two.
- **Thinking is off.** The backbone is driven with `enable_thinking=False`, and its chat template then opens every assistant turn with an empty `<think></think>` block; answer positions and token budgets count from after it. With thinking on, nothing here has been measured.
- **Inputs with two decimals, inside the training ranges.** The readout was trained on numbers formatted like the training set; `psilm_infer.py` rounds inputs to two decimals and warns outside [0.5, 1.5] × [0, 6.28].
- **Evaluation scope.** The guard-rail covers GSM8K, a five-subject MMLU slice, BoolQ and this physics set at n=100 each (`results/bench/guardrail_qwen35_guardrail_summary.json`); nothing else has been measured, and the leaky-gate sweep of the paper has not been run on this backbone.

## Beyond physics

The bridges here couple a frozen language model to a frozen *physics* model, but the recipe (read a fixed set of quantities from text; let a frozen quantitative model compute; return one value through a selective gate) is not specific to PDEs. A calibrated market or event-probability model in the physics model's seat would be the same architecture, and the appeal is the same: a language model's forecast grounded in a model that can be validated separately, with a gate that stays shut when the model does not apply. Nothing in this repository has been trained or tested on financial data; the physics results relied on exact oracles, deterministic targets and no distribution shift, none of which markets provide. This is a research direction, not a capability, and not a basis for investment decisions.

## Files

```
psilm_infer.py          the one-command CLI (PsiLM / backbone alone / physics model)
requirements.txt        pip dependencies, including the psilm package from GitHub
bridges/qwen3.5-9b-nvfp4-mlx-1d-value-selective/
    bridges.safetensors     the trained bridges (28.4M params, fp32, 114 MB)
    config.json             construction, coupling depths, training record, per-chunk held-out scores
physics/fno_burgers_singlemode.safetensors   the frozen FNO (torch key names; complex spectral weights as .real/.imag)
MANIFEST.md             what each file is and its sha256
psilm-banner.png        banner
README.md               this card
```

## Related

- Code, training scripts, evaluation records and the paper: [github.com/ryoji-info/PsiLM](https://github.com/ryoji-info/PsiLM)
- Bridges for every backbone (Qwen2.5-0.5B, Qwen3-1.7B, Qwen3-8B, Gemma 4 12B, Qwen3.5 9B): [ryoji-info/PsiLM-bridges](https://huggingface.co/ryoji-info/PsiLM-bridges)
- The frozen physics models: [ryoji-info/PsiLM-physics](https://huggingface.co/ryoji-info/PsiLM-physics)
- The Gemma 4 12B release, same layout: [ryoji-info/Gemma-4-12B-PsiLM](https://huggingface.co/ryoji-info/Gemma-4-12B-PsiLM)

## Support

PsiLM is independent research, run on a single Apple M2. If it is useful to you, you can support the work at [ko-fi.com/ryojifurui](https://ko-fi.com/ryojifurui).

## Citation

```bibtex
@misc{furui2026psilm,
  title  = {PsiLM: Coupling Frozen Language and Physics Models through Trainable Latent Bridges},
  author = {Furui, Ryoji},
  year   = {2026},
  url    = {https://github.com/ryoji-info/PsiLM},
  note   = {Research generated by Claude (Anthropic) under the author's direction}
}
```

## License

The bridges, the FNO and the code in this repository are released under **Apache-2.0**. The backbone at the root is a repackaging of the Qwen Team's Apache-2.0 Qwen3.5 release ([Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)) as shipped by Ollama; its licence text is in `LICENSE`.

## AI generation disclosure

This model, its training recipe, the evaluations and this card were generated by **Claude** (Anthropic; on the recipe the earlier backbones' campaigns established), operating as an autonomous research agent under the direction and review of Ryoji Furui, who set the research question and the hardware constraint, approved each stage, and bears responsibility for the published claims. All numbers on this card are taken from committed evaluation records in the repository, cited by file name above.