LFM3-1.7B-Base-MLX-5bit

Unofficial MLX build of LiquidAI/LFM3-1.7B-Base, the pretrained (not instruction-tuned) 1.7B LFM3 model, for Apple Silicon. Liquid AI publishes MLX builds of the instruct model (LiquidAI/LFM3-1.7B-MLX-5bit and others) but none of the Base model. This is a community conversion, not a Liquid AI release; the weights are LiquidAI's, quantized.

• Weights: 1.189 GB (model.safetensors).
• Quantization: 5-bit affine, group size 64, on every Linear layer and the tied embedding/LM head (lfm3_quant_recipe.json: all:5/64).
• Made with mlx-vlm main (230fc308) and MLX 0.32.4 from the bf16 checkpoint (LiquidAI/LFM3-1.7B-Base revision 545fc8f4). Same file layout as Liquid's own MLX text builds (language_model.model.* keys).
• Sibling build: alexwengg/LFM3-1.7B-Base-MLX-mixed-5-6-8.

This is a base model: use it for completion, few-shot prompting, scoring and fine-tuning. It is not tuned to follow chat instructions, and it does not stop on its own the way the instruct model does.

Quality

Measured on an Apple M5 Pro with MLX 0.32.4, against float32 references of the original checkpoint:

• HF fp32 goldens: the original checkpoint run in float32 by Hugging Face Transformers 5.21.0 on CPU. Two sets: 25 prompts (chat, tool-call and raw text; final-position next-token distribution) and 5 raw-text prompts with 64-token greedy continuations (teacher-forced next-token agreement along the fp32 continuation).
• MLX fp32 reference: the checkpoint upcast to float32 in mlx-vlm; it matches the HF fp32 run exactly on the goldens (teacher-forced agreement 1.0000, 5/5 identical greedy continuations). Wikitext-2 test, 16 x 2048 tokens; fp32 perplexity 10.588.

build	goldens (25): final top-1	goldens (25): final KL	goldens (5 raw): teacher-forced agreement	goldens (5 raw): final KL	wikitext-2 KL	wikitext-2 PPL vs fp32
bf16 (unquantized, for reference)	1.00	0.0005	n/a	n/a	0.0005	+0.05%
5-bit (this repo)	0.88	0.0305	n/a	n/a	0.0146	+1.36%
mixed-5-6-8	0.92	0.0182	n/a	n/a	0.0078	+0.75%

KL is KL(fp32 || build) in nats over the reference's top-200 next-token support. The bf16 row is the original checkpoint loaded in bf16 by the same harness. Final-position top-1 is below 1.0 even for bf16 on a few prompts whose two best next tokens are near-tied.

Speed

Apple M5 Pro (24 GB), macOS 27.0, MLX 0.32.4, mlx-vlm main (230fc308). Greedy decoding, 256 generated tokens after a 512- or 2048-token prompt, median of 3 runs after a warm-up, nothing else running on the machine.

build	decode tok/s (512 ctx)	decode tok/s (2048 ctx)	prefill tok/s (2048-token prompt)	time to first token (2048-token prompt)	peak memory (2048 ctx)
5-bit	178	173	6,300	327 ms	2.13 GB
mixed-5-6-8	166	164	6,220	331 ms	2.23 GB

Sustained: in a 60-second run of continuous writing (a relay of ~400-token pieces with sampling at temperature 0.8), this build produced 9,772-9,952 tokens, i.e. 163-166 tok/s averaged over the whole minute, peak 174 tok/s, 1.3 GB.

Use with mlx-vlm

No released mlx-vlm or mlx-lm version supports LFM3 yet (October 2026). mlx-vlm main does:

pip install "git+https://github.com/Blaizzy/mlx-vlm.git@main"

from mlx_vlm import load, generate

model, processor = load("alexwengg/LFM3-1.7B-Base-MLX-5bit")
# plain completion: pass the raw text (the generate CLI always applies a chat template)
out = generate(model, processor, "<|startoftext|>The capital of France is", max_tokens=64, temperature=0.0)
print(out.text)

python -m mlx_vlm.generate wraps the prompt in the chat template, which suits the instruct model rather than this base model; use the Python API above for raw completion.

Use with mlx-lm

mlx-lm support is in review (ml-explore/mlx-lm#2016, not merged). With that branch installed, this checkpoint loads as is; against mlx-vlm on 512 positions of text the two give the same argmax at 98.8% (5-bit) / 99.4% (mixed) of positions, max |logit difference| 0.5 (bf16 kernel differences):

pip install "git+https://github.com/ml-explore/mlx-lm.git@refs/pull/2016/head"
mlx_lm.generate --model alexwengg/LFM3-1.7B-Base-MLX-5bit --prompt "The capital of France is" --max-tokens 64 --temp 0.0 \
  --ignore-chat-template

Loading through transformers' AutoConfig may ask to trust remote code (config.json keeps Liquid's auto_map, as Liquid's own MLX builds do); MLX does not run that code.

Limitations

• Quantization moves the next-token distribution (table above); greedy continuations diverge from fp32 after a few dozen tokens on average, as for any 5-6-bit build.
• Evaluated on English text, short chat/tool prompts and wikitext-2 only; contexts up to about 2,000 tokens were tested, although the architecture supports 262,144.
• Measured on one machine (M5 Pro); speeds on other Apple chips will differ.

License

The weights are a quantized derivative of LiquidAI/LFM3-1.7B-Base and are distributed under the LFM Open License v1.0, the license of the original model. The remote-code files (configuration_lfm3.py, modeling_lfm3.py, lfm3_conv.py) are copied unchanged from the original repository.