Replace the removed bench command with mtplx serve and mtplx tune (youssofal/MTPLX#517)
Browse files
README.md
CHANGED
|
@@ -52,18 +52,15 @@ mode: affine
|
|
| 52 |
|
| 53 |
## MTPLX Usage
|
| 54 |
|
| 55 |
-
|
| 56 |
|
| 57 |
```bash
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
--assistant-model ./assistant \
|
| 61 |
-
--prompt-suite mtplx/benchmarks/prompts/flappy.jsonl \
|
| 62 |
-
--max-tokens 1000 \
|
| 63 |
-
--draft-block-sizes 6 \
|
| 64 |
-
--allow-unverified-gemma
|
| 65 |
```
|
| 66 |
|
|
|
|
|
|
|
| 67 |
The Gemma 4 assistant is a separate drafter model. MTPLX uses exact speculative sampling with target verification and residual correction.
|
| 68 |
|
| 69 |
## Local Benchmark
|
|
|
|
| 52 |
|
| 53 |
## MTPLX Usage
|
| 54 |
|
| 55 |
+
MTPLX loads the pair straight from this repository:
|
| 56 |
|
| 57 |
```bash
|
| 58 |
+
pip install mtplx
|
| 59 |
+
mtplx serve --model Youssofal/Gemma4-MTPLX-Optimized-Speed
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
```
|
| 61 |
|
| 62 |
+
In the Mac app, pick "Gemma 4 31B Optimized Speed". To find the best draft block size for your Mac, run `mtplx tune --model Youssofal/Gemma4-MTPLX-Optimized-Speed`.
|
| 63 |
+
|
| 64 |
The Gemma 4 assistant is a separate drafter model. MTPLX uses exact speculative sampling with target verification and residual correction.
|
| 65 |
|
| 66 |
## Local Benchmark
|