The plain-decode rows are the same model decoded without MTP in MLX; that is the baseline MTPLX's multipliers are computed against. MTPLX's own rows were measured on an M5 Max; the Mirai Labs rows were measured on Mirai's machine. Created by Youssof Altoukhi, who brought native MTP to the Mac in April 2026.
What mlx-lm does well
mlx-lm is Apple's reference MLX LM library, the library that runs language models on MLX. MTPLX builds on it.
The numbers
Three sets. The first set is MTPLX's first public number, from the README at tag v0.1.5 (6 May 2026): Qwen 3.6 27B Optimized Speed with MTP depth 3 beside the same pack decoded without MTP, on one M5 Max, at temp 0.6 / top-p 0.95 / top-k 20. The second is the MTPLX 2.9.0 pack table (20 Aug 2026), where "plain decode" means the same model without MTP; it is a different instrument from the 2.7.0 coding task, so its values differ from the 2.7.0 numbers on the other comparison pages. The third is Mirai Labs' public board, dataset benchmarked 1 Sep 2026.
| Engine | Model | Machine | Task | Sampling | Decode tok/s | Multiplier | Source | Date |
|---|---|---|---|---|---|---|---|---|
| MLX plain decode: same pack, MTP off (MTPLX README at tag v0.1.5) | Qwen 3.6 27B Optimized Speed | M5 Max, MTPLX's own machine | README benchmark (prompt: see source) | temp 0.6 / top-p 0.95 / top-k 20 | 28.156 | baseline, 1x | MTPLX README at tag v0.1.5 | 6 May 2026 |
| MTPLX v0.1.5, MTP depth 3 | Qwen 3.6 27B Optimized Speed | M5 Max, MTPLX's own machine | README benchmark (prompt: see source) | temp 0.6 / top-p 0.95 / top-k 20 | 63.056 | 2.24x the row above | MTPLX README at tag v0.1.5 | 6 May 2026 |
| MTPLX 2.9.0, MTP depth 3 | Qwen 3.8 27B Optimized Speed: 4-bit dynamic, 32-weight groups | M5 Max, MTPLX's own machine, fans verified at max | 2.9.0 pack-table instrument (a different instrument from the 2.7.0 coding task) | official Qwen 3.8 sampling: temp 1.0 / top-p 0.95 / top-k 20 | 46.8 | 2.3x the same model without MTP | MTPLX 2.9.0 release note | 20 Aug 2026 |
| MTPLX 2.9.0, MTP depth 3 | Qwen 3.8 27B Bare Speed: flat 4-bit, 64-weight groups | M5 Max, MTPLX's own machine, fans verified at max | 2.9.0 pack-table instrument (a different instrument from the 2.7.0 coding task) | official Qwen 3.8 sampling: temp 1.0 / top-p 0.95 / top-k 20 | 49.9 | 2.3x the same model without MTP | MTPLX 2.9.0 release note | 20 Aug 2026 |
| MTPLX 2.9.0, MTP depth 3 | Qwen 3.8 27B Optimized Quality: 8-bit, 64-weight groups | M5 Max, MTPLX's own machine, fans verified at max | 2.9.0 pack-table instrument (a different instrument from the 2.7.0 coding task) | official Qwen 3.8 sampling: temp 1.0 / top-p 0.95 / top-k 20 | 39.2 | 3.0x the same model without MTP | MTPLX 2.9.0 release note | 20 Aug 2026 |
| MLX, 4-bit, autoregressive (version: see source) | Qwen 3.6 27B, 4-bit (quantization: see source) | M5 Max 128 GB, Mirai Labs' machine | Mirai's dataset (see source) | see source | 25.7 | AR row | trymirai.com/metrics, third party | 1 Sep 2026 |
| MTPLX 2.9.0 | Qwen 3.6 27B Speed-V2 (Optimized Speed V2: calibrated mixed-precision, W4G32 + W8 promotions) | M5 Max 128 GB, Mirai Labs' machine | Mirai's dataset (see source) | see source | 55.4 | published as tok/s only | trymirai.com/metrics, third party | 1 Sep 2026 |
"Plain decode" means the same model decoded without MTP in MLX, on the same machine, measured by MTPLX as the baseline for its multipliers. In the 2.9.0 pack table the two 4-bit packs decode at 2.3x plain decode and the 8-bit Quality pack at 3.0x (M5 Max, MTP depth 3, official Qwen 3.8 sampling, 20 Aug 2026). Cells that read "see source" are recorded only in the linked source.
mlx-lm's own MTP pull request, #990, added residual sampling on 5 May 2026 and is unmerged as of 3 Sep 2026.
What is different
- Native MTP head. mlx-lm has no merged MTP as of 3 Sep 2026; its MTP pull request #990
added residual sampling on 5 May 2026 and is unmerged. MTPLX decodes with the model's own MTP head: on the
Qwen 3.8 27B packs the model drafts three tokens ahead, the target verifies them in one forward pass, and the
speculative path runs the same model as the AR path.
mtplx start --no-mtpruns plain autoregressive decoding on the same loaded model. - Exact at any temperature. Acceptance is probability-ratio min(1, p/q) with residual (p - q)+ resampling (Leviathan–Chen), so output follows the model's distribution at any temperature. MTPLX has never shipped a greedy-only path. Activations stay bf16 (fp16 on M1/M2) on every pass.
- Prefix caching together with speculation. Since 2.0.0 (6 Jul 2026) MTPLX checkpoints the attention KV cache plus the recurrent and conv GDN state at commit boundaries. A 100k-token session restores in about 2 s after a restart instead of a five-minute cold prefill, and mid-session tool rounds restore warm in under 2 s.
- Depth chosen by measurement. During onboarding MTPLX runs the model itself at each draft
depth with fans pinned, keeps plain AR as the baseline, and saves a depth only if it beats AR.
mtplx tune --model <m> --retuneruns it again. - One server, two dialects.
mtplx startor the app's play button serves an OpenAI-compatible API on 127.0.0.1:8000 (/v1/chat/completions,/v1/completions,/v1/models,/v1/embeddings,/v1/rerank) and an Anthropic-compatible/v1/messages, with streaming and tool calls in both styles. The app and the CLI share one server:mtplx startattaches to the app's loaded model instead of loading a second copy.