Both row pairs are third-party measurements. Each author ran both engines on one machine with one prompt set. MTPLX's own measurements are on the benchmarks page. Created by Youssof Altoukhi, who brought native MTP to the Mac in April 2026.
What llama.cpp does well
llama.cpp runs GGUF models on a wide range of hardware, and Ollama runs GGUF through it. It merged MTP on 16 May 2026 (PR #22673) and MTP for Qwen3-Next hybrid GDN models on 3 Aug 2026 (PR #25589).
The numbers
Two pairs, each measured by a third party on one machine. The first is a blog post from 19 May 2026: a MacBook, Qwen 3.6 27B, thinking off. The second is Mirai Labs' public board, dataset benchmarked 1 Sep 2026: an M5 Max with 128 GB, Qwen 3.6 27B.
| Engine | Model | Machine | Task | Sampling | Decode tok/s | Source | Date |
|---|---|---|---|---|---|---|---|
| llama.cpp, MTP on (version: see source) | Qwen 3.6 27B (quantization: see source) | MacBook, the author's machine | thinking off; TTFT 0.8 s (prompt: see source) | see source | 10.5 | vinoth12940.github.io, third party | 19 May 2026 |
| MTPLX, MTP depth 3 (version: see source) | Qwen 3.6 27B (pack: see source) | MacBook, the author's machine | thinking off; TTFT 0.66 s (prompt: see source) | see source | 18.3 | vinoth12940.github.io, third party | 19 May 2026 |
| llama.cpp + Unsloth MTP GGUF (version: see source) | Qwen 3.6 27B, Unsloth MTP GGUF | M5 Max 128 GB, Mirai Labs' machine | Mirai's dataset (see source) | see source | 29.7 | trymirai.com/metrics, third party | 1 Sep 2026 |
| MTPLX 2.9.0 | Qwen 3.6 27B Speed-V2 (Optimized Speed V2: calibrated mixed-precision, W4G32 + W8 promotions) | M5 Max 128 GB, Mirai Labs' machine | Mirai's dataset (see source) | see source | 55.4 | trymirai.com/metrics, third party | 1 Sep 2026 |
Cells that read "see source" are recorded only in the linked source. MTPLX's own measurements of Qwen 3.6 27B and Qwen 3.8 27B, with version, sampling and date on each, are on the benchmarks page.
MTP, dated
Both engines run MTP speculative decoding today. The dates:
| Date | Event | Source |
|---|---|---|
| 2 May 2026 | MTPLX v0.1.0-preview, the first public release, ships native MTP speculative decoding on the Mac. | MTPLX history |
| 16 May 2026 | llama.cpp merges MTP, two weeks after MTPLX's first public release. | PR #22673 |
| 3 Aug 2026 | llama.cpp merges MTP for Qwen3-Next hybrid GDN models. | PR #25589 |
What is different
- MTP on both engines, since different dates. MTPLX shipped native MTP speculative decoding on the Mac on 2 May 2026 (v0.1.0-preview). llama.cpp merged MTP on 16 May 2026 (PR #22673) and MTP for Qwen3-Next hybrid GDN models on 3 Aug 2026 (PR #25589). On the Qwen 3.8 27B packs MTPLX drafts three tokens ahead with the model's own MTP head, the target verifies them in one forward pass, and the speculative path runs the same model as the AR path.
- Exact at any temperature. MTPLX acceptance is probability-ratio min(1, p/q) with residual (p - q)+ resampling (Leviathan–Chen), so output follows the model's distribution at any temperature. MTPLX has never shipped a greedy-only path. Activations stay bf16 (fp16 on M1/M2) on every pass.
- Prefix caching together with speculation. Since 2.0.0 (6 Jul 2026) MTPLX checkpoints the attention KV cache plus the recurrent and conv GDN state at commit boundaries. A 100k-token session restores in about 2 s after a restart instead of a five-minute cold prefill, and mid-session tool rounds restore warm in under 2 s.
- Model format. llama.cpp runs GGUF; the Mirai row above is an Unsloth MTP GGUF. MTPLX runs MTP-ready MLX builds published under huggingface.co/Youssofal; Forge converts a Hugging Face repo into an MTP-ready MLX build on your own Mac, measures the speedup, and refuses incompatible models instead of silently falling back.
- One server, two dialects.
mtplx startor the app's play button serves an OpenAI-compatible API on 127.0.0.1:8000 (/v1/chat/completions,/v1/completions,/v1/models,/v1/embeddings,/v1/rerank) and an Anthropic-compatible/v1/messages, with streaming and tool calls in both styles. The app and the CLI share one server:mtplx startattaches to the app's loaded model instead of loading a second copy.