MTPLX accelerates decoding on Apple Silicon with the model's own multi-token-prediction heads. There is no second draft model and no distillation: the head drafts, one batched forward verifies, and exact rejection sampling decides what is kept. Built by Youssof Altoukhi, who brought native MTP to the Mac in April 2026.
The drafter is the target's own MTP heads, so there is no second model in memory and no external drafter to maintain.
Probability-ratio acceptance with residual (p − q)+ correction. Verified max_diff = 0.0 against reference single-token AR.
MLX source fork plus custom Metal kernels tuned for the verify hot path. OpenAI- and Anthropic-compatible serving on top.
Per cycle, the MTP head drafts K tokens, the target verifies all K in one batched forward, and acceptance is decided per position by the exact rule. When all K are accepted, the verify pass also yields one extra token.
K tokens drafted from the target's own built-in MTP heads, with proposal probabilities q.
Target evaluates all K positions in one forward via GraphBank-compiled verify shapes.
Per-position acceptance via Leviathan–Chen rejection sampling. fp32 ratio path because BF16 underflows.
On rejection, sample a clean replacement from (p − q)+. Rejected drafts never enter committed history.
Committed-history KV writeback. When every position is accepted, the verify pass also yields the token at K+1.
Draft acceptance by position on Qwen 3.6 27B, depth 4, 192-token coding bench, sampled at temperature 0.6. Every accepted position is a token that never needed its own forward pass. Measured 28 April 2026.
That is the burst lane: a 192-token generation with thinking off, the shape of a short chat reply. The same configuration on an uncapped 11,390-token Flappy Bird generation with reasoning on ran 62.95 tok/s, and a 3k-token chat answer on the current Qwen 3.8 flagship runs 64.3 tok/s (MTPLX 2.10.0, M5 Max, stock settings). Qwen 3.8 Flash-Next, a 125B MoE, decodes 68.4 tok/s at 16k context and 44.2 tok/s at 206k (MTPLX 2.11, M5 Max 128 GB, temperature 1, coding prompts). Speed falls with context on every engine; the full curve is on Benchmarks.
Speculative decoding is only worth it if the output distribution is untouched. MTPLX accepts drafted tokens via the Leviathan–Chen probability ratio min(1, p/q) and, on rejection, resamples from the residual (p − q)+, the textbook construction that provably preserves the target distribution at any temperature. MTPLX verifies it empirically too: logits_max_abs_diff = 0.0 against single-token autoregressive decoding, across turns, with the session cache on.
oMLX, the 21,000-star continuous-batching server, ships Lightning MTP on MTPLX kernels. Its README: "Lightning MTP's verify-shape Metal kernels are powered by MTPLX by Youssof Altoukhi, which also inspired the depth-k pipeline." mlx-serve credits MTPLX for its verify-width split-K quantized matmul family and its M5 NAX tensor-ops tile. Ivan Fioravanti's llm_context_benchmarks ships an MTPLX benchmark, and edgequake-llm ships an MTPLX provider. More on the press page.