Every mlx-serve number and quote on this page comes from the mlx-serve repository at tag v26.9.3 (16 September 2026), with the file and the line. Every MTPLX number links to its release note. MTPLX has not yet run both engines on one Mac itself; the one same-machine table that exists is mlx-serve's own, and it is reproduced below as they published it. Created by Youssof Altoukhi, who brought native MTP to the Mac in April 2026.
What mlx-serve does well
mlx-serve is written in Zig with no Python in the serving path. It loads MLX weights and GGUF files, speaks the OpenAI and Anthropic APIs, ships a signed menu-bar app called MLX Core, and generates images, video, music and speech from the same server. Its default speculative decoding on Qwen 3.8 Flash Next uses the same acceptance rule family as MTPLX, probability-ratio acceptance with residual resampling, so at its defaults it is exact. Its README credits MTPLX for two of its Metal kernel families (see kernel credit).
The numbers
Three sets. The first is mlx-serve's own measurement of both engines on one M5 Max, from their repository. The second is their published release-by-release table, which is measured on an M4 Max at temperature 0 with MTP forced on. The third is MTPLX on its own M5 Max. Different machines and different prompts are not comparable with each other, so read each set on its own.
1. Their M5 Max table, both engines
Source: mlx-serve docs/mtp-acceptance-port.md, lines 42 to 46, at v26.9.3. Their conditions: llmprobe 0.6.7, creative sampling (temperature 1.0, top-p 0.95), thinking off, three timed samples, sampled drafts, MTP depth 3. The MTPLX column is their run of an MTPLX build (David Tai's pull request 475 tree) on the MTPLX Optimized Speed pack with bf16 KV. Their own note on the table: "different packs, KV formats, sampler filter orders, and unpaired random streams".
| Engine and mode | Short prompt, tok/s | About 16K context, tok/s | Exact? |
|---|---|---|---|
| MTPLX, exact (their measurement) | 102.0 | 90.9 | Yes |
| mlx-serve, exact default | 92.5 | 82.4 | Yes |
| mlx-serve, Typical 0.2 (opt-in) | 106.3 | 86.7 | No, in their words |
| mlx-serve, TokenV3 0.95 (opt-in) | 106.5 | 89.9 | No, in their words |
Exact against exact, MTPLX is ahead at both lengths on their machine. At 16K, MTPLX exact is also ahead of both of their opt-in modes. The two rows where mlx-serve is ahead are the modes their own documentation calls not distribution-exact.
2. Their published release table
Source: mlx-serve benchmarks.md at v26.9.3. Their rules for that table: Apple M4 Max 128 GB only, llmprobe median of three on its own code-completion prompt, MTP forced on. The numbers are temperature 0. Their Hugging Face card for the Flash Next pack reports the same class of number on the same machine: about 60 tok/s serial and 78 tok/s with MTP on mlx-serve 26.8.11.
| Model, their pack | 26.8.11 | 26.9.1 | 26.9.2 | 26.9.3 |
|---|---|---|---|---|
| Qwen3.8 27B 4-bit, MTP | 70 | 71 | 68 | 73 |
| Qwen3.8 Flash-Next mixed 4-8-bit, MTP | 70 | 68 | 80 | 80 |
Their changelog records that through 26.9.2 the greedy speculative path could commit a reserved or padding token id, and that before 26.9.3 sampled requests barely speculated (their commit a758a93: 19 percent of tokens from MTP rounds against 96 percent for greedy). The 26.8.11 to 26.9.2 columns were measured on builds with both of those behaviours.
3. MTPLX on its own M5 Max
MacBook Pro M5 Max with 128 GB, fans verified at maximum, sampled at the model's own settings. Conditions and sources for every row are on the benchmarks page.
| Model and run | tok/s | Conditions | Source |
|---|---|---|---|
| Qwen 3.8 Flash Next, one OpenCode request | 125.8 | 1,301 tokens generated, 18,539-token prompt with 18,364 tokens served from cache, MTP depth 3, OpenCode Desktop, 16 September 2026 | 2.11.3 |
| Qwen 3.8 Flash Next, 9k-token code prompt | 79.3 | 1,500 tokens generated, seeded sampler, thinking off, two alternating boots each; 62.5 on 2.11.2 | 2.11.3 |
| Qwen 3.8 Flash Next, 109k-token OpenCode turn | 61.8 | mean of two runs (60.8 and 62.7); 48.8 on the same build before the launcher and depth-policy fix | 2.11.3 |
| Qwen 3.8 Flash Next, 200k-token OpenCode turn | 50.3 | warm, mean of two runs (50.0 and 50.6) | 2.11.3 |
| Qwen 3.8 27B Optimized Speed, rewriting a file it just wrote | 87.6 | stock settings, 29 August 2026 | 2.10.0 |
| Qwen 3.8 27B Bare Speed, fresh coding task | 65.2 | official Qwen 3.8 sampling, generation to the model's own stop, 15 August 2026 | 2.7.0 |
Exactness, in their words
Speed only counts when the tokens are the tokens the model would have written. Here is what each engine says about that, and how it is checked.
- Their default is exact. The default Flash Next path in mlx-serve accepts a draft with probability min(1, p/q) and resamples from the residual on a rejection, the same rule family as MTPLX. Their source states the law and pins it with a test.
- Their faster modes are not, in their words. "Neither mode is distribution-exact relative to the target model." (docs/mtp-acceptance-port.md, lines 5 to 6.) The server logs "NOT distribution-exact" when either mode runs (src/generate.zig, lines 2412 to 2413). Both modes are off by default and are 7 to 27 percent faster than their exact path by their changelog.
- Temperature 0 through 26.9.2. Their changelog for 26.9.3 (line 23, commit 62c4f8d): the MTP, drafter and DFlash verifies committed an argmax without the reserved-token mask, so speculative decoding at temperature 0 could emit a reserved or padding token. Fixed in 26.9.3.
- Byte identity at temperature 0. Neither engine has it on a mixture-of-experts model with selected keys. Their gotchas page says greedy speculative output matches serial output only for the first 30 to 80 tokens (docs/gotchas/engine-mlx.md, line 112). MTPLX sits on the same bf16 floor: on six prompts its fast path and plain path part only where the model's two best candidates are an exact tie or one bf16 step apart, and the 2.11.3 release notes say so rather than claiming byte identity.
- How MTPLX checks it. Every release draws a thousand four-token samples from the fast path and a thousand from the plain path at temperature 1, top-p 0.95, top-k 20, and compares the joint distributions. On 2.11.3 the distance sits within the plain path's own split-half noise at every joint length on both Flash Next and the 27B Quality pack. 2.11.3 also fixed eight exactness defects MTPLX found in its own engine, each with a test that pins it. The same thousand-draw test passed on mlx-serve's default mode on its own pack; the gap between the engines is not the default path, it is the lossy switches shipped next to it and the temperature-0 path through 26.9.2.
The prefix cache
A restored session should decode as if it had never been paused. mlx-serve's own notes put its restored session 0.14 to 0.30 nats away from a cold run on the top-5 log probabilities (tests/qwen38_flash_next_score.py, lines 12 to 14; docs/gotchas/models-media.md, lines 1635 to 1637), and their standing rule for a byte-stable greedy run is to turn the prefix cache off (docs/gotchas/engine-mlx.md, lines 2012 to 2015).
MTPLX measured its own restore three ways for 2.11.3. A seeded warm turn and the same turn recomputed cold part only on a literal coin toss (two candidates at probability 0.493 each on Flash Next, 0.488 each on the 27B), a three-token restore is byte-identical for 400 tokens on both packs, and a restore from the SSD tier across a daemon restart parts from cold by one bf16 step. In the installed app a 96,760-token conversation restored in 8 ms and a 54,530-token one in 4 ms.
Pack quality
Both engines publish how close their quantized packs sit to the bf16 model. The methods differ, so this is an order-of-magnitude comparison, their number against ours.
| Pack | Top-1 agreement with bf16 | KL divergence | Method and source |
|---|---|---|---|
| mlx-serve Qwen 3.8 Flash Next, 3.3-bit pack | 85.6% | 1,200 held-out positions, one greedy token each, their CHANGELOG at 26.9.3 (89.1% for their 70 GB mixed pack) | |
| mlx-serve Qwen 3.8 27B, 4-bit | 83.6% | 0.322 | same method, docs/gotchas/models-media.md, lines 1495 to 1497 |
| mlx-serve Qwen 3.8 27B, 8-bit | 95.5% | 0.0136 | same method, same source |
| MTPLX Qwen 3.8 27B Optimized Speed, 4-bit dynamic | 96.0% | 0.012 | teacher-forced over 2,389 positions of code, prose, JSON and a multilingual notice against the bf16 checkpoint, 16 September 2026 |
| MTPLX Qwen 3.8 27B Optimized Quality, 8-bit dynamic | 99.3% | 0.0005 | same run |
MTPLX's 4-bit 27B pack sits closer to bf16 than mlx-serve's published 8-bit 27B pack. MTPLX has not published a pack-quality number for its Flash Next packs.
Kernel credit, dated
| Date | Event | Source |
|---|---|---|
| July 2026 | mlx-serve's README performance chart compares v26.8.3 with MTPLX 2.5.3 on Gemma 4 and Qwen 3.6 on an M4 Max and claims +10% decode over MTPLX. That is a July build of MTPLX, not Qwen 3.8, not Flash Next and not an M5. | mlx-serve README.md, line 152, at v26.9.3 |
| 16 September 2026 | The mlx-serve README acknowledgements credit MTPLX by Youssof Altoukhi for "the verify-width split-K quantized matmul family and the M5 NAX tensor-ops tile", and print MTPLX's preferred credit line. | mlx-serve README.md, line 186, at v26.9.3 |
| 16 September 2026 | Their acceptance-mode document measures MTPLX exact above mlx-serve exact on an M5 Max (the table above). | docs/mtp-acceptance-port.md, lines 42 to 46 |
What is different
- One exact path, no lossy switch. MTPLX ships one speculative path and it is exact at every temperature. mlx-serve ships an exact default plus two opt-in modes that its documentation calls not distribution-exact.
- Measured exactness every release. MTPLX compares its fast path with its plain path by token id on every release and publishes the result in the release notes.
- Warm turns that equal cold turns. MTPLX's session cache restores the attention KV and the recurrent state at commit boundaries and was measured to restore to the cold state up to one bf16 step. mlx-serve documents a 0.14 to 0.30 nat drift on restore.
- Packs. MTPLX publishes its own dynamic 4-bit and 8-bit packs with the MTP head kept and the vision tower preserved, built and checked with Forge. Its 4-bit 27B pack measures 96.0 percent top-1 agreement with bf16.
- Scope. mlx-serve also runs GGUF files and generates images, video, music and speech. MTPLX is a language-model engine: chat, coding agents, an OpenAI and Anthropic compatible local server, embeddings and reranking.
- Language. mlx-serve is Zig. MTPLX is a Python and MLX engine with custom Metal kernels and a native Swift app; the app bundles its own runtime, so nothing is installed by hand.