MTPLX/Benchmarks

Qwen 3.8 27B at 64 tok/s
on a MacBook.

MTPLX 2.10.0 on an M5 Max, stock settings: Qwen 3.8 27B Optimized Speed decodes 64.3 tok/s on a 3k-token chat answer, 30.4 tok/s at 88k context, 18.4 tok/s at 147k, and 87.6 tok/s rewriting a file it just wrote. The record lane runs 81.74 tok/s on Qwen 3.6 27B at 2.69x plain decode. Qwen 3.8 Flash-Next, a 125B MoE, decodes 63 to 76 tok/s, and a 262,144-token prompt prefills on a laptop.

Conditions

All first-party numbers: single stream on a MacBook Pro M5 Max with 128 GB, fans pinned, sampled at the model's own sampler. Each row states the MTPLX version, pack, task or context length and date, and links to its release note or raw log. Measured by Youssof Altoukhi, who brought native MTP to the Mac.

Lanes

Decode speed on a laptop falls with context length, because every token reads the whole KV cache. A number is only comparable to another number in the same lane.

LaneWhat it isWho it represents
BurstA 192-token generation on a short coding prompt, thinking off (mtplx bench tune)A short chat reply. Where speculative decoding looks best.
ChatA 3k-token answer to a chat promptA normal conversation turn with reasoning.
Coding taskA medium-reasoning coding prompt, generation to the model's own stopOne agent turn writing code.
Long answerUncapped 11k to 52k-token answersA full game or document in one turn; decay within one response.
Agent contextDecode at 88k and 147k tokens of contextAn agent that has read files for an hour. Where most agent sessions run.
RewriteRewriting a file the model just wrote, drafts copied from contextEdit-heavy agent turns. New-text decode speed is in the chat and agent-context rows.

Qwen 3.8 27B, the current flagship

Official Qwen 3.8 sampling throughout: temperature 1.0, top-p 0.95, top-k 20. Packs are described on the model page.

LanePacktok/sConditionsSource
Coding taskBare Speed65.2M5 Max, mtplx serve, medium reasoning, fans verified, to the model's stop, 15 Aug 20262.7.0
Coding taskOptimized Speed58.7same run; acceptance by depth 0.95 / 0.88 / 0.802.7.0
Coding task, in the appBare Speed64.4installed app, cold session, 17.0 GB peak2.7.0
Coding task, in the appOptimized Speed55.5installed app, cold session, 23.6 GB peak2.7.0
Coding task, in the appOptimized Quality (8-bit)48.3installed app, cold session, 32.7 GB peak2.7.0
Long answerBare Speed32.4one 52,740-token answer, 27.2 minutes, ended at the model's own stop2.7.0
Long answerOptimized Speed35.1 to 37.3xhigh reasoning, 28k and 20k-token answers2.7.0
Depth-3 pack tableOptimized Speed / Bare / Quality46.8 / 49.9 / 39.22.3x / 2.3x / 3.0x the same pack decoded without MTP, quantized draft heads, 20 Aug 20262.9.0
ChatOptimized Speed64.33k-token chat answer, stock settings, 29 Aug 2026 (55.9 on 2.9.2)2.10.0
Agent contextOptimized Speed30.488k tokens of context (23.6 on 2.9.2)2.10.0
Agent contextOptimized Speed18.4147k tokens of context (12.0 on 2.9.2)2.10.0
RewriteOptimized Speed87.6rewriting a file the model just wrote, cache-copy drafting (73.8 on 2.9.2)2.10.0
PrefillOptimized Speed535prefill tok/s at 88k context (379 on 2.9.2)2.10.0
48 GB seatOptimized Speed33simulated 48 GB Mac, 42k context, 645 tok/s prefill, memory governor2.10.0
Agent first tokenOptimized Speed0.11 smid-session turn with preserved reasoning (1.8 to 2.2 s before)2.10.0
Claude Code, 165k first turnOptimized Speed3.9 sfollow-up time to first token, 165,165 of 165,502 tokens from cache, 1 Sep 20262.10.2
Other engines, same night, same task, same sampling (15 Aug 2026): Qwen 3.6 27B Optimized Speed V2 on MTPLX 59.9 to 60.1 tok/s; oMLX 0.5.7 serving its own Qwen 3.8 4-bit MTP quant 63.3 tok/s; LM Studio on the 52,740-token long answer 17.40 tok/s against 32.4 here.

Decode by context length

Qwen 3.8 27B Optimized Speed, M5 Max, stock settings, MTPLX 2.9.2 against 2.10.0, from the 2.10.0 release note. Decode at 3k runs 3.5 times faster than at 147k on the same Mac; every engine has a curve like it.

3k · 2.10.064.3 tok/s
3k · 2.9.255.9 tok/s
88k · 2.10.030.4 tok/s
88k · 2.9.223.6 tok/s
147k · 2.10.018.4 tok/s
147k · 2.9.212.0 tok/s

Between turns the session cache matters more than peak decode speed: a 100k-token session restores in about two seconds after a restart instead of a five-minute cold prefill, and mid-session tool rounds restore warm in under two seconds (2.0.0).

Records

Records by lane, each with the long-form number from the same configuration where one exists.

DateRecordtok/sConditionsSource
29 Apr 2026First past 6060.169Qwen 3.6 27B, depth 3, 192-token coding bench, temp 0.6 / top-p 0.95 / top-k 20, fans pinned; no-MTP control 23.59 (2.55x)History
2 Jul 2026Burst lane record81.74Qwen 3.6 27B Optimized Speed, depth 3, 192-token bench, thinking off, temp 0.6, fans verified 7,821 to 7,830 RPM, twin runs 81.74 / 81.73; plain decode 30.37 (2.69x)raw logs
2 Jul 2026Same config, long answer62.95uncapped 11,390-token Flappy Bird, reasoning on, seed 0, clean stop; 75.5 over the first 128 tokens, 39.9 over the last 128notes
7 Jul 2026Qwen 3.5 9B, 6-bit verify kernels82.9 to 112.5Qwen 3.5 9B 6-bit, short context, M5 Max; 61.6 to 99.7 at 8k2.0.1
18 Jul 2026Qwen 3.5 4B, rebuilt draft head227.8Qwen 3.5 4B Optimized Speed, depth 3, 1.71x over 133.6 plain decode, M5 Max2.2.0
29 Aug 2026MTP on a 125B MoE63 to 76Qwen 3.8 Flash-Next through the MTPLX server, M5 Max, 61 plain decode, workload dependent2.10.0
30 Aug 2026Prefill at 131k810Flash-Next block-sparse prefill, 131k-token prompt, tok/s of prefill; 262,144 tokens cold in 355 s2.10.1

Memory

DateWhatPeakConditionsSource
6 May 2026Sustained Mode27.5 GBQwen 3.6 27B at 32k context on Ivan Fioravanti's M5 Max benchmark, down from 98.6 GBHistory
7 May 2026Long-context curve22.1 / 27.2 / 37.5 GB32k / 64k / 128k, Sustained, with 620.6 / 504.3 / 372.1 prefill tok/s and 39.1 / 31.2 / 25.3 decodev0.2.0
6 Jul 2026Packed verify attention−8 GB / −16 GBat 64k and 128k context, Qwen 3.6 27B; decode at 128k from 17 to 20+ tok/s2.0.0
15 Aug 2026Qwen 3.8 27B packs17.0 / 23.6 / 32.7 GBBare Speed / Optimized Speed / Optimized Quality, coding task in the app2.7.0
29 Aug 2026Memory governor196,608 tokenscontext window resolved for a 48 GB Mac on the 27B Speed pack instead of the nominal 262,144; allocator growth 8.6 GB to 0.6 GB2.10.0
30 Aug 2026262k cold prompt87.4 GBFlash-Next, block-sparse prefill, 355 s; 98k prompt peak 91.4 to 83.0 GB2.10.1

Other engines

Same-machine runs against Ollama, LM Studio, llama.cpp, mlx-lm and oMLX are on the compare pages.

Notes

  • One chip. Everything first-party is an M5 Max with 128 GB. The app measures your own Mac during onboarding and picks the depth that wins there; MTPLX has published no M1, M2, M3 or M4 numbers of its own. Community-measured numbers with the command line and the log are welcome in issues.
  • Sampled, never greedy. Every number is sampled at the model's shipped sampler.
← Models Compare →