MTPLX 2.12.0

MTPLX 2.12.0 adds three models and makes Flash-Next much faster at reading long prompts. Ternary Bonsai 2 27B from Prism ML brings a 27B-class model with image input to Macs with 16 GB of memory. MiMo V2.6 Qwen 9B is Xiaomi's coding model built on Qwen 3.5 9B, and Flash-Next Optimized Quality is an 8-bit Flash-Next for Macs with 256 GB or more.

Flash-Next now reads a 4,061-token prompt 85 percent faster than 2.11.3, and its first token after a 65,502-token prompt arrives 25 seconds sooner. The release also fixes thinking that leaked into answers, conversations that were processed again after Stop or in long sessions, a memory guard that could not see all of the memory in use, and a repetition stop that cut real code.

Highlights

These pairs were measured on September 22 on an M5 Max with 128 GB. Both versions used the same Python runtime and were booted in the order 2.11.3, 2.12.0, 2.12.0, 2.11.3, with one model loaded at a time and the fans at maximum. Thinking was off, each answer was 512 tokens, sampling used the model's own settings with seed 1731, and the GPU memory limit was raised to 120 GiB (iogpu.wired_limit_mb=122880; the default is about 96 GiB). The prompt kernels described under Speed were added after this run. In their own paired run they added another 16 percent to prompt processing at 16K tokens and 9 percent at 64K, and with them 2.12.0 read a 65,529-token prompt at 1,548 tok/s, so the release is faster than this table shows.

The rows in this second table come from bug reports and from the tests of each fix.

New models

Ternary Bonsai 2 27B (#515)

Prism ML's Ternary Bonsai 2 27B is Qwen3.8-27B rebuilt with ternary weights, and it reads images. The pack Youssofal/Ternary-Bonsai-2-27B-MTPLX-Optimized-Speed is 8.85 GB. It carries Prism ML's language model and vision tower unchanged (8.60 GB) and adds the Qwen3.8-27B draft head (0.24 GB). The served id is mtplx-bonsai-2-27b-optimized-speed. Full credit for the model goes to Prism ML, and the pack carries their license and notice.

Measured on the same M5 Max, alternating boots of 2.12.0 without and with the kernels, one model loaded at a time, temperature 1.0, top-p 0.95 and top-k 20, thinking off, 512-token answers. The 27B ran in the same session. Bonsai is now faster than the 4-bit 27B at both lengths, in half the memory.

• Two GPU kernels written for this pack. One applies Prism ML's Hadamard rotation in a single step and returns the same bits as the four operations it replaces. The other runs the ternary matrix multiply for decoding. It is as close to Prism ML's own float32 reference as the stock kernel (mean KL 2.92e-6 against 3.02e-6 over 1,630 positions of code, prose and reasoning), and greedy output is identical with the kernels on and off. Together they also make Bonsai read prompts 7 to 8 percent faster.

• Every Bonsai load checks the kernels first. Both kernels are ordinary GPU code and run on every Mac from M1 to M5. On each load they are tested on that Mac's GPU against the stock operations. A kernel that fails is switched off until the engine restarts, the stock path takes over, and /health says so. Only the M5 Max has measured their speed.

• Compiled verification. Bonsai now uses the compiled verifier. It matched eager verification in all 613 rounds checked.

• Draft depth 1. The draft head is on by default at depth 1. With the new kernels, depth 3 was 14 percent slower than depth 1 over a 3,000-token answer. The head comes from Qwen3.8-27B and accepts 75 percent of its drafts on Bonsai.

• Medium reasoning by default. Bonsai supports two reasoning efforts, medium and xhigh, and medium is the default in the app, the CLI and every connected client. Prism ML states that low is not supported. On a stopwatch coding task, xhigh spent 577 s and 21,848 reasoning tokens without reaching an answer, while medium finished the task and a follow-up correction.

• It runs on 16 GB Macs. The memory planner refused any model that could not also fund a warm cache in RAM, which ruled out Bonsai on 16 GB. It now admits Bonsai there with an 8,192-token window (see Recommendations and memory). Measured by limiting a 128 GB M5 Max to each class's memory budget, with the draft head and the vision tower loaded:

With the draft head active under the 16 GB budget, a 7,006-token prompt and a 1,024-token answer peaked at 11.54 GiB of GPU memory and 12.79 GiB for the whole process, with no swap growth. In every run the peak sat about 3.1 GiB above the weights and the KV cache, which is what the planner sets aside.

MTPLX applies Prism ML's Hadamard rotation while it runs the model, and it refuses a pack whose rotation metadata or vision tower is missing.

MiMo V2.6 Qwen 9B

Xiaomi's MiMo-V2.6-Distill-Qwen-9B is Qwen3.5-9B fine-tuned for coding and agent work. It is in the catalog as MiMo V2.6 Qwen 9B Optimized Speed. The pack Youssofal/MiMo-V2.6-Qwen-9B-MTPLX-Optimized-Speed is 8.70 GB: the model at 6 bits with group size 64, the same recipe as Qwen 3.5 9B Optimized Speed, with Xiaomi's vision tower in BF16. Xiaomi's checkpoint has no draft head, so the pack uses the Qwen3.5-9B one. The served id is mtplx-mimo-v26-qwen-9b-optimized-speed. Full credit for the model goes to Xiaomi MiMo, and it is MIT licensed.

Xiaomi's model card reports it well ahead of the Qwen3.5-9B it started from:

• Close to the original. Against Xiaomi's BF16 checkpoint over 19,265 tokens of code and prose, the 6-bit pack has a KL divergence of 0.0054 and picks the same top token 97.3 percent of the time. That is ten times closer than a mixed 4-bit build of the same model.

• Draft head. The head was trained on Qwen3.5-9B, so it accepts fewer drafts on MiMo: 78.5 percent at the first position and 56.5 percent at the second, on a long game prompt with thinking on. Depth 2 is the default, as on the Qwen 9B.

• Memory. It plans the same windows as Qwen 3.5 9B Optimized Speed: 20,480 tokens on 16 GB, 45,056 on 18 GB and 192,512 on 24 GB. The measured peak was 8.70 GiB at a 15K-token context.

• Where it is suggested. On M3, M4 and M5 Macs with 16 GB or more, the app and the CLI list it right before Qwen 3.5 9B. On 16 to 31 GB Macs that makes it the second suggestion, after Bonsai 2. M1 and M2 keep the FP16 9B.

• Speed. Not measured yet. Each forward pass does the same work as on Qwen 3.5 9B Optimized Speed, which has the same architecture and size, but the draft head accepts fewer tokens, as noted above.

Flash-Next Optimized Quality

An 8-bit build of Qwen 3.8 Flash-Next for Macs with 256 GB or more. The main model and the draft head are at 8 bits with group size 64, the structural weights stay in BF16, and the n-gram table is at 4 bits with group size 32. The download is 169.96 GB at Youssofal/Qwen3.8-Flash-Next-MTPLX-Optimized-Quality, and with the n-gram table on SSD the weights need about 128.5 GiB of memory. All 2,860 tensors were checked, and sampled values in 25 groups were compared with the BF16 source.

It has not been run on a 256 GB Mac yet, so on those Macs the app and the CLI recommend Flash-Next Optimized Speed first and list Optimized Quality second. Its speed has not been measured.

• Build it yourself. Forge builds it with the flash-next-optimized-quality recipe. One command builds the pack, loads it, and tests chat, a tool call and an image:

bash scripts/build_flash_next_quality_pack.sh Qwen/Qwen3.8-Flash-Next <output directory>

• Forge converts Flash-Next source models (#508 by @bpmforge). It used to stop with Model type qwen4_exp not supported. The 51B n-gram table is now quantized one shard at a time straight into its own file, and the rest of the model is converted one tensor at a time with MTPLX's own model code. A follow-up reads the converted files back through the loaders the runtime uses.

Speed

Flash-Next prompt processing

• Bigger prompt chunks on M5. On M5 chips, Flash-Next now reads prompts 4,096 tokens at a time and switches to sparse attention from 16K tokens. Other chips keep 2,048-token chunks and the 32K switch point. Before a request uses the bigger chunk, the engine checks that it fits in memory, and that check now counts what a chunk really holds, so a 128 GB Mac gets the bigger chunk at 64K and 128K too. When the check says no, the request uses 2,048-token chunks, and /health, the request log and mtplx doctor --explain record it.

• Five more prompt steps run as single GPU kernels. The hyper-connection read, the combine of the expert outputs, the gated-delta prework and the gated-delta norm gate each run as one kernel now, and on M5 chips the wide input projections run as one dequantize and one dense matrix multiply. Measured on the same M5 Max, alternating boots without and with the kernels:

Every boot with the kernels was faster than every boot without them, and peak memory was the same (92.0 GB at 64K). The output is bit-identical: every position's logits and 256 greedy tokens matched on a code prompt and a prose prompt at 16K and on a code prompt at 64K. Four of the kernels run on every Mac, and every Flash-Next load first checks them on that Mac's GPU against the stock operations. A kernel that fails is switched off until the engine restarts. The check adds about 58 ms to the first load. The projection step runs only on M5 chips, where MLX's quantized and dense matrix multiplies give the same bits. M1 to M4 keep the quantized multiply and 2,048-token chunks, and their speed was not measured. Each step has its own switch: MTPLX_QWEN4_HC_PREFILL_READ=0, MTPLX_QWEN4_MOE_PREFILL_COMBINE=0, MTPLX_QWEN4_GDN_PREFILL_PREWORK=0, MTPLX_QWEN4_GDN_GATED_NORM=0 and MTPLX_QWEN4_PREFILL_DQ_GEMM=0.

• Faster kernels behind every chunk. These were rebuilt too, with the same results as before:

• The draft head does less work during prompt processing. Its pass over each chunk used to run the whole layer, although only the cache it writes is used. It now writes the cache and stops, with the same cache bits. The full pass cost 0.09 to 0.12 s per 4,096-token chunk, 3.3 s of a 104 s prompt at 128K tokens. MTPLX_QWEN4_MTP_HISTORY_CACHE_ONLY=0 restores the full pass.

• Less memory between chunks. A small copy of each layer's state kept that layer's whole input stream alive until the next chunk. It no longer does, which frees about 3.0 GB between 4,096-token chunks (85.84 GB instead of 88.86). Long forward passes also hand finished layers to the GPU every four layers, so each layer's temporary data is freed with the layer and the GPU can start before the whole pass is built. MTPLX_QWEN4_PREFILL_MIDLOOP_EVAL sets the number of layers, and 0 turns it off.

• Restore points are recorded during the last forward pass. The points a warm turn restores from used to need two extra passes after every cold prompt and one after every warm agent turn. They are now recorded inside the last pass, at the same positions. A cold 4,061-token prompt now takes 2.79 to 2.81 s instead of 3.06 to 3.10 s on the same build. MTPLX_GDN_BOUNDARY_INFORWARD=0 restores the old way.

• A warm agent turn reprocesses at most 64 tokens at the end of its prompt instead of up to 256, because the last saved restore point now sits 64 tokens before the end.

• The n-gram table is read in a better order. A cold prompt's first chunk spent 0.65 s, 20 percent of a 4K cold prompt, gathering n-gram rows, because the read-ahead filled part of the 23.8 GiB weight file and neither of the two 3.0 GiB scale and bias files. It now fills the small files first, and the warm-up pass before a large read skips files that are already in memory, which takes a third of the system calls.

Flash-Next decode

• The attention cache is no longer copied on every step. At many prompt lengths, the key and value cache was copied in full on every verify round: 24 banks of 135 MB each at 128K tokens. The cache is now sized so that every write happens in place. At 131K tokens the 24 writes went from 7.8 ms to 0.6 ms, and the attention part of each verify round from 30.9 ms to 11.2 ms, with identical output. The 65,502-token prompt in the highlights table is one of the lengths that had this copy in 2.11.3.

• Drafting takes one sync with the GPU instead of one per token. The GPU predicts the host's draw for every draft token, and the host checks each pick with the same arithmetic as before. When a pick does not match, the round finishes the old way, so the tokens are identical for every seed. Draft preparation at 4K went from about 5.2 ms to about 4.4 ms per round.

• Draft tokens go to the GPU as they are made, and the n-gram rows for the next check are read while the GPU works. A later comparison at equal GPU clocks was flat, so no speed gain is claimed for this change on its own.

• A new setting for very long contexts. MTPLX_MLX_COMMAND_BUFFER_MB sets how much memory MLX lets one GPU command buffer touch before it closes it (50 MiB on a Max). At 128K tokens, decode went from 51 to 54 tok/s with MLX's limit to 65.9 tok/s without it. MTPLX keeps MLX's default, because without the limit peak memory during prompt processing rose from 92.0 to 103.7 GB at 16K, and once the process was close to its memory limit the gain shrank to 45.6 against 49.4 tok/s. /health shows the value the engine was given.

Qwen 3.8 27B

The 27B is unchanged in this release. Measured the same way as the highlights table, with the Optimized Speed pack:

Prompt processing is within 2.1 percent of 2.11.3. One decode round costs 3.6 percent more at 4K and 0.2 percent more at 16K. Decode speed moved both ways, 5.4 percent lower at 4K and 4.3 percent higher at 16K, because the two builds accept different draft tokens from the same seed.

Recommendations and memory

The app and the CLI now read one catalog of 26 models, with the same names, sizes and recommendation rule. Model descriptions follow the app language in all 13 languages without a restart. The first recommendation on M3, M4 and M5 Macs:

MiMo V2.6 Qwen 9B is the second suggestion on 16 to 31 GB. M1 and M2 keep their FP16 recommendations, and a model you already picked stays selected.

• The n-gram table streams from SSD on every Mac. On Macs with 160 GB or more, earlier releases also kept the 29.8 GiB table in GPU memory, and nothing in 2.12.0 reads that copy. It now streams from SSD on every Mac. On 256 GB and 512 GB Macs, this raises the memory Flash-Next Optimized Quality has for context and the session cache from 29.7 to 59.5 GiB, and lowers the memory it keeps wired from 164.3 to 134.5 GiB.

• The GPU wired limit stays within what macOS allows. When the engine asked to wire more memory than macOS allows, MLX refused, the error was ignored, and the model ran with none of its memory wired and no GPU keepalive. That meant extra time after every idle pause, and memory macOS could take back under pressure. The engine now asks for the most the Mac accepts, so most of the model stays wired and the keepalive runs. The startup log says how much macOS allowed and gives the sudo sysctl iogpu.wired_limit_mb command that allows all of it. On a stock 96 GB Mac the limit is about 72 GiB.

• 96 GB Macs are pointed to Flash-Next Bare Speed. Optimized Speed peaks at 87 GiB and Bare Speed at 78 GiB. On 96 GB, the catalog, the memory planner and the verify memory check share one 84 GiB budget for Flash-Next, with a planned window of 86,016 tokens on M5 and 20,480 tokens on chips without tensor units.

• A model that does not fit gets the smallest window, not the largest. A plan that said "does not fit" was treated as "no limit", so the 27B picked by hand on a 16 or 24 GB Mac, and Flash-Next on 96 GB, got the full 262,144-token window under a MODEL DOES NOT FIT banner. They now get the 4,096-token minimum and one startup line that says so. An explicit --context-window still wins, and a Mac that fits plans as before (the 27B: 57,344 tokens on 36 GB, 204,800 on 48 GB and 262,144 from 64 GB).

• Small Macs can run small models without a warm cache in RAM. The planner refused any model whose budget could not also fund the 1 GiB session cache floor. That floor only sets the size of the warm cache and is not needed to run. When it is the only thing that does not fit, the planner now admits the model with no warm cache in RAM, and restores come from the SSD. This applies to packs no larger than Bonsai's 8.83 GB of weights, where the margin was measured, and to larger packs only when they carry their own measured memory table. Qwen 3.5 9B Optimized Speed, for example, gets a 20,480-token window on a 16 GB Mac.

• A lighter model never gets less context than a heavier one. On a 16 GB Mac, a 7.53 GiB pack planned 12,288 tokens while an 8.08 GiB pack planned 20,480, because only the heavier pack used the rule above. A lighter pack now drops the cache floor too when that gives it more context, up to what the rule grants: 24,576 tokens for the 9B layout and 12,288 for the 27B and Bonsai layout. No catalog model's window changes.

• The decode context ceiling follows the model. Every model was budgeted with the 27B's 65,536 KV bytes per token (Flash-Next uses 24,576), and every Flash-Next start printed a warning about it. The value now comes from the model's config. On 16 and 24 GB Macs the 27B ceiling is 65,536 and 98,304 tokens instead of a flat 131,072, and no default on 32 GB or more changes.

• The SSD session cache size follows RAM from the terminal too. mtplx serve used a flat 100 GB, so a 16 GB Mac got a 100 GB store while the app scaled it. The default is now the same everywhere: 16 GiB on Macs with 16 GB of RAM or less, 24 GiB up to 32 GB, 32 GiB up to 64 GB (100 GiB when the disk has at least 150 GiB free), and 100 GiB above 64 GB. A size you set still wins.

• The memory guards count the whole Mac (#516, and #500 by @ArctifoxNL). A 128 GB M5 Max serving a long OpenCode session with allow_swap on was restarted by the macOS watchdog with the MTPLX worker at 118 GiB. The engine's guard never warned, because it only measured memory held by the GPU allocator. The guards now read the process footprint that macOS itself uses, so memory outside the allocator counts, and they read how much memory macOS can still hand out (the number behind memory_pressure -Q). A large prompt is admitted only when both the engine and the rest of the Mac have room for it. The stop floor is 2.5 percent of RAM (3.2 GiB on a 128 GB Mac, never under 1 GiB), and the shed floor is twice that. Below the shed floor the check runs every 2 s instead of 10, so a request under sustained pressure stops about 6 s after the floor is crossed instead of 30. allow_swap still admits a prompt that is larger than the Mac can hold, but it no longer turns this guard off. MTPLX_SYSTEM_MEMORY_GUARD=0 turns it off, and MTPLX_HOST_MEMORY_ALLOWANCE_BYTES sets how much memory a daemon may hold outside the GPU allocator before it counts.

Images

• Flash-Next image positions stay correct through the whole reply. Since 2.10.1, only the main verify step used the image positions. The copy rounds, the repairs and the final save of a reply used plain text positions, so after a 1,024-token image those rows sat about 990 positions from where they belonged, in RAM and in the SSD copy. All eight places now use the image positions. Saved Flash-Next image conversations get a new cache key, so after the update an existing image conversation is processed again once from its first image. The text before that image is still reused.

• Flash-Next image requests use the compiled verifier. An image request used to fall back to eager verification for the whole conversation. Requests with different images now share one compiled program, and text requests are unchanged. A 2,439-token image prompt decoded at 101.6 tok/s compiled against 97.2 eager, with the same answer. Per round, the compiled route costs 4.5 percent less than eager and 2.3 percent more than a text request on the same boot. An OpenCode image task through the app ran all 678 verify steps compiled, at 77.1 tok/s. Across 440 image rounds the compiled route agrees with eager as closely as it does for text. Six unusual prompt shapes stay on the eager route, and MTPLX_QWEN4_VISION_COMPILED_VERIFY=0 turns the compiled route off.

• Dense 27B image requests use the compiled verifier too. Eighteen complete requests, half with thinking on, compared 762 compiled image rounds and 511 text rounds with the eager verifier from the same cache. The logits, hidden states and cache state were identical. MTPLX_DENSE_VISION_COMPILED_VERIFY=0 keeps dense image requests on eager verification.

• Qwen 3.5 and 3.8 27B, and Bonsai, place image tokens at their grid positions, the way the models were trained. A small reference test agreed within 0.000005.

• Adding an image keeps the conversation before it. A Pi conversation processed 154,899 tokens again, taking 184 s, after an image was attached. Image requests now restore the saved text history like text requests do, and the image contents are checked before any history that depends on an image is reused.

• Image placeholder text stays text (based on #519 by @fradav). A client that sent the image placeholder string back as plain text, for example an earlier turn's result saved as text, made the chat template treat it as a real image slot, and the request failed with "prompt contains more image placeholders than images". Only real image parts create image slots now. Placeholder text is kept as readable, escaped text in messages, in reasoning and in tool-call arguments, so an agent that edits code containing a placeholder keeps working.

Fixes

• The repetition stops are off by default. Up to 2.11.3, the server ended a reply when a block of up to 96 tokens repeated four times in a row, and it deleted every copy. Code repeats exactly, so this cut real work: a Tetris board at its ninth identical row, an empty sudoku grid, a 256-entry table of zeros, 64 identical <div> lines, and a correct patch that renamed a type in 15 places, which left the tool call unfinished. Replaying the stop over 301,194 saved replies on the test Mac (104.6 million tokens from the MTPLX app, OpenCode, Hermes, Pi and Codex) found two real patches it would have cut. A reply now ends only at the model's own stop, at the request's token limit, or when you press Stop.

• MTPLX_REPETITION_STOP=1 turns it back on, together with a new check for exact loops that repeat every 97 to 8,192 tokens. That check ends a loop once three whole copies are in, never before token 768, and costs 0.4 to 1.5 microseconds per step. With it on, a 4-bit test build of a 9B model that repeated a 129-token block 94 times stopped at 5,202 tokens instead of 16,916, and /health and the request log give the reason long_cycle. This check can also cut real code: it ends a platformer map with 8 identical empty rows of 60 tiles.

• Quoted tool markup no longer ends the thinking. Pasting a traceback that contained </parameter> made the rest of the model's thinking show up in the answer. Thinking now ends only at </think> or at the start of a real tool call: the request must declare tools, the call must start on its own line outside a code block, and a bare <function=NAME> must name a declared tool. The same rule decides whether a reply that was only thinking gets retried, and a retry that receives a tool result starts its stream clean. Ten streamed 27B requests that quote up to 43 tool tags in one thinking block all kept their thinking in the reasoning pane, and the rule has 117 test cases.

• A made-up tool call in a reply without tools is hidden completely. The hidden part used to end at the first </function>, so the call's closing </tool_call> showed up as text. It now waits for the tag that matches the opening one.

• Gemma 4 streamed replies finish again (#517). A reply without tools was generated and then failed with 'Gemma4ThinkingContentStreamSplitter' object has no attribute 'suppressed_tool_markup_chars'. Every stream splitter now provides the fields the stream reads. A streamed reply now finishes with stop after 418 frames, with the reasoning kept apart from the answer.

• Stop, then send again, keeps the conversation. After Stop, the app gave the conversation a new session id, so the next message reached the engine as a new conversation and processed the whole prompt again (26,294 tokens and 24.5 s to the first token in the report). The app now keeps the session across a Stop, and the engine lets the next request wait up to 30 s (MTPLX_SESSION_CANCEL_HANDOFF_WAIT_S) while the stopped reply releases the session, instead of refusing it. A retry 1.975 s after Stop reused 10,918 of 10,928 prompt tokens, and a full retry reused 6,144 of 6,225 and finished a 9,555-token answer.

• Long sessions keep what they need to resume (#499). When a conversation was larger than its RAM cache allowance, the cache kept a reference to the live state instead of a copy, and that entry dropped the draft head's history. The next turn could not restore, the SSD copy was refused for the same reason, and the whole prompt was processed again on every turn: 570 s to the first token for a 149,649-token prompt in the report. The history is now kept in RAM and in the SSD copy, at about 4 KB per token next to 64 KB per token of model state. An SSD copy saved by an earlier release is processed once more after the update.

• Old conversation states are counted and freed (#456, diagnosed by @peterloron). Those same reference entries recorded 0 bytes, so no memory check could see them, and a turn that could not reuse one built a new cache and left the old one in memory. Memory grew by 3.7 GiB per turn on a 64 GB M4 Max. The entries now report their real size, a session keeps at most one, and eviction and /admin/cache/clear free the memory they hold. /health shows lease_entries and lease_nbytes.

• Saving a session to SSD pauses for a waiting request (#505). On a 64 GB Mac with the 27B at about 104K tokens, the SSD save held the model for more than 500 s while a request waited, and the request timed out at 300 s. The save now works in pieces of at most 32 MiB and checks for waiting work after each one. A deliberately slowed test interrupts it in under 200 ms, and the files on disk are exactly the same. /health reports encode_units, encode_slow_units and encode_longest_unit_s.

• Tool conversations keep their prompt history. With thinking off, the prompt split the empty thinking block in two, so from the second request of a tool session the history no longer matched the saved state. An audit of 2,020 conversation turns found the unexplained differences from the model's own chat template drop from 848 to 0. The predicted reusable history matched the next request in 834 of 834 cases, up from 498. For OpenCode with thinking on, turns that continue the saved stream went from 28 of 36 to 36 of 36.

• A stuck request can no longer hold the fans at maximum (#295 by @El-Patronum). In Smart fan mode, one report had both fans at maximum for about 15 hours behind a request whose client had gone. A request now counts as activity only while the model makes progress. After 180 s without progress (MTPLX_FOREGROUND_STALL_DEADLINE_S) it stops counting, and the fans return to automatic 120 s later. Batched prompt processing now reports its progress too, so a healthy long prompt no longer trips the 300 s stream timeout.

• Plain decoding has one path (#507 by @davidtai). David Tai found that the optional pipelined plain decoder drew its first token from a different random stream than every later token, so one seed gave different text depending on the path, and his fix made them agree. After this release's n-gram change no model offers the pipelined mode anymore, so that path and its MTPLX_AR_PIPELINE switch are removed, and plain decoding always uses the standard loop.

• Draft heads imported from a raw checkpoint work again (#511 by @stooit). Hugging Face checkpoints store the draft head's norm weights centered on zero, and only Forge added back the +1.0. A head added any other way loaded without an error and then accepted about 0 percent of its drafts (the right token sat at a median rank of 247,513 out of 248,320). The loader now applies the same correction as Forge, and a head that is already correct is left exactly as it is.

• Eager BF16 verification rounds the attention gate the same way as compiled verification. This covers the gated attention of the BF16 Qwen3-Next, Qwen 3.5, 3.6 and 3.8 models when verification runs eagerly: image requests, contexts past the compiled route's 32,768 tokens, fallbacks when a buffer grows, warm-restore rounds and single eager passes inside a compiled request. Compiled verification, prompt processing and plain decoding are unchanged, and so are the Float16 and Float32 gates, Bonsai, Gemma 4 and Flash-Next.

• Downloads behind a company proxy explain the certificate error (#495). A CERTIFICATE_VERIFY_FAILED error now names a TLS-inspecting proxy as the likely cause and gives both fixes: set SSL_CERT_FILE, or install the truststore package. With truststore installed, downloads check certificates through the macOS keychain; MTPLX_SYSTEM_TRUST=0 turns that off.

• Forge builds a model that ships as one weights file (#492). A single model.safetensors has no index file, so Forge found no draft head, wrote no mtp.safetensors, and stopped with return_hidden requires an MTP-patched runtime. Forge now reads the file headers the way mtplx inspect does.

• A Qwen model with MiMo in its name keeps the Qwen settings. Xiaomi also publishes Qwen-based models named MiMo, such as MiMo-V2.6-Distill-Qwen-9B. MTPLX chose the model family from the folder name, so these got the MiMo settings: draft depth 1 only and reasoning off. The family now comes from the checkpoint's own architecture, so such a model keeps the Qwen 3.5 settings, depths 1 to 3 and reasoning on.

• The packed-GQA verify counter counts accepted windows too (#506). It counted only the windows it declined, and only while the compiled verifier was being traced, so a healthy engine showed 96 declines and no accepts. /health now counts both.

• /health no longer swallows a cancellation or an interrupt while it reads its optional counters.

The app

• Use MTPLX settings for connected apps. A new switch in the inference settings decides whether MTPLX or the connected client controls the request settings. It is on by default, and the per-chat controls work either way. Pi follows the live reasoning setting, and gets its own choice back when the switch is off. Reading the settings used to take about 1 s and now takes 2 to 10 ms.

• Remove downloaded models from the app (#377 by @PhilipJohnBasile). Removal asks first, removes exactly the folder shown, refuses the model that is being served, and says why when it refuses. Extra model folders stay read-only, and the controls are in all 13 languages. mtplx remove now treats a folder name you type exactly as that folder, even when it matches a public alias, and a --cache-dir you type selects that folder.

• The model list icons line up, and the trash appears on hover. The checkmarks sat 5 to 6 points to the right of the trash cans. Every icon now sits in the same 28-point slot, in left-to-right and right-to-left languages, and a test renders the rows to check where each icon lands. The trash for a downloaded model appears only while the pointer is at the right end of its row or the button has keyboard focus. Right-clicking a row also offers Remove downloaded files.

• Every model folder keeps its own name and row. When a pack declared no name of its own, the list named it after the model it was built from. Two builds of one model, such as a 4-bit and a 6-bit, shared one row, and that row launched whichever build it found first. A downloaded Qwen 3.8 27B Optimized Speed also showed up a second time as Qwen3.8-27B. On the test Mac, seven folders were all named Qwen3.5-4B. Each folder now shows the name its pack declares, or its folder name, and gets its own row.

• Long model names stay readable. A name that does not fit is now shortened in the middle instead of at the end, so the part that tells two builds apart, such as 6-bit or a folder suffix, stays visible. Hovering over a name in the list, or over the model name in the top bar, shows the full name.

• The inference panel no longer pins a 2,048-token prompt chunk. When nothing was saved, the panel showed 2,048 and sent it with every settings change, such as a new temperature. The app then saved it, and every later launch passed --prefill-chunk-tokens 2048, which overrides Flash-Next's own 4,096-token chunk on M5. A cold 40K prompt read at about 978 tok/s with that setting and about 1,400 without it. The panel now sends a chunk size only when you choose one, an Auto option leaves the choice to the engine, and a 2,048 saved by an earlier build goes back to Auto once.

• One launch setup for the app, mtplx start and mtplx serve. The same client on the same model used to get different engine settings depending on how the engine was started. Pi now uses the serial scheduler everywhere (the OpenCode preset measured 51.4 against 36.8 tok/s decode at 8K), the SSD cache size is automatic everywhere, the minimum saved prefix is 512 tokens everywhere, and the app's presets no longer fix the prompt chunk at 2,048 tokens for every model. A test builds the app's launch command for 294 combinations of client, model family and Mac size and compares it with the engine's own table.

• The open inference menu no longer loads the CPU. Live metrics redrew the whole app, including the open menu. With the menu open during inference, the app's CPU use went from 47.9 and 27.4 percent to 2.2 and 3.3 percent, and Bonsai decoded at 48.5 tok/s in both runs instead of 45.5 and 46.9.

• Activity and Cache show every saved prefix. Several prefixes saved from one conversation shared one row, which hid entries the engine still held (19 prefixes across two sessions in the report). Each row is now keyed by its session and token hash.

• Setup uses the runtime and model settings it selected. An app runtime started from inside another MTPLX folder could load that folder's Python package, report the wrong version and fail setup. When tuning is skipped, the pack's own draft depth now applies, so Bonsai starts at depth 1 instead of the general default of 2.

• A failed start leaves a log that mtplx doctor can show (#504). The app used to show one cut-off line. It now writes the engine's last output to ~/.mtplx/logs/last-failed-start.log, with secrets masked, and mtplx doctor includes its last 60 lines while the file is less than 14 days old. A start that you cancel with Stop no longer writes a failed-start report.

• Cancelling a start no longer corrupts the app's memory. The engine supervisor's default health checks were written in a way that let two copies disagree on how much memory they need, so cancelling a start could write past the end of an allocation. Debug builds crashed with "freed pointer was not the last allocation", 3 times in 3 runs. The release build did not crash in a manual check, but the fault was there in every build, and it is now fixed.

• The free-disk check before a download matches the download. The app asked for 2.5 times a model's size in free disk, about 396 GiB for Flash-Next Optimized Quality. It now asks for the bytes still to download plus 5 GiB, the same rule mtplx pull uses.

• Clearer setup error on macOS 26.0 and 26.1. MLX for macOS 26 needs 26.2 or later. On older versions, setup now says so and asks you to update macOS, instead of pointing to the network.

Server, CLI and Forge

• Performance reports explain a slow response. /health, the request log and mtplx doctor --explain now count every time a request leaves the fast path, with the reason. Pi traces include compaction requests and the image context, and the trace report shows how many verify steps ran compiled and what a full round costs. The dashboard computes its draft totals from the per-depth counts when a report has no totals of its own (#490 by @dajiaohuang).

• Request captures hold no conversation content by default (#356 by @PhilipJohnBasile). Capturing is still off unless MTPLX_REQUEST_CAPTURE_DIR is set. When it is on, a record holds the settings, token counts and SHA-256 digests, and no prompt, answer, messages, error text or token ids unless you opt into each one. For a 200,000-token prompt, the default record takes 12 ms and 468 bytes, against 37 ms and 2.0 MB with every token id kept.

• mtplx inspect and mtplx serve describe the new official packs correctly. They used to say "needs contract repair", ask for the pack to be rebuilt with Forge, and mtplx inspect exited with code 3. A pack whose accuracy measurement is not published yet now shows "Official MTPLX pack, qualification pending", runs, and exits with 0. Bonsai 2's measurement is published, so it shows as verified. Flash-Next Optimized Quality stays pending until it has run on a 256 GB Mac.

• A pack that needs a newer MTPLX is refused before any weights are read. When a pack's min_engine_version is newer than the running engine, mtplx serve and mtplx inspect stop with "This model needs MTPLX X or later (you have 2.12.0). Update MTPLX, then try again." No unsafe flag overrides it.

• A server started without mtplx serve uses each model's own sampling settings. python -m mtplx.server.openai sampled Qwen 3.8, Bonsai 2 and Flash-Next at temperature 0.6 while /health reported 1.0. With no sampling flags it now uses 1.0, top-p 0.95 and top-k 20, the settings mtplx serve and the app already use. Flags you pass still win, and Qwen 3.5 and 3.6 stay at 0.6.

• Flash-Next Optimized Quality packs built with Forge carry the Flash-Next sampling settings. The flash-next-optimized-quality recipe now writes temperature 1.0, top-p 0.95 and top-k 20 into mtplx_runtime.json, as the two Flash-Next Speed packs have. The published Optimized Quality pack has them too.

• mtplx tune and Forge leave the fans alone unless asked. Forge's check runs mtplx tune, and tune pinned the fans at maximum even without --max, for about a minute during a 9B build. Tune now pins them only with its new --max flag or --require-max-fans, and Forge passes them only when it gets --max. The app's tuning and mtplx start still ask for maximum fans.

• Forge keeps a draft head's declared settings unless the evidence is clear. Calibration switched a Qwen 3.5 9B head to the wrong setting (pre_norm) based on one prompt and six tokens, where every option accepted nothing. A switch now needs at least 4 prompts, 48 draft rounds per option and a lead of 0.25 accepted tokens per round. Otherwise the declared setting, or the family default, stays, and mtplx_runtime.json records the calibration as inconclusive. The check now samples 8 prompts and 4 windows. On the Qwen 3.5 9B pack it takes 49 s including the model load, and it keeps the pack's setting (1.55 against 1.59 accepted tokens per round, inside the margin).

• The --reasoning-effort help lists each model's levels and default. Qwen 3.8 27B offers xhigh, medium and low, and Bonsai 2 offers xhigh and medium, and both default to medium. Flash-Next defaults to xhigh in chat and to medium in coding agents.

• mtplx inspect reports whether a pack takes images. Packs that keep their vision tower in one weights file with no index, like Prism ML's, are now served with vision.

• A pack can recommend plain decoding (recommended_generation_mode) and still ship and load its draft head. No shipped pack uses it yet.

• If Flash-Next's compiled verifier cannot run on a GPU, the request continues. Its first run is now guarded. On a GPU that refuses the kernel, the compiled path is switched off with one log line and the same round runs eagerly, with the same output.

• The sparse prompt kernel for M1 to M4 checks its numbers before it is used. The app builds this kernel against MLX's macOS 15 release but runs MLX's macOS 26 release, a difference the version check cannot see. Before its first use in a process it now runs one small call on random inputs and compares the result with the pure MLX path. It is used only when the largest difference is within 0.005. Otherwise prompt processing uses the pure MLX path, and /health gives the reason.

• The bundled native kernels load on a fresh Mac. The app's native package now finds its libraries inside the installed package instead of at the build machine's paths, and it is built against MLX's macOS 15 release, so it loads on macOS 15 and later. A fresh app setup with the bundled Python 3.14.5 and MLX 0.32.2 passed its compatibility and GPU checks. The source package no longer includes local profiling scripts.

• Open WebUI setup (#513). The server guide now says to leave Open WebUI's Controls on Default, so the settings MTPLX tuned for the model apply.

• AnyIO 4.14.2 (#512 by @dependabot).

For developers

• New measuring tools, all off by default. MTPLX_QWEN4_PREFILL_PROFILE=1 times each part of a Flash-Next prompt pass. MTPLX_QWEN4_EXPERT_OVERLAP_PROBE=1 counts how many different experts the verify rows use. MTPLX_FIXED_M4_DONATION_PROBE=1 reports when a verify step copies the attention cache instead of reusing it, and it can name every piece of state that moved. MTPLX_EXPERIMENT_DEPTH_CEILING raises the draft depth limit for acceptance tests only. The compiled verifier can also report how its host time splits between gathering rows, preparing inputs, replaying the graph and encoding.

• Two experimental prompt switches for the dense attention band. MTPLX_QSA_DENSE_BAND_SDPA=1 applies Flash-Next's attention mask inside the score matrix multiply below the sparse switch point, with bit-identical output in tests. It stays off until a full-model check on real prompts confirms it.

• Model-tuned settings live in one place per model family, each with its source and its measurement. This changed no behavior. One check now decides whether a Mac's GPU has tensor units for every fast path, and MTPLX_FORCE_GPU_FAMILY_FALLBACK=1 now also covers Flash-Next prompt processing, so an M5 can run MTPLX's M1 to M4 paths.

• New scripts check the prompt the server builds for an agent session against the model's own chat template token by token, and prove the compiled image routes on the real Flash-Next and 27B packs with one command each. Flash-Next's accuracy check now requires completed image replies.

• Release checks. The release script refuses to publish while the notes or README still contain draft wording, runs the model checks after the unit suites finish, and fails when a catalog repository cannot be found. The native package tests now run in CI instead of being skipped.

Validated with the real harnesses

Everything below ran on 2.12.0 builds on an M5 Max with 128 GB, one model loaded at a time. The two suites ran on the release branch. The only code change after them restyles the connected-apps switch to match the Performance mode switch.

Accuracy measurements

Known in this release

• Flash-Next Optimized Quality has not been run on a 256 GB Mac yet, and its speed has not been measured. It is listed second on those Macs.

• Bonsai on a 16 GB Mac gets an 8,192-token window. This was measured by limiting a 128 GB Mac to a 16 GB budget, not on a real 16 GB Mac. OpenCode's system prompt (about 18,700 tokens) and Hermes's 64K minimum do not fit in that window, so agent clients need 18 GB (20,480 tokens) or 24 GB (94,208 tokens).

• On Flash-Next, compiled and eager verification differ by rounding, for images as for text (see the table above). The sampled-distribution test that 2.11.3 ran for text has not been run for images.

• Long answers still slow down as they go, on Bonsai and on the dense 27B. On a 26,290-token Bonsai answer, speed fell from 50.9 to 30.3 tok/s before the new kernels. Most of that is macOS lowering the GPU clock during long decoding, from about 1,450 to about 1,000 MHz on Bonsai and from about 1,450 to about 670 MHz on the 27B within 150 s, while the GPU stays busy and cools down. The Mac was in High Power mode with the fans at maximum. At a fixed clock, the context itself adds about 0.2 ms per round for every 1,000 tokens, about 5 ms by 24,000 tokens. The new kernels make every round cheaper, which helps. The clock itself is set by macOS.

• This release was measured on an M5 Max only. The M1 to M4 code path was rehearsed on the M5 (see the validation table), which checks the paths and the results but not an older GPU's compiler or speed. M1 to M4, M5 Ultra and macOS 27 were not tested, and the macOS 27.2 beta start failure in #504 is not diagnosed yet.

• MLX 0.32.2 for macOS 26 needs macOS 26.2 or later, so MTPLX does not load on macOS 26.0 or 26.1. On those versions, setup says so and asks you to update macOS.

Upgrading

Run brew upgrade mtplx or pip install -U mtplx, or use the app's update check. You don't need to change any settings. The repetition stop is now off; to turn it back on, set MTPLX_REPETITION_STOP=1 in the engine's environment. An existing Flash-Next image conversation is processed again once from its first image, and a long session that an earlier release saved to SSD without the draft head's history is processed once more. Nothing is deleted. The Bonsai 2 and Flash-Next Optimized Quality packs need 2.12.0 or later, and MiMo V2.6 Qwen 9B gets its correct settings only from 2.12.0.

Contributors

• A stuck request can no longer hold the fans at maximum, by @El-Patronum in #295
• Request captures hold no conversation content by default, by @PhilipJohnBasile in #356
• Remove downloaded models from the app, by @PhilipJohnBasile in #377
• Dashboard draft totals from the per-depth counts, by @dajiaohuang in #490
• Memory guards that read the real process footprint, by @ArctifoxNL in #500
• One random stream for the pipelined plain decoder, by @davidtai in #507
• Forge converts Flash-Next source models, by @bpmforge in #508
• Draft heads imported from a raw checkpoint work again, by @stooit in #511
• Image placeholder text stays text, based on @fradav's fix in #519
• AnyIO 4.14.2, by @dependabot in #512

New contributors

• @El-Patronum made their first contribution in #295
• @dajiaohuang made their first contribution in #490
• @bpmforge made their first contribution in #508
• @fradav made their first contribution in #519

Thanks to @adsharma (#515), @shamusj-create (#516), @plkx (#517), @nomishbhardwaj (#456), @peterloron (the #456 diagnosis and #505), @Deesha08 (#499), @r-near (#506), @stevebutler (#495), @Lancelotbronner (#492 and #504) and @akhst7 (#513) for the reports.

Full changelog: https://github.com/youssofal/MTPLX/compare/v2.11.3...v2.12.0