MTPLX 2.11.2

A correctness release for every Mac that is not an M5, seven session-bank and memory fixes for long agent sessions, native vision for Flash-Next in Hermes, OpenCode and Pi, a verifier-depth fix that restores Flash-Next's compiled verify route in agent turns, and a set of app and CLI repairs. If you saw Qwen 3.8 27B produce unrelated text, loops or another language on 2.11.1 once a conversation passed a few thousand tokens, this release is for you.

27B on M1-M4: the flash-decoding verify route is now gated by hardware (#459, #464, #467, #461)

2.11 turned on a flash-decoding verify route for the 27B models in the turbo profile. It uses the tensor units of the M5 GPU, and it shipped without a hardware gate. It engages once a conversation's KV buffer reaches 8,192 tokens, which on a Mac without those units means wrong attention from that point on: reasoning about a different codebase, imaginary tasks, mixed languages, tool calls that never happen. On macOS 15 the same kernel failed to build and the request failed with a Metal library error. Reporters on an M1 Max, an M1 Ultra, an M3 Ultra, an M4 and an M4 Max hit it through OpenCode, Pi, Hermes and the app's own chat; one of them bisected it to the route with MTPLX_NAX_FLASH_ROUTE=0.

The route now runs only on an M5-class GPU on macOS 26.2 or newer. Everywhere else the verify path is the packed-GQA kernel 2.10.2 shipped, validated at startup by the kernel self-check. /health says which one you have: degradation.nax.available is false and nax_flash_bail_counts.gpu_family_or_os counts the refusals on an M1-M4 Mac; on an M5, degradation.nax.flash_dispatch_counters counts the route's dispatches.

Rehearsed here on an M5 Max with the reporters' shape (a 14,000-token and a 32,000-token git diff followed by "summarize the changes in three bullet points", Optimized-Speed pack, temperature 0, fans at maximum): with the GPU family forced to the M1-M4 path both answers are correct summaries of the diff, the NAX lanes show as skipped in the self-check and the packed kernel as ok; with the native route both answers are the same summaries and the route counts 66 dispatches with no bails.

The M5 decode gain from this route is unchanged. On M1-M4 the verify path is the same packed kernel 2.10.2 shipped; this build has not been measured on an M1-M4 Mac, so no speed figure is claimed for those chips here, and the 2.11 release notes' 88k row is an M5 number and is now labelled as such. Item 3 of #455 (incoherent output with MTP on, through Pi on an M3 Ultra) matches this route's engagement point, since Pi's system prompt and tools put every request past 8,192 tokens, but it has not been confirmed from that machine; item 1 of #455, AR decode at 15-23 tok/s where 2.10 gave more, is not explained by this change and stays open. An M1-M4 receipt on #459 or #455 is wanted and welcome.

AR-only sessions restore again: Hermes and Pi with --no-load-mtp (#465)

A target-only AR runtime (--no-load-mtp) banked each turn's prefix under one history policy while every lookup asked for another, so the bank refused its own entries with policy_mismatch and the whole prompt re-prefilled on every top-level turn: 14,500 tokens, about two minutes per turn on the reporter's M1 Max. One policy per runtime is now derived in one place and used by every store, lookup and the cache fingerprint. MTP runtimes are unchanged. The "90 to 116 s stall after idle" in #455 runs the same configuration on an M3 Ultra and is the same mechanism on paper; it is not yet confirmed from that machine.

Memory on 128 GB: refuse before prefill instead of swapping (#450, #447)

The pre-prefill memory guard admitted a 136k-token prompt whose projection stayed 2.3 GB over the engine's limit after clearing the allocator cache; nothing downstream stops such a request safely, and the reporter's Mac kernel-panicked four times before any error could be returned. The guard now re-projects after every reclamation step (allocator cache, superseded and idle session snapshots, sibling chain snapshots from #447) and, if the projection still crosses the hard limit, answers with a structured 507 before prefill that names the projection, the limit and the uncached token count. The engine stays up with every resident session. Between the 0.97 warning line and the limit nothing changes, and --allow-swap keeps the operator's explicit choice.

Also from the #447 work already on main: the guard asks the live sessions and the bank for the reusable prefix the restore will actually achieve before projecting, walks the bank's chain snapshots before giving up, and reclaims allocator storage before evicting useful snapshots.

Session bank

• A deep conversation keeps its session at its turn boundaries (#446). A live session's committed stream carries the reasoning it streamed, and clients resend the history without it, so from the second turn on every request's raw shared prefix with its session ends where the first turn started generating. The resolver accepted that match only as a fraction of the new prompt, with a floor of 25 %: 22,437 shared tokens passed at 89k and failed at 112k, the request minted a new anonymous session and block-restored 2,048 tokens of the first turn's snapshot, and a turn that had taken 24 s took 105 s. That is the reporter's 14,336 and 18,432-token remnants at turn 4 or 5 in five chains of five, and the same line crossed one chain in three on 2.10.x. Sessions now record the prompt length of every turn they generated from, and a shared prefix on one of those boundaries keeps the session whatever fraction of the prompt it is; edited histories keep the fraction rule. The request log's prefix diagnostic names the rule that kept it (reuse_rule: turn_boundary).
• Later conversations are admitted again (#454). After a restart only the first conversation was banked; every later distinct conversation re-prefilled at 0 % cache on each turn, about nine times slower. The background-task heuristic (a short-answer request with a different system prompt) classified every conversation-continuing short turn as a title job and served it sessionless. Only the task shape (a system prompt and a single user turn) infers a background task now. Reproduced with the reporter's script on the 27B Quality pack: sessions two to four went from 0 % to 100 % cached on their third turn.
• SSD prefix restore reads only the needed prefix and slices the committed MTP history to it (PR #444 by @softpudding).
• Fixed verifier capacity is renewed for adaptive depths and copy windows, shared verifier programs are released when a model unloads, and completed request banks are released while the shared programs stay.

Flash-Next: your agent's screenshots reach the model, and images keep sparse attention

• Hermes, OpenCode and Pi now advertise image input when the installed pack has a vision tower. All three registrations said text-only, so Hermes sent every screenshot through its Analyze Image tool as a separate request to the same MTPLX model (the auxiliary description path: the main conversation never saw the pixels, and a cancelled analysis kept generating for 35 s), and OpenCode and Pi could not attach an image at all. The app and mtplx read the pack's config.json and weight index (vision_config plus a vision_tower. or model.visual. weight), never the model name, and write supports_vision into the Hermes profile, modalities.input: ["text", "image"] into OpenCode's provider entry and input: ["text", "image"] into Pi's models file, upgrading an existing text-only entry. /v1/models carries supports_vision and modalities too. The 27B packs have no tower and stay text-only.
• Multimodal requests keep QSA sparse attention. The engine ran every image-bearing request through dense causal attention, on the belief that the reference implementation serves multimodal that way. It does not: Qwen's reference applies the sparse indexer to multimodal input and rotates the indexer's queries and its pooled block-start keys with the image-aware M-RoPE positions. The engine now does the same, checked against an independent NumPy position oracle. On this M5 Max an image-bearing Hermes conversation at 54,000 tokens ran at about 19 tok/s on the dense path with a 128 GiB allocator peak; the corrected path completes turns with up to fifteen screenshots at 43,000-74,000 tokens with a 92 GiB peak under the 96 GiB ceiling. Vision cache entries written by the dense path carry a different policy identity and are not reused. MTPLX_QWEN4_VISION_QSA=0 restores the dense path for diagnosis only.
• Known limit: image-bearing turns decode on the eager verifier. The compiled fixed-M4 verify route is not yet used while images are in the context (its indexer graph has no proof for the image-aware positions), so those turns ran at 33-50 tok/s here against 54-61 tok/s for the text-only turns of the same session. That route is the next Flash-Next item.
• A warm image turn is admitted as warm. The pre-prefill memory guard compared raw image-pad tokens against snapshots keyed by image content, so a warm turn after the first screenshot projected as a full miss, shed the snapshot it needed and could refuse with a 507. Admission now asks the same content-keyed question as restore, an anonymous image turn rejoins the session that owns the matching image snapshot (session_source: vision_bank_prefix), and different pixels can never adopt an older image's state.

Long agent sessions on Flash-Next: three more re-prefill causes

Found by rehearsing a Hermes Desktop coding session (a Pygame Flappy Bird, 16,000-token system prompt, screenshots of the running game) on the 2011010 candidate: the tool turn after the game was written reused 2,048 of 32,955 tokens although the bank held an exact 15,688-token prefix, and decode sat at 41-47 tok/s.

• A longer bank entry no longer erases a shorter exact prefix it cannot restore. Flash-Next's recurrent state carries checkpoints at 2,048-token boundaries; the exact end of a prompt is not one of them. The generation-final entry (prompt plus answer) superseded the exact prompt entry on token containment alone, and the next tool turn restored from the nearest checkpoint, 2,048 tokens, then re-prefilled 31,000. A longer entry now replaces a shorter one only when it carries every restore point the shorter one supplies; the per-session budget still bounds the bank.
• Hermes sends its reasoning back. Hermes stores reasoning locally but strips it from the next request unless reasoning_echo is on, so the history it sent never matched the committed stream. The profile enables it; the next tool turn reused 15,754 of 16,005 tokens.
• The cache producer strips the tool-call preamble the way Hermes' wire does. Hermes suppresses the preamble text before a tool call; the committed snapshot, canonicalization and the next-turn comparison now normalize the same way, and an intentionally empty visible answer stays empty instead of being replaced by raw generation. That mismatch rejected the committed reasoning turn and re-prefilled about 23,000 tokens after the game was written.
• A response's own reasoning survives an older interrupted turn. When an older turn's history had been rewritten, the substitution walk stopped before the current response and its reasoning was left out of its snapshot; the current turn is now matched on its own, and the full byte comparison still decides reuse.
• Hermes' own three-screenshot window. Hermes keeps only the three newest tool images in the outgoing history, so a fourth screenshot rewrites the earlier prompt and the engine must recompute it. The MTPLX profile now requests compression.tool_image_retention: until_compaction; that key needs a Hermes change that is not upstream yet (the patch is kept with this release's evidence, Hermes commit 8cf78e7386), and current Hermes versions ignore it. Until it lands, a conversation with more than three screenshots re-prefills from the first retired image.

Flash-Next verifier depth: measured costs, not assumed ones

The expected-value depth policy priced a depth-2 draft as cheaper than depth 3. On Flash-Next depth 2 runs the eager verifier while depth 3 runs the compiled fixed-M4 route, so the policy kept stopping at depth 2 and only about 11 % of the verifier calls in the rehearsal's 15,000-token coding turn used the compiled route. With the compiled verifier engaged, the policy now measures draft and verify cost per depth on the running machine, leaves one-time compilation out of the recurring estimate, and re-probes both depths so a first expensive trace cannot pin it. The same prompt replayed at 56-59 tok/s with 90-96 % of cycles on the compiled route, against 41-47 before; diagnostic figures from a machine in use, not a quiet A/B. MTPLX_ADAPTIVE_VERIFY_COST_FEEDBACK=0 restores the configured prior. The 27B packs do not use this path and are unchanged.

CLI

• mtplx run, mtplx ask and one-shot mtplx chat work on Flash-Next (#463). The one-shot path applied only the profile defaults, never the family lanes serve stamps at boot, and crashed in the legacy capture walker. It now resolves the same runtime contract as serve and tune; the legacy capture commit declines family-native captures instead of raising. Receipt here: mtplx run --model Qwen3.8-Flash-Next-MTPLX-Optimized-Speed --max-tokens 32 "hi" answers at 52 tok/s with MTP depth 3 where 2.11.1 raised AttributeError: 'DecoderLayer' object has no attribute 'input_layernorm'.
• The interactive terminal chat resolves that contract too. The REPL (mtplx chat with no prompt, mtplx start cli) still applied only the profile defaults; it now runs the same environment as serve before the model loads.
• The in-process generators use the family's verifier. mtplx run and the terminal chat hardcoded the legacy capture-commit verifier over the batched verifier the Flash-Next contract selects; on Flash-Next a two-turn terminal session degenerated into repetition and a later run ended in a Metal GPU address fault. The resolved strategy and core now reach generation, as they always did in serve. The same prompts and seeds complete on the supported verifier and the generated programs pass their own assertions.
• The terminal chat keeps reasoning in its own channel. It stored thought</think>answer as the assistant's content, so Qwen 3.8's template nested that after an empty thinking block and the next turn's history was malformed. Reasoning and content are stored separately, as the server does.
• --expect-python validates the final answer. It compiled the reasoning and the Markdown fence as Python and reported a valid program as a failure. It now splits off the reasoning with the model's codec, unwraps one enclosing fence, and still rejects malformed or missing programs.

Agent harnesses

• The app's Hermes profile keeps a configured reasoning effort on its own .env line. With a reasoning effort set in Performance, the profile's .env carried TERMINAL_CWD="…"HERMES_MTPLX_REASONING_EFFORT="xhigh" as one statement; Hermes rejected it on every launch ("could not parse statement") and silently lost both the working directory and the effort. Found by this release's harness run through Hermes v0.21.0.
• The Hermes profile no longer stamps terminal.backend: local (#460), and a root sandbox choice reaches the profile. MTPLX wrote and re-wrote that key into ~/.hermes/profiles/mtplx/config.yaml on every launch, overriding a Docker sandbox the user had configured. The key is Hermes' own now, and the merge preserves whatever the user set. Because Hermes profiles do not inherit the root config, a profile that has no terminal.backend of its own now receives the terminal section of ~/.hermes/config.yaml, so a Docker backend configured there applies to the MTPLX profile as well; an explicit backend in the profile always wins, and nothing else from the root config is copied.
• The stream stall watchdog is a flag and a setting (#448). --stream-stall-deadline-s on mtplx serve (0 disables) and Performance › Advanced › Stall watchdog in the app, in all twelve languages; the Flash-Next sparse-prefill loops tick the owner heartbeat so a healthy long page-in is not mistaken for a stall.
• mtplx trace reads Hermes sessions: --hermes-db ~/.hermes/profiles/mtplx/state.db (with --hermes-log for the API token counts) joins each Hermes turn to its engine receipt by input and output token counts and the completion clock; ambiguous joins are left unmatched rather than guessed. Missing per-second samples no longer draw as zero and observation gaps no longer connect in the trace charts.
• Traces are inspectable across harnesses: exact Pi joins, tool results retained, prefix diagnostics kept with their request, verifier route costs exposed, new tool content distinguished from reduced prefix reuse, and an MTP on/off verdict in the trace economics: mtp_pays compares the tokens a run delivered per second with the matched AR rate you supply, and the break-even acceptance is reported under the run's observed cycle cost, non-draft output and depth mix, for fixed and adaptive depth alike. Proposal cycles are counted from the first draft position, not from verifier calls.

App

• The setup wizard measures free space on the volume that holds your models (#466). With ~/.mtplx/models on an external drive (a symlink, or MTPLX_MODEL_DIR), the wizard read the home volume, refused every catalog model as "insufficient space" and disabled Next. The download step and Forge share the fix.
• Installation survives a failing native-wheel selector. If the optional step that picks the native sparse-prefill wheel for your Mac fails, the app installs the bundled pure wheel and records that choice instead of stopping the installation; installation health and the wheel fingerprint checks are unchanged.
• A foreign-looking occupant of the daemon port is re-probed for five seconds before the app moves ports (#409), so a daemon still settling is not abandoned for a new port.
• The prefill gauge and the Avg Prefill card measure the same work. The card averaged completed-request rates, so a 100-token cached follow-up counted like a 15,000-token prompt and setup time counted as prefill: it read 422 while the prompt processed at 1,040 tok/s. The gauge preferred a cumulative rate that included the setup before the first chunk, 296 tok/s for a chunk that ran at 1,613. Both now use measured chunk work: the gauge shows the current chunk's tokens over its compute time and no longer discards a rate above its scale; the card averages the last 100 chunks and shows their peak, recorded once per chunk on the server so polling cannot double-count.
• Adaptive depth is a switch under Draft depth. It shows the daemon's live policy. The app's Pi and Hermes launches have always run the engine's expected-value depth policy (it may stop a draft short of the chosen depth when the next token is unlikely to be accepted); chat and the other launch targets draft to the chosen depth every cycle. Flipping the switch applies live like the depth slider itself and the choice is remembered across launches for every target (adaptive_depth in the app settings; the daemon then receives an explicit --adaptive-policy). The switch is hidden for a family that owns its own draft policy. /v1/mtplx/settings accepts adaptive_policy and reports adaptive_depth_supported.
• The hero gauge no longer flickers through an agent's tool loop. Each turn's prefill morphed the dial to its prefill face and back, and a tool turn re-prefills a few hundred cached-suffix tokens in well under a second, faster than the morph itself; nothing on the dial could be read. The dial now morphs only for prefill work of at least 1,024 new tokens (a cold prompt, a file read, an edited history), judged from the resolved session's known prefix on the first frame and from the measured cache split after; short suffix prefills keep the decode face, which holds the last decode reading exactly as it does between requests. No timer and no hold: a morph happens at once or not at all.
• The context tile grows with the answer, the verify waterfall shows the first live request and receives the per-second verifier split, and the Live tab's decode reading is the engine's own decode time.
• Latency receipts are complete. Time to first token now includes admission and pending-history waits (a 39 s wait for a previous turn's history rebuild had vanished from the receipt), and decode tok/s uses the generator's decode time instead of re-deriving it from prefill compute, which charged the prompt setup to decode a second time. ttft_s in the request log is therefore larger than 2.11.1's for a queued request, by exactly the wait.
• The app test suite leaves your settings alone. The daemon-supervisor tests used the real settings file and their lifecycle callbacks overwrote the saved model and onboarding choices when the full suite ran; they use private settings paths now. The Python suite's synthetic requests no longer enter the user's request log or flight recorder.
• Composer view lookup isolated to the main actor in the app tests (PR #372 by @PhilipJohnBasile).

Engine and tooling on main since 2.11.1

• Forge extracts MTP heads stored outside the mtp. prefix (PR #442 by @stooit, Stuart Rowlands): GLM-4 MoE, GLM-5.3-Flash, DeepSeek-V3.2 and MiMo layouts; MiMo's output head bound instead of random; the MiMo family reaches tune, and forge takes verification depths from the tune policy.
• Flash-Next tuning uses the same family contract as serving (PR #457 by @stooit), with real draft-cycle means and explicit budgets.
• Sparse prefill packaged for the bundled Python with a compatible fallback (#423 by @humanrouter), validated through the real app installer.
• --compare-static fixed-depth baselines for mtp-adaptive (PR #276 by @rinaldofesta).
• The sparse-prefill native wheel is signed for notarization. Its QSA kernel library and extension module carry the Developer ID, hardened runtime and a secure timestamp before they enter the runtime wheel, and the release script verifies every Mach-O in that wheel before the app is submitted.
• The expected-value depth policy measures conditional acceptance correctly; prefill pipeline resolution stays out of decode and ineligible chunks; the memory budget and allocator reclamation messages say what they mean.

Experimental and off by default: interleaved n-gram rows (#449)

David Tai showed in #449 that Flash-Next's n-gram table cannot be compressed or replaced by a function, and that a cold row costs three separate page-ins because its weights, scales and biases live in three regions of the file. 2.11.2 ships his interleaved layout as an opt-in derived cache: python -m mtplx.ngram_row_layout <ngram-table.safetensors> --out <cache> converts the table with a bit-for-bit check of every row, and MTPLX_NGRAM_ROW_FILE=<cache> serves from it; the model pack itself is untouched and the cache is bound to that exact file. It is not enabled by default and carries no speed claim yet: the cold-versus-warm measurement on a Mac that cannot keep the table resident is the next step, together with seeding the hot-row cache at load. Details in docs/diagnostics/ngram-row-cache.md.

Validated with the real harnesses

Before packaging, the candidate served three coding agents on this M5 Max (Qwen 3.8 Flash-Next Optimized-Speed, fans at maximum), each in a fresh project folder, with the session traces read back through mtplx trace:

• OpenCode CLI 1.18.29. Flappy Bird in one file then a follow-up edit, and a second project, an endless runner, over two invocations: 10 turns, 97.2 % of prompt tokens served from the session bank, and the continuation re-prefilled 2,499 of 54,077 tokens. The first project's continuation re-prefilled 25k tokens because OpenCode's system prompt changes once a file exists in the project, and the trace names that in one line (REDUCED PREFIX REUSE). That client-side churn is the next structural item, not a 2.11.2 change.
• Pi 0.85.1. A terminal Snake game with tests, then a follow-up edit, run through a harness runner that records the real exit code: both processes exited on their own, 30 model responses, 98.0 % of prompt tokens served from the bank, and the generated tests pass.
• Hermes v0.21.0. A todo CLI with pytest tests: 18 tests written and passing after the agent fixed its own seven failures, through the profile MTPLX writes for Hermes.
• Hermes Desktop v0.21.0 with the image-retention patch, the exact Flappy Bird session that exposed the defects above. The first 15,143-token request decoded at 58.7 tok/s and the text-only turns at 54-61 tok/s; fifteen screenshots went through the model's own vision encoder in the main conversation, later turns reused 46,000-53,000 tokens instead of restarting, peak allocation stayed at 92 GiB, and the game passed an independent 900-frame run (scoring, death, restart). Turns with images in the context decoded at 33-50 tok/s (eager verifier, see above).
• OpenCode and Pi with an image attached: a fix-plus-tests task with the game's screenshot and a follow-up in each client, all four processes exited 0, each client sent the image natively, and the generated functions pass independent assertions. Pi's follow-up stayed warm; OpenCode's follow-up ran in a new CLI process and started cold by design.
• Four pillar gates and the 40,000-token agent-session gate passed on the installed candidate: image-cache correctness (a warm turn after an image reuses 15,716 of 15,985 tokens; a different image is refused), the allocator ceiling, long-output stability (three 6,000-token answers, last-to-first token-rate ratio 1.008) and uncapped streaming cadence; warm agent turns at 0.5-1.1 s to first token with no postcommit waits.
• The reporters' cases, 2.11.1 against this build. The #446 chain (five turns of 17,000 words): turn five keeps 89,549 cached tokens where 2.11.1 kept 2,048. The #465 case (AR-only, 45 s gaps): the follow-ups restore 12,447 and 12,482 tokens where 2.11.1 restored none.

Throughput figures from these runs are not quoted: part of the validation ran while the machine was in other use.

Upgrading

pip install --upgrade mtplx, brew upgrade mtplx, or the app's Check for Updates. The 2.11.1 workaround MTPLX_NAX_FLASH_ROUTE=0 is no longer needed on M1-M4 Macs and can be removed. The Hermes, OpenCode and Pi registrations are rewritten on the next launch from the app or mtplx start, which is when a Flash-Next pack gains image input in those clients. MTPLX_QWEN4_VISION_QSA and MTPLX_ADAPTIVE_VERIFY_COST_FEEDBACK are diagnostic switches and should stay at their defaults.