MTPLX 2.10.1

Patch release: faster long-prompt processing on M4 and M5 Macs, working image input for Flash-Next, and fixes for 96 GB and M2/M3 machines.

Prefill

Flash-Next gains a sparse prefill lane, adapting the fused Metal kernels from PR #397 by @maceip. The model already scores which blocks of the context matter for each new token; the lane now feeds those scores straight into a block-sparse FlashAttention kernel instead of building full attention masks.

Measured on an M5 Max with 128 GB against 2.10.0 on the same machine:

• 98k-token prompt: processing time drops 35 percent (175.7 s to 114.5 s), peak memory drops from 91.4 to 83.0 GB.
• 131k-token prompt: 810 tok/s.
• 262,144-token cold prompt: completes in 355 s at 87.4 GB peak. On 2.10.0 the same request climbed to 119 GB and produced nothing (#393, reported by @blackjose007-stack).

The lane turns on automatically where its kernels are supported (M4 and M5 generation GPUs) and engages on prompts past 32k tokens; other machines keep the dense path. MTPLX_QSA_PREFILL=0 turns it off. The kernels also handle YaRN rope scaling for long-rope configurations.

Memory admission understands the lane: serve resolves the model's full 262,144-token window on 128 GB machines and prints the plan, and a request that genuinely does not fit answers HTTP 507 instead of wedging the machine.

Flash-Next image input

The Flash-Next packs have always shipped their vision weights, but the runtime refused to serve images. 2.10.1 adds the family's multimodal position encoding, so image input now works in app chat, over the API, and in Pi (#328, reported by @nmqanh). Pi model configs written by an older MTPLX are upgraded in place, and user edits survive.

96 GB Macs and M2/M3 GPUs

Two fixes from @JordiPosthumus's report (#400):

• The preload memory check used reserve constants tuned on 128 GB machines and refused Flash-Next on 96 GB Macs that actually fit it. The reserve now scales with the machine, so the packs load on 96 GB. Machines with 128 GB or more are unchanged.
• Three Metal kernels requested 1,024-thread threadgroups. M2 and M3 generation GPUs cap these kernels at 896 threads and crashed at launch. The runtime now probes the GPU once at startup and takes the standard path on machines that reject the fused kernels.

Empty answers

With tools declared, which agent clients always do and app chat with web search does too, a first turn that ended entirely inside the model's reasoning channel came back as an empty message. The server now continues such a turn until the visible answer exists, and if that continuation itself fails it returns what it has instead of an error.

Also fixed

• Greedy Turbo on the 27B routes verification through stock kernels at temperature 0, restoring token-exact agreement with plain decode.
• The app verifies that its Python runtime actually imports before trusting it, and rebuilds it from the bundled wheel when it does not. This ends the crash loop where reinstalling the app never helped because the broken runtime folder survived the reinstall.
• Speed receipts attributed session-restore time to decode, understating decode speed on warm long-context turns. Attribution is now correct; generation behavior is unchanged.