Ready for the shape of Qwen 3.8
The MTP runtime no longer assumes a single draft layer. It now honors
mtp_num_hidden_layers = N, and a checkpoint with a new model_type can load
through the architecture class it declares instead of waiting for a hard-coded
name to ship in MTPLX.
Because Qwen 3.8 weights were not public during this release gate, we did not
pretend to benchmark them. We instead ran a synthetic qwen3_8 alias drill
through model loading, generation, CLI discovery, and the app catalog. This
removes the known integration blockers and leaves real-weight validation as
the first task when the checkpoint appears.
Coding-agent tool calls survive real work
The bridge used to confuse a large JSON write body with hidden raw tool markup and abort an otherwise valid call. It now distinguishes the dialects correctly and repairs common near-miss argument keys at the protocol boundary.
The release was exercised from the user's seat, not only with unit tests:
- OpenCode CLI completed a fresh multi-file parser change with ten real tool actions and all 31 generated tests passing.
- OpenCode Desktop connected to the same candidate daemon and completed a visible prompt at 42.2 tok/s.
- Pi and Hermes completed native file-read tool loops.
- OpenAI and Anthropic streaming endpoints emitted structured tool calls with raw markup suppressed.
App and CLI launches now use the same coding-agent engine settings, and
start / quickstart --dry-run show the profile that will actually run.
HY3 becomes a first-class target
HY3 now has an MTP-capable model implementation, official serving defaults, model discovery, suffixed think-tag handling, and native OpenCode tool calls. AR-only exports are detected and stay on the safe AR path instead of touching an uninitialized draft head.
Experimental: DeepSeek V4 gets a real fast path
David Tai's DeepSeek V4 work adds shape-specialized output-LoRA routes, adaptive speculative width, exact M3 attention projection, sinkhorn and attention-island kernels, and compiled post-attention verifier islands.
We downloaded the exact model used by the contribution
(mlx-community/DeepSeek-V4-Flash-2bit-DQ) and combined it with the official
DeepSeek MTP shard. It loads on this 128 GB Mac. Under verified maximum fans,
the candidate measured about 31 AR tok/s and 36 MTP tok/s, compared with about
4 AR and 6 MTP tok/s on the conservative path. Memory peaked around 102 GB.
The fast routes are opt-in. A real OpenCode edit proved that the engine and native tool call work, but the model then over-generated instead of completing the edit. That is why this release calls the lane experimental: the speed work is real, while broader agent-quality calibration is still required.
Thank you to David Tai (@davidtai) for the DeepSeek work in #223. His original commits and authorship are preserved in the release history.
Experimental: Laguna and GDN kernels
Also from David Tai:
-
#222 ports the Laguna S-2.1 decode and prefill lanes to
mlx.fast, including a size-gated prefill MoE-combine route. Unsupported shapes fail loudly back to the established implementation. - #209 adds an env-gated GDN headquarter execution layout for verify tape capture, with bit-exact tests and a loud fallback contract.
Both remain explicit opt-ins. They expand the performance frontier without quietly changing behavior for an existing installation.
Regression gate
The Qwen V2 baseline and candidate were run on separate installs in an alternating A/B sequence under verified maximum fans. The repeat pair put the candidate slightly ahead on decode and both prefill sizes; aggregate variation stayed within the heat/order noise band, with equal-or-lower candidate memory. There is no consistent decode, prefill, or memory regression.
The candidate also passed the complete Python suite (about 3,179 tests, zero failures), 547 Swift tests, signed-app build checks, and real visible app QA.
Upgrade
- App: Sparkle offers 2.5.0 (build 25000); the app provisions its runtime from the bundled wheel on the next Start.
- pip:
pip install -U mtplx - Homebrew:
brew upgrade mtplx
No speculative-depth, cache, sampler, or speed default changed. Experimental kernel lanes fail closed to their established implementations.