What is the fastest way to run Qwen 3.5 4B on a Mac?

MTPLX, which runs a calibrated native-MTP draft head as an exact speculative decoder. On an M5 Max with fans at max, MTPLX 2.2.0, the card's deterministic suite (18 July 2026), the Optimized Speed pack decodes 227.8 tok/s at depth 3 against a 133.6 tok/s plain autoregressive baseline, 1.71x, with first-position acceptance 0.95. Optimized Quality, with its own calibrated draft head, runs 191.7 tok/s at depth 3 on the same suite, 2.19x over its plain baseline. It is the fastest model MTPLX ships.

How much RAM does Qwen 3.5 4B need on a Mac?

Optimized Speed is 2.47 GB on disk with about 2.9 GiB peak at load and runs on any Apple Silicon Mac with 8 GB or more of unified memory. The MTPLX README states that 16 GB runs the 4B and 9B models comfortably.

Run Qwen 3.5 4B on an 8 GB Mac with MTPLX: speeds, install

Q: Why was the Qwen 3.5 4B pack rebuilt in July 2026?

Earlier revisions shipped a defective MTP sidecar: the RMSNorm weights were stored in the raw zero-centered convention and never restored, so the draft head proposed garbage and MTP made the model slower than plain decoding (issue #176). The current revision is rebuilt from source with the fixed forge, and MTPLX 2.2.0 and later detect and heal the old sidecar at load, so existing downloads recover without re-downloading.

Two packs

Both packs carry a calibrated native-MTP draft head and run the same exact speculative path. Optimized Speed is the 4-bit build and the fastest model MTPLX ships; Optimized Quality posts a larger speculative multiplier at a lower absolute speed on the same suite.

Pack	Body	Download	Peak at load	Pick it for
Optimized Speed	4-bit, 64-weight groups, calibrated native-MTP draft head	2.47 GB	~2.9 GiB	Top speed. Any Apple Silicon Mac with 8 GB or more.
Optimized Quality	Calibrated draft head			2.19x over its plain baseline on the 2.2.0 suite (see speeds).

The engine reads the tuned depth from mtplx_runtime.json; no manual configuration is needed. The served model id for Optimized Speed is mtplx-qwen35-4b-optimized-speed.

Measured speeds

M5 Max, fans at max, MTPLX 2.2.0, the card's deterministic suite, 18 July 2026. "Plain AR" is the same pack decoded without its draft head.

Pack	Plain AR	MTP depth 3	Multiplier	Acceptance
Optimized Speed (rebuilt, 2.5 GB)	133.6 tok/s	227.8 tok/s	1.71x	0.95 first position
Optimized Quality, calibrated draft head		191.7 tok/s	2.19x

Optimized Quality's multiplier is larger and its absolute speed lower on the same suite; Optimized Speed wins on tok/s. Auto-tune repeats this measurement on your own Mac during onboarding: it runs the model itself at each draft depth with fans pinned, keeps plain autoregressive decoding as the baseline, and saves a depth only if it beats it.

RAM and Macs

8 GB or more: Optimized Speed runs on any Apple Silicon Mac with 8 GB or more of unified memory, at about 2.9 GiB peak at load.
16 GB: the README states that 16 GB of memory runs the 4B and 9B models comfortably. Qwen 3.5 9B is the stronger option at this tier.
32 GB or more: the default becomes Qwen 3.8 27B Optimized Speed; the app checks your Mac before recommending anything.
Memory governor: the serve banner prints engine budget, weights, resolved context window and session bank (2.10.0); requests that cannot fit are refused up front with HTTP 507 (2.10.2).

Install

Mac app: download the DMG, pick "Qwen 3.5 4B Optimized Speed". The app downloads the pack, sets up its engine, and measures your machine to pick the fastest decoding depth.

Command line:

brew install youssofal/mtplx/mtplx
mtplx serve --model Youssofal/Qwen3.5-4B-MTPLX-Optimized-Speed

For Optimized Quality, pass --model Youssofal/Qwen3.5-4B-MTPLX-Optimized-Quality. Then point OpenCode, Pi, Claude Code, Cline, Cursor or anything that speaks the OpenAI or Anthropic API at http://127.0.0.1:8000; mtplx connect claude-code and mtplx connect opencode print the exact client config, and setup pages for each client are in the docs. The Sustained profile is the default for this pack.

How it is built

Optimized Speed is Qwen3.5-4B at 4-bit with 64-weight groups plus a calibrated native-MTP draft head. Activations stay bf16 on every pass (fp16 on M1 and M2); the speculative path runs the same model as the autoregressive path. The tuned depth ships in mtplx_runtime.json.

July 2026 rebuild. Earlier revisions of the repo shipped a defective MTP sidecar: the RMSNorm weights were stored in the raw zero-centered convention and never restored, so the draft head proposed garbage and MTP made the model slower than plain decoding (issue #176, reported by lBroth). The current revision is rebuilt from source with the fixed forge. MTPLX 2.2.0 and later also detect and heal the old sidecar at load, so existing downloads recover without re-downloading.

Exactness

Speculation in MTPLX is exact. Drafts from the MTP head are accepted with the probability-ratio rule and rejected drafts are resampled from the residual, so what you sample is what the model would have sampled without speculation, at any temperature. The draft sampler is a speed knob only. The quantization is the one approximation.

← Qwen 3.5 9B Gemma 4 →