What is the fastest way to run Qwen 3.5 9B on a Mac?

MTPLX, which runs the model's own multi-token-prediction head as an exact speculative decoder. On an M5 Max, MTPLX 2.0.1 (7 July 2026) decodes the Optimized Speed pack at 112.5 tok/s at short context and 99.7 tok/s at 8k context, up from 82.9 and 61.6 on 2.0.0, with the 6-bit verify kernels. Every run is sampled; the pack's shipped sampler is temperature 0.6, top-p 0.95, top-k 20.

How much RAM does Qwen 3.5 9B need on a Mac?

16 GB of unified memory runs the 4B and 9B models comfortably, per the MTPLX README. It is the pack MTPLX offers on 16 GB Macs; on 32 GB or more the default is Qwen 3.8 27B. M1 and M2 Macs get the FP16 sibling with the same weights.

Does speculative decoding change Qwen 3.5 9B's output?

No. Drafts from the MTP head are accepted with the probability-ratio rule and rejected drafts are resampled from the residual, so the output follows the model's own distribution at any temperature. The 6-bit quantization of the body is the only approximation.

Run Qwen 3.5 9B on a 16 GB Mac with MTPLX: speeds, install

The pack

Optimized Speed is the 9B speed checkpoint: a 6-bit MLX body with bf16 MTP heads, tuned as the stronger small-Mac option above the 4B release. MTPLX uses the model's own MTP heads to generate draft tokens, then verifies those tokens with the main model; when the draft heads are well matched you get higher throughput without running a separate drafter model. MTPLX reads mtplx_runtime.json and selects the measured defaults automatically. M1 and M2 Macs get the FP16 sibling (same weights, native precision for chips without bf16). The served model id is mtplx-qwen35-9b-optimized-speed.

Measured speeds

M5 Max, Optimized Speed pack, MTPLX 2.0.1 against 2.0.0, release measurements published 7 July 2026. The 2.0.1 change was 6-bit verify kernels for this pack's body. Every run is sampled; MTPLX has no greedy path. The pack's shipped sampler is temperature 0.6, top-p 0.95, top-k 20, the runtime contract in mtplx_runtime.json.

Lane	MTPLX 2.0.0	MTPLX 2.0.1
Decode at short context	82.9 tok/s	112.5 tok/s
Decode at 8k context	61.6 tok/s	99.7 tok/s
Prefill of a 2k prompt		up 43 percent

Decode rose 33 to 62 percent across the two lanes. Auto-tune measures the depth that wins on your own Mac during onboarding: it runs the model itself at each draft depth with fans pinned, keeps plain autoregressive decoding as the baseline, and saves a depth only if it beats it.

RAM and Macs

16 GB: the README states that 16 GB of memory runs the 4B and 9B models comfortably. This is the tier the 9B is built for.
32 GB or more: the default becomes Qwen 3.8 27B Optimized Speed; the app checks your Mac before recommending anything.
M1 and M2: the app and CLI pick the FP16 build of the same pack.
Image input: PNG, JPEG and WebP with MTP intact since 1.0.3 (11 June 2026).
Memory governor: the serve banner prints engine budget, weights, resolved context window and session bank (2.10.0); requests that cannot fit are refused up front with HTTP 507 (2.10.2).

Install

Mac app: download the DMG and pick the Qwen 3.5 9B Optimized Speed pack. The app downloads it, sets up its engine, and measures your machine to pick the fastest decoding depth.

Command line:

brew install youssofal/mtplx/mtplx
mtplx serve --model Youssofal/Qwen3.5-9B-MTPLX-Optimized-Speed

mtplx run "hello" --model Youssofal/Qwen3.5-9B-MTPLX-Optimized-Speed answers one prompt in the terminal. Then point OpenCode, Pi, Claude Code, Cline, Cursor or anything that speaks the OpenAI or Anthropic API at http://127.0.0.1:8000; mtplx connect claude-code and mtplx connect opencode print the exact client config, and setup pages for each client are in the docs. The Turbo profile is the default for this pack: verify-specialized quantized-matmul kernels plus a compiled verify step, default since 2.0.0.

How it is built

Component	Format
Main body	6-bit MLX affine, 64-weight groups
MTP heads	bf16 native MTP sidecar (`mtp.safetensors`)
Sampler	Target and draft at temperature 0.6, top-p 0.95, top-k 20
Default depth	2, recorded in `mtplx_runtime.json`

Activations stay bf16 on every pass (fp16 on M1 and M2); the speculative path runs the same model as the autoregressive path. This is a quantized checkpoint built for fast local use on Apple Silicon through MTPLX.

Exactness

Speculation in MTPLX is exact. Drafts from the MTP head are accepted with the probability-ratio rule and rejected drafts are resampled from the residual, so what you sample is what the model would have sampled without speculation, at any temperature. The draft sampler is a speed knob only. The 6-bit quantization of the body is the one approximation.

← Qwen 3.6 35B-A3B Qwen 3.5 4B →