MTPLX 2.8.3

Why this patch

2.8.2 fixed the start wizard's re-tune loop, but field reports of slow, stuttering chats kept coming, and they were right. Two more regressions shipped in 2.8.0 and survived until now because both live on a code path release QA never exercised: the uncapped request, which is what every desktop, web, and agent chat actually sends. Benchmarks and gates always cap max_tokens, and capped requests take a different path.

The streaming freeze

2.8.0 added a wire safeguard for the runaway-loop trimmer: while an uncapped request is armed, the stream held a fixed ~448-token tail so a trim could never chase bytes already delivered. At chat speeds that silenced the wire from roughly token 320 to token 768 on every uncapped response: reasoning froze for 6 to 11 seconds while the speed readout collapsed, then the backlog arrived in one burst, and the last ~448 tokens of every response arrived the same way. The engine was decoding normally the whole time, which is why every server-side number looked healthy.

The holdback is now engaged only while the output actually shows a forming loop. Healthy responses stream live, byte for byte, exactly like 2.7.1. A genuine runaway loop still gets trimmed with at most a short prefix of the repeated run ever reaching the wire, still strictly better than 2.7.1, which streamed all of it. MTPLX_REPETITION_STREAM_HOLDBACK=candidate|strict|off selects the new default, the 2.8.0 to 2.8.2 behavior, or the pre-2.8 wire.

The boot burn

2.8.0's turbo profile walked a background warm ladder up to 32,768-token prefills after every boot so deep-context benchmark rows would start warm. On real machines that meant 30 to 60+ seconds of full-throttle GPU after every launch (2.7.1: about 4.5 s), warm rungs re-firing between chat turns, and a chat sent mid-rung waiting several seconds for its first token.

The product ladder is back to the two rungs interactive chat actually reaches early. Benchmark harnesses that want deeper buckets pre-warmed opt in with MTPLX_WARMUP_LADDER. Background warm steps now also wait for 90 seconds of request quiet (MTPLX_WARMUP_IDLE_GRACE_S) before touching the model, so warming never competes with a conversation.

The app was the other half

Founder testing at temperature 1.0 on a long, multi-turn conversation caught what a fresh chat never showed: the desktop app itself rendered in O(entire transcript) per frame. The window's SwiftUI hosting view re-measured every realized message on every constraint invalidation (a third of the main thread at idle, up to 62×/s while streaming, the guard written to prevent exactly this had silently never applied), the markdown pipeline re-counted every character of every block per frame, and per-delta paths copied the whole answer to ask if it was empty. When those walks stalled a frame, the typewriter's catch-up path pasted the whole backlog at once, the literal freeze-then-vomit. The engine was streaming cleanly the entire time; replaying the wire gate over the founder's exact 37k tokens showed zero holdback engagements.

2.8.3's app build fixes all of it: the transcript min-size walk is dead, fence classification and syntax-lex state are computed once per block instead of once per frame, scroll pacing no longer invalidates the view tree, catch-up is rate-limited to read as fast typing (never a paste), the 10 Hz metrics chip no longer re-evaluates every bubble or burns a core parsing its stream byte-by-byte, and markdown tables no longer draw rows on top of each other (cells measured single-line but drew wrapped).

QA changes

The release pillar gate now includes an uncapped streamed chat measured for delivered cadence, any gap over 2 seconds fails the release. The capped-only blind spot that let both regressions through is closed.

App-side, the bar moved too: streaming QA runs on a heavy multi-turn conversation (fresh chats hide every O(transcript) render term), and "smooth" is judged at three layers, engine wire, app ingest telemetry, and the actual on-screen text, because the first two were green while the third was frozen.

Measured on an M5 Max, 27B Bare-Speed, uncapped chat

Round two: what the founder's retest caught

The first 2.8.3 candidate fixed the measured pathologies and then failed a human in three ways the instruments weren't pointed at. This round is those three, fixed and re-measured:

• Blank transcript mid-generation, the candidate's lazy transcript culled every visible row under the app's own scroll driver. Eager again; streamed tables (wrapping cells included) verified in pixels while arriving.
• Thinking well rewriting itself, reasoning now renders as plain text, and the live ticker's window anchors at line breaks so a rendered thought line never changes after you've read it.
• Freeze-then-burst cadence, server-side: whitespace-free runs flush incrementally, token emission precedes cache housekeeping barriers, and the auth middleware no longer buffers stream frames. Same-prompt A/B: ~30 sub-second delivery silences per answer → single digits; generator gaps over 200 ms → zero; every request now records a producer gap census so this regression class can never ship silent again.

Round three: the stutter that only humans could see

The founder retested with his own hands and the stutter was still there, while every hands-off verification pass stayed clean. Both observations were correct, and the difference between them was the bug:

• Interaction starved the UI's layout guard. The guard that stops the window from re-measuring the entire conversation ran in a run-loop phase macOS skips while input events keep arriving. Touch the mouse and the guard stopped running; every layout pass walked the whole transcript and screen updates coalesced into visible freezes. Phase-aligned A/B with synthesized human input on the same build and prompt: hands-off was clean on both binaries; 40 s of wheel-scrolling went from 70 stalls / 18.7 s frozen / worst 1.31 s to one 197 ms stall, and 30 s of cursor movement went from 91 stalls / 26.7 s frozen to zero. The guard now runs every run-loop turn, input storms included.
• Auto-follow fought the user. Scrolling up mid-generation raced a synchronous bottom-pin whose user-scroll signal was set asynchronously; momentum and classic wheel mice weren't covered at all. The pin now yields to any user scroll instantly and re-engages only at the bottom.
• Cancelled runs were unmeasured. The founder's stutter report came from a run he cancelled, and cancelled requests logged no stream census at all. They now log the full producer gap census and sliding windows for the streamed portion.

About multi-turn TPS

Follow-up turns in a conversation decode slower than the first, the founder measured 43 → 40 → 33 tok/s across three turns at under 10k context, and this release deliberately does not paper over it. Two real mechanisms, both now precisely attributed in every request record: each verify cycle costs more as context grows (~60 ms/cycle at 1k context → ~77 ms at 8k, honest attention physics plus a context-linear draft cost), and speculative acceptance tracks content entropy, at temperature 1.0 it collapses in free-form prose (down to ~0.4 by depth 1 in wrap-up passages) so each cycle commits fewer tokens. Neither is a regression: the same curves are measurable in every 2.8.x build. The per-cycle cost work is the standing decay track, continuing in 2.8.4; acceptance-vs-entropy is the 3.8 MTP head calibration campaign. What 2.8.3 ships is the instrumentation that makes both visible per-request instead of anecdotal.