Faster warm turns in agent sessions
- Tool turns re-used less cache than they should have. Every tool round carries a short transient hint that shifted the cached prefix by ~200 tokens, so each turn re-processed more prompt than it needed to. The engine now records the stable boundary and restores from it directly.
- A postcommit that was about to finish is worth waiting for. When your next message arrives while the engine is a few hundred milliseconds from finishing the previous turn's cache commit, it now waits briefly (bounded, 0.6s) instead of throwing that work away and re-processing the difference. In our agent harness this turned a 1,449-token re-prefill into 436 tokens and cut that turn's time-to-first-token from 2.7s to 1.1s.
- Background cache maintenance no longer sneaks into your turn. The engine processes a request as more than one internal job, and background SSD cache work could slip into the tiny gap between them, its GPU work then drained ahead of your prompt, showing up as a ~0.8s stall and a scary-looking dip on the tokens-per-second gauge (the stream itself was fine; the average lied). Idle work now waits out those gaps, and the SSD encode/writer additionally yield to any queued or running request. If you've seen unexplained pauses at the start of a turn in agent sessions: this was it.
- The SSD tier no longer hydrates candidates that can't win. If a fresher in-RAM snapshot already matches more of your prompt, the multi-gigabyte disk read for an older SSD candidate is skipped entirely.
The session cache tells you what it's doing (#229, #230)
- The daemon now prints the resolved cache budget at startup, total, per-session cap, and whether sizing is automatic, plus the exact environment variables to override it. (This line was promised in the 2.4.2 notes; it was being logged at a level nobody sees. Sorry.)
- If a long conversation outgrows the per-session cap, you get one clear warning with the numbers and the setting that raises the ceiling, instead of silent cold prefills after a restart.
MTPLX_SESSION_BANK_MAX_BYTES=8GBnow parses ("8G", "8GB", "8GiB" all work). Unparseable values warn instead of silently using the default.- The app no longer drops explicit cache sizes you set in Settings when the policy is "target default".
Long-context decode on 32k+ agent sessions (#228)
The app was forcing a paged-attention route at 32k context with launch-day thresholds that were never re-measured. Reporters measured it 4-7x slower at 43k. The app now defers to the engine's measured thresholds (64k).
Smaller things
- Vision sessions: the near-prefix cache restore lane is now explicitly capped at the first image token, so it can never resurrect cache computed from a different image's pixels. Same-image warm reuse (the content-keyed path) is unchanged.
mtplx serve --no-auth, explicit auth off-switch for localhost binds (#235). Non-localhost binds still require a key.- Chat completion responses can now include a llama.cpp-style
timingsobject (#237, thanks to the contributor) for clients that read prompt/decode throughput from the response body. - The expected-value adaptive depth policy's cost constants now reflect measured reality on current kernels, so depth-3 drafting engages when it should (it was firing on 13% of eligible rounds despite 65% acceptance).
For the curious
The gate for this release ran a Pi-shaped agent transcript (16k context, 7 tool turns) alternating baseline and candidate under fan-verified thermal control. Full receipts live in the repo's measurement logs.
Get it. Download the current DMG, or
brew upgrade mtplx / pip install -U mtplx. Every published speed
number with its conditions is on the benchmarks page; the archive of every version is
on the releases page.