The MTPLX API.

Every endpoint the server exposes, the request fields it reads, and what comes back. mtplx start or the app's play button starts the server; the app and the CLI share it. Created by Youssof Altoukhi, who brought native MTP to the Mac in April 2026.

Base URLs and auth

The Anthropic SDK appends /v1/messages itself, so its base URL is the bare server root; a /v1 base would request /v1/v1/messages, which has no route.

On localhost no key is required. Clients that insist on one can send any non-empty string. Non-localhost binds require --api-key, and requests then authenticate with either header:

Authorization: Bearer <key>
X-API-Key: <key>

To reach MTPLX from other devices on your network, or from a Windows VM on the same Mac, bind all interfaces with a key file. If the file does not exist yet it is created with a fresh key and printed once, and startup prints a Network OpenAI API Base URL for the other machine to use.

mtplx serve --host 0.0.0.0 --port 8000 --api-key-file ~/.mtplx/api-key

Endpoints

Chat completions

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"mtplx","messages":[{"role":"user","content":"hello"}],"stream":true}'

The server has one chat model loaded and answers with it whatever chat model name the request carries; the served id is listed at /v1/models. Request fields the server reads: messages, stream, stream_options, max_tokens, temperature, top_p, top_k, presence_penalty, frequency_penalty, seed, tools, tool_choice, response_format, reasoning_effort and generation_mode.

Streaming. Server-sent events, one chunk per committed token by default. Start the server with --stream-interval N to batch committed-token chunks when a client prefers fewer events.

Sampler. With no sampler fields set, the server uses the pack's own sampler: for Qwen 3.8 the official Qwen 3.8 sampling, temperature 1.0, top-p 0.95, top-k 20. Acceptance of drafted tokens is exact probability-ratio rejection sampling with residual correction, so the output follows the model's distribution at any temperature. The penalties default to 0, an exact no-op that preserves MTP exactness.

Plain decoding per request. generation_mode may be "mtp" or "ar". "ar" uses target-only decoding and reports mtp_depth: 0; it does not unload the MTP weights, so the next request can use MTP again.

Tool calls. When tools are active, Qwen XML tool calls are translated into OpenAI delta.tool_calls chunks as the function name and arguments stream. Unknown or malformed tool-shaped output falls back to assistant content rather than hanging or returning a server 500.

Structured output. response_format with a json_schema has run with full MTP speed since 2.3.0 (21 Jul 2026).

Messages (Anthropic)

curl http://127.0.0.1:8000/v1/messages \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "mtplx",
    "max_tokens": 128,
    "system": "Be concise.",
    "messages": [
      {"role": "user", "content": [{"type": "text", "text": "Write one sentence about native MTP."}]}
    ]
  }'

Requests are translated into the same internal chat path as /v1/chat/completions and returned as Anthropic-shaped message payloads. Supported: system as text or text content blocks; messages[].content as text or text and tool-result content blocks; max_tokens, temperature, top_p, top_k; tools, tool_choice, stop_sequences, thinking; stream false or true.

Streaming. Server-sent events with message_start, content_block_start, content_block_delta, content_block_stop, message_delta and message_stop. Qwen reasoning maps to Anthropic thinking blocks: a content_block_start with content-block type thinking, then thinking_delta events, with answer text resuming in a separate text block. Since 2.10.2 (1 Sep 2026) the bridge emits prefill keep-alives as empty thinking_delta events inside that block, so a first turn whose prefill outlives a client's idle window survives.

Cache accounting. The Anthropic bridge reports cache_read_input_tokens and input_tokens as disjoint fields, so a client's context meter shows the true delta (2.10.2, PR #417). Claude Code's complete client-tool loop has run on this route since 0.3.7 (17 May 2026); setup is on the Claude Code page.

Usage and caching

Every response reports usage.cached_tokens: the number of prompt tokens served from the session cache instead of being prefilled again. Since 2.0.0 (6 Jul 2026) MTPLX checkpoints the attention KV cache plus the recurrent and conv GDN state at commit boundaries, with speculation on. A follow-up turn that extends an earlier prompt restores that prefix: mid-session tool rounds restore warm in under 2 s, and a 100k-token session restores in about 2 s after a restart instead of a five-minute cold prefill. The SSD session cache that makes the restart case work is on by default; disable it with --ssd-session-cache off.

Embeddings and rerank

The same daemon can serve retrieval models, so a RAG or agent-memory setup needs no second inference server. Point it at any MLX embedding or reranker model, by Hugging Face id or local path, optionally with a REF=served-id alias. Both flags repeat, and the same reference listed as both roles loads one copy of the weights.

mtplx serve \
  --embedding-model mlx-community/Qwen3-Embedding-8B-4bit-DWQ \
  --reranker-model vserifsaglam/Qwen3-Reranker-4B-4bit-MLX

curl http://127.0.0.1:8000/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"Qwen3-Embedding-8B-4bit-DWQ","input":["hello","world"]}'

curl http://127.0.0.1:8000/v1/rerank \
  -H 'Content-Type: application/json' \
  -d '{"query":"where is the cache?","documents":["the cache lives in ~/.mtplx","unrelated text"]}'

Retrieval models load on first request and are capped by --retrieval-max-resident (default 2), which unloads the least recently used one beyond the cap. With nothing configured the endpoints answer 404 and chat behaves as before. /v1/models stays chat-only by default; list retrieval models with ?capability=embedding or ?capability=rerank, and a chat completion that names a retrieval id gets a clear 400. Checkpoints that bundle their own Python inference code are refused with a 403 until you opt in with --retrieval-trust-remote-code. These models skip the MTP path: multi-token prediction makes next-token decoding cheaper, which means nothing for a model that returns a vector. Persist them in ~/.mtplx/config.toml as embedding_models and reranker_models, or in the app under Settings, Retrieval endpoints.

Health and metrics

GET /health reports model load state, profile, exactness baseline, MLX and runtime information, fan mode and warmup status. The payload includes generation_mode, load_mtp, mtp_enabled, depth, api_key_required, rate_limit_per_minute, stream_interval, warmup and reasoning_parser, so a client harness can confirm the active serving policy.

GET /metrics returns a JSON snapshot of runtime KPIs: latest (the most recent turn), recent (the last 32 turns) and tool_parse_counters.

curl http://127.0.0.1:8000/health
curl http://127.0.0.1:8000/metrics

Python clients

The stock SDKs work unchanged. OpenAI, streaming:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="local")

stream = client.chat.completions.create(
    model="mtplx",
    messages=[{"role": "user", "content": "Write a tiny TOML parser example."}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)
print()

Anthropic, with the bare server root as the base URL:

from anthropic import Anthropic

client = Anthropic(api_key="local", base_url="http://127.0.0.1:8000")

message = client.messages.create(
    model="mtplx",
    max_tokens=256,
    system="Be concise.",
    messages=[{"role": "user", "content": "Write a tiny Python function that clamps a number."}],
)

for block in message.content:
    if block.type == "text":
        print(block.text)

Server flags

mtplx serve --port 8000
mtplx serve --host 0.0.0.0 --api-key "$MTPLX_API_KEY"
mtplx serve --rate-limit 120
mtplx serve --stream-interval 4
mtplx serve --warmup-tokens 16
mtplx serve --reasoning-parser qwen3
mtplx serve --no-mtp

--warmup-tokens runs a small startup generation after model load and reports the result in /health; --strict-warmup makes a warmup failure fatal. --no-mtp serves plain autoregressive decoding on the same loaded model. Sampler defaults can be set at startup with --default-presence-penalty and --default-frequency-penalty, or changed live with mtplx settings set. Every command takes --help. Deeper reference lives in the repository: api.md, server.md, concurrency.md.