MLX vs llama.cpp on Apple Silicon: Same Model, Two Engines

Short answer

On a MacBook Pro M2 with 24 GB unified memory, running the same Qwen3 14B 4-bit model through two different local-LLM engines:

• MLX wins on chat and code — 1.42× faster end-to-end on short chat (31 s vs 44 s wall time) and dramatically faster on long generations.
• llama.cpp (Ollama) wins on long-context RAG — 1.7× faster when the prompt is ~3 000 tokens and the answer is short, because Ollama's prompt-eval is an order of magnitude faster than MLX's.
• MLX cold-loads in half the time (3.7 s vs 6.5 s) because it's just a Python library; no HTTP server to spin up.

If your agent pipeline is chat-heavy or code-heavy, pick MLX. If it's RAG-heavy (long document, short answer), pick Ollama. Most real pipelines are mixed, so the honest answer is "measure your workload" — but the numbers below give you a working baseline.

The question

There are two viable ways to run a local LLM on an Apple Silicon Mac in 2026: the long-established llama.cpp family (which Ollama wraps, plus LM Studio, plus the raw llama-server binary), and Apple's own MLX framework, which is tightly coupled to the Metal stack and the unified memory architecture. The user-facing claim for MLX is that it should be faster on M-series chips than a cross-platform C++ engine ported to Metal. But how much faster, in which workloads, and at what trade-off — that's the part nobody has measured cleanly in public.

The 2026-07-28 release of the Model Context Protocol turned "local inference" from a developer toy into a real backend for production agents. [Editorial framing, not a sourced claim — see the MCP spec for the actual change.] Anyone wiring an agent today will pick one of these two engines and live with the choice for the next year. The decision is being made on vibes.

This article replaces vibes with numbers.

Why this matters

When the agent framework hands a prompt to the model, every millisecond of prefill is a millisecond the user is waiting for the answer to start appearing. Every token of decode is a token the user is reading. If you swap engines and your wall time drops 40 %, the user just feels the product got faster. If your wall time jumps 70 %, the user files a bug.

For a tool-calling agent — the kind our tool-calling benchmark covered — most prompts are short, most answers are short JSON. That's the case where MLX wins. For a RAG agent, the prompt is the entire retrieved document, often thousands of tokens; the answer is short. That's the case where Ollama wins. Knowing which case you're in is the entire optimization problem.

Methodology

Hardware

• Apple M2 (8 cores: 4 performance + 4 efficiency), MacBook Pro
• 24 GB unified memory, no external GPU
• macOS 15.0 (Sequoia)
• Python 3.13.1 (system Homebrew)

Software

• Ollama 0.12.8 as the llama.cpp wrapper. Served the GGUF Q4_K_M weights over the local HTTP API at localhost:11434.
• mlx-lm 0.31.3 + mlx 0.32.2. Ran the MLX-community 4-bit weights in-process from Python — no HTTP server in the middle.

Both engines hit Apple's Metal backend for GPU compute. Same Metal, same hardware, same chip — the only thing varying is the inference engine.

Models

• Ollama path: qwen3:14b — the official Ollama build of Qwen3 14B Instruct, quantized to GGUF Q4_K_M, 9.3 GB on disk, 14.8 B parameters.
• MLX path: mlx-community/Qwen3-14B-4bit — the MLX-community conversion of the same upstream Qwen/Qwen3-14B model, 4-bit MLX quant, 8.3 GB on disk.

Both are 4-bit quants of the same chat-tuned Qwen3 14B base. The post-training is identical; only the runtime quant format differs. This is the closest apples-to-apples comparison you can run on a single laptop without going to per-layer instrumentation.

Workloads

Three workload shapes × three trials per engine. temperature: 0 and seed: 42 for every call. The model is allowed to use its default thinking mode (Qwen3-Instruct thinks before it answers; this is the out-of-the-box behaviour a real user would see).

Workload	Input (target)	Output (max)	What it stresses
short_chat	~20 tokens	256 tokens	Realistic chat: prompt-eval is small, decode dominates
long_context_rag	~3 000 tokens	256 tokens	RAG / doc-Q&A: prompt-eval dominates
long_generation_code	~80 tokens	1 024 tokens	Code generation: decode dominates with longer output

Results

Cold load

The first request against each engine also pays the model-load cost. MLX is in-process, so its "load" is just a mlx_lm.load() call. Ollama has to start a subprocess, read the GGUF, and warm up the Metal context. Result:

Engine	Cold load (first request)
Ollama 0.12.8	6.5 s
mlx-lm 0.31.3	3.7 s

MLX loads ~1.7× faster. That matters for short-lived agent scripts that spin up, run one request, and exit — like a CLI tool that answers a single question. It also matters for cold start in serverless deployments.

Short chat (20 in / 256 out)

Wall time median across 3 trials. Both engines produced the full 256 tokens of answer on every trial.

Engine	Wall time	Decode rate (output tok/s)
Ollama	44.1 s	5.8–7.4 (median 7.3)
MLX	31.1 s	6.9–9.7 (median 9.0)

MLX is 1.42× faster wall-time on a short chat. The decode-rate numbers are not directly comparable because Ollama's gen_tok_s is pure-decode throughput while MLX's is total throughput (prefill + decode combined) — the mlx_lm Python API does not expose prefill timing separately. The wall-time number is the clean comparison, and it favours MLX.

Long-context RAG (3 000 in / 256 out)

A 3 000-token prompt (a long document plus a question) with a 256-token answer. This is the case where Ollama's strength shows up.

Engine	Wall time	Decode rate (output tok/s)
Ollama	42.9 s	6.0–6.2 (median 6.1)
MLX	75.0 s	(total) 38–44

Ollama is 1.7× faster wall-time here. The reason is the prefill stage: Ollama reports a median prompt-eval rate of ~16 000 tok/s after warm-up on this 14 B Q4_K_M model — that's an order of magnitude faster than what MLX achieves in the same window. MLX spends most of its 75 s building the KV cache for the long prompt; Ollama finishes that step in under 200 ms and then spends the rest of the time decoding.

If you have a long-context RAG agent — and most retrieval-augmented pipelines do — Ollama is the right choice today.

Long generation (80 in / 1 024 out)

A short prompt, a long output (function-generation).

Engine	Wall time	Output tokens produced
Ollama	166.4 s	1 024 (hit max)
MLX	24.7 s	220 (hit EOS after writing the function)

MLX is 6.7× faster wall-time. Read the numbers carefully though: MLX stopped at 220 tokens because the model produced a complete function and emitted EOS, while Ollama kept generating until the 1 024 token cap. So this isn't a clean "MLX is 6.7× faster at the same work" result — it's "MLX finished a code task in 25 s; Ollama wrote four times more output in 167 s." The per-token decode rate is similar (6.1–6.2 tok/s for Ollama; ~9 tok/s for MLX). The wall-time gap is mostly "MLX hit EOS faster" plus the actual MLX decode advantage on the long stream.

For an agent writing a function and then a unit test, the time to finished code is what matters, and MLX wins it.

What the numbers mean

The single most useful framing is: what fraction of the request is prefill, and what fraction is decode?

• If prefill dominates (long document, short answer), Ollama wins. Its prefill kernel is dramatically faster.
• If decode dominates (chat, code, long generation), MLX wins. Its decode path is faster and the Python-process overhead is amortised over more tokens.
• For mixed workloads, the wall-time difference is the right number, and it usually lands in MLX's favour by 20–40 %.

There's also a secondary axis that matters: how many concurrent users are you serving? Both engines are single-stream in this benchmark. llama.cpp's server mode and vLLM (not in this test) handle concurrent batching much better. MLX is naturally per-process, so concurrency means multi-process, which means a different engineering setup. For one user at a time, MLX is great. For 50 concurrent users, neither MLX nor Ollama is the right tool — vLLM or llama.cpp's server with -np is.

When to pick which

• RAG-heavy agent (large prompts, short answers): Ollama / llama.cpp. The prefill is 10× faster.
• Chat or code agent (small prompts, longer answers): MLX. End-to-end wall time is 30–50 % lower.
• CLI tool that loads once, runs once, exits: MLX. Cold-load is nearly half the time and you don't have to manage a subprocess.
• Server with many concurrent users: Neither in this benchmark. Use vLLM.
• You don't know yet: Start with Ollama. It has the better DX (model registry, Modelfile, one-command install). Switch to MLX when you have a workload to measure against.

Limitations

A few honest caveats:

• Qwen3-Instruct thinks before it answers in both engines. The thinking tokens are part of eval_count for Ollama and part of wall_ms for both. The comparison is fair because both engines pay the same thinking cost, but if you switch to a non-thinking model (Llama 3.x, Mistral, etc.) the absolute numbers will change.
• The mlx_lm API does not expose prefill timing. That's why we report wall time as the primary comparison and decode rate as a secondary one. The prefill-rate gap on the RAG workload is large enough that the wall-time comparison still tells the right story.
• The code workload produced different output lengths on the two engines (220 vs 1 024 tokens). The 6.7× wall-time number is partly driven by that length difference. The decode-rate gap is the cleaner signal there.
• Both engines are single-stream. A concurrent-load comparison would be a different article.
• M2 is the slowest of the M-series that MLX still supports well. On an M3 Pro or M4 Max, MLX's advantage widens. We did not measure that.

Reproduction code & raw data

The exact benchmark script and the results JSON that fed every table in this article are open-source:

• Repo: github.com/Pitambarmahato/hardnumbers-experiments
• This experiment: mlx-vs-llama-cpp/

The benchmark.py in that directory is what generated results.json. Re-run on any M-series Mac (or adapt for another local-LLM runtime) and the numbers will land within the same ballpark — same models, same prompts, same scoring logic. The README in that directory has the exact reproduction recipe, including the huggingface-cli download mlx-community/Qwen3-14B-4bit command to fetch the MLX weights.