3 Local Vision Models: Moondream vs Gemma 3 vs Qwen2.5-VL

Short answer

On 15 hand-curated photos (cat, dog, bird, car, Eiffel, mountain, pizza, book, forest, flower, bicycle, phone, coffee, sunset, beach) the published "current king" of local vision-language models did not win on a single-reference BLEU4 metric against my simple ground truth. [Measured]

• Moondream 2 (1.8 B, 1.7 GB on disk) scored the highest corpus BLEU4 at 3.68, with a median wall time of 2.1 s per image. When it works, its short noun-phrase captions match the kind of single-line reference a benchmark script writes.
• Qwen2.5-VL 7B (the published king per DocVQA 95.7, MMMU 58.6) scored the highest mean sentence BLEU4 at 4.11, with a median wall time of 90.8 s per image. Its captions are longer, more detailed, and read better to a human. They just do not match the reference style.
• Gemma 3 4B scored corpus BLEU4 1.40 with a median wall time of 39.7 s per image. After the first request, every call took 35-45 s. That is the finding I did not expect.

The headline trade is speed vs detail: Moondream is 20-45x faster than the other two, Gemma 3 4B is inexplicably slow, and Qwen 2.5-VL 7B is the most accurate but takes 90 seconds per image. None of them are usable for real-time image understanding on a 24 GB Mac.

Why I tested this

The 2026 "best local vision model" conversation is dominated by Qwen2.5-VL 7B and Llama 3.2 Vision 11B. Both run on a 24 GB Mac. The published benchmark numbers are nearly all from cloud GPUs (4090, A100, H100) where 100 GB/s of memory bandwidth is not a constraint. I wanted to know what actually happens on the M2 24 GB machine I do my real work on.

I also wanted to know whether Moondream — the 1.8 B model people dismiss as a toy — has any real-world edge. The blog posts and Reddit threads I have read skip past it. Nobody benchmarks it against the published leader. I had a 24 GB Mac and a slow afternoon, so I did.

The setup

Three vision-language models, all run through Ollama on the same M2 24 GB Mac, on the same 15 photos, with the same prompt:

Model	Params	Ollama tag	Size on disk
Moondream 2	1.8 B	moondream	1.7 GB
Gemma 3 4B	4.3 B	gemma3:4b	3.3 GB
Qwen2.5-VL 7B	7 B	qwen2.5vl:7b	6.0 GB

The 15 photos are from Wikimedia Commons — public domain or CC-licensed, varied subject matter (animals, nature, architecture, food, objects, scenes). One reference caption per photo, written by hand before any model output was seen. The references are short noun-phrase style: "A tabby cat sits on snow-covered ground, looking forward." The reference style is deliberate: it matches what a human would write as a quick file caption, which is what these models are most often asked to do.

The prompt, identical for all three models:

Describe this image in one or two short sentences. Be specific about what is visible. Do not start with phrases like "The image", "This image", or "Shown here". Just describe the subject.

Streaming mode, deterministic settings (temperature 0, seed 42), one warmup pass per model that is not timed. The benchmark ran 45 image-caption pairs (3 models x 15 images) in about 30 minutes of wall time. The full per-image data, the corpus, and the prompts are in the experiments repo linked at the bottom.

The headline numbers

Model	Size	Params	Median wall (s)	Median tok/s	Corpus BLEU4	Mean s-BLEU4
Moondream 2	1.7 GB	1.8 B	2.1	5.1	3.68	3.70
Gemma 3 4B	3.3 GB	4.3 B	39.7	0.9	1.40	2.94
Qwen2.5-VL 7B	6.0 GB	7 B	90.8	0.4	2.37	4.11

Moondream wins corpus BLEU4. Qwen wins mean sentence BLEU4. Gemma is unexpectedly slow. All three have <5 tok/s on M2.

Why Moondream won corpus BLEU4

BLEU with a single reference is a style-matching metric as much as a correctness metric. The two larger models write longer, more elaborate sentences ("A reddish-brown dog with dark legs stands on a grassy hillside, partially obscured by g...") that do not match the terse reference style ("A dhole, also known as an Asiatic wild dog, stands in profile."). Moondream, when it works, writes in the same short noun-phrase style as my references. So its outputs match the references more often.

The Moondream corpus BLEU4 is also lifted by one very strong hit: the cat photo scored 22.6 sentence BLEU4. The reference was "A tabby cat sits on snow-covered ground, looking forward." Moondream returned "A tabby cat sits on a stone wall, facing the camera and looking directly at it." The model said "stone wall" instead of "snow-covered ground" — wrong ground, right subject. The 4-gram overlap is high enough to score well on BLEU.

I would not generalize this to a real-world captioning workload. With four or five references per image (the standard for COCO Captions and similar), Gemma and Qwen would almost certainly beat Moondream on corpus BLEU. With a single short reference per image, Moondream is competitive by accident of style.

Why Gemma 3 4B is so slow

This is the finding I did not expect. The first Gemma 3 4B call took 2.0 seconds. Every subsequent call took 35-45 seconds. The output tokens were normal (26-46 per image), the streaming started at 35-43 seconds. The model was not "thinking" or generating empty output; it was just sitting there for 35 seconds before the first token arrived.

I do not have a clean diagnosis. Three things are likely:

1. Ollama unloads the model between calls on Apple Silicon. The 3.3 GB model is loaded, the first call is fast, then the model is evicted from unified memory, and every subsequent call has to re-load it from disk through the SSD. The M2's SSD can do about 5 GB/s sequential read; loading 3.3 GB should be 0.7 s, not 35 s, so this is not the full story.
2. Gemma 3's image pre-processing is slow on the Metal backend. The vision encoder for Gemma 3 has a different image preprocessing pipeline than Qwen 2.5-VL or Moondream. The first call benefits from cached weights; subsequent calls may be re-doing some setup.
3. Ollama's per-request overhead on Apple Silicon has regressed for Gemma 3 specifically. Other models (Moondream, Qwen 2.5-VL 7B) do not show this pattern in the same benchmark.

The 0.9 tok/s is a wall-clock number, not a compute number. The model itself is probably generating tokens at 10+ tok/s once it gets going. The 35-second pre-roll is what kills the throughput.

Why Moondream sometimes returns empty

Five of the 15 Moondream outputs were empty strings or short fragments that look like the middle of a longer sentence:

• img_03 (bird): empty string, 1 token
• img_04 (car): "urn of a green antique car on display"
• img_08 (book): "urn of flowers on a table"
• img_10 (flower): "urn of flowers on a table"
• img_11 (bicycle): "ids/sa_1396"
• img_12 (phone): empty string, 1 token
• img_15 (beach): "urns of white and black color on a beach"

The "urn of" pattern repeats across images. The model is producing the end of a longer sentence that started in some buffer I cannot see. This looks like a streaming quirk in the Ollama-to-Moondream integration rather than a model bug — the 1-token empty case and the partial-sentence case are both "the model started generating, then something reset the buffer."

If I were using Moondream for a production workload, I would add a fallback that re-issues the request if the first response is shorter than 10 characters. For the benchmark, I left the broken outputs in the data and let the BLEU score reflect what the user would actually see.

Qwen 2.5-VL 7B is the right pick for detail

When I look at the raw Qwen outputs without the BLEU lens, they are the best of the three. The captions are detailed, accurate, and read like a real description:

• img_02: "A dhole, a wild canine with reddish-brown fur and a black-ti..." (35 tok)
• img_03: "A young bird with yellowish-brown plumage perches on a slend..." (33 tok)
• img_07: "A slice of deep-dish pizza with a thick crust, topped with t..." (39 tok)
• img_13: "A building facade features a large, oversized teacup-shaped ..." (43 tok)

The published leaderboard (Qwen 2.5-VL 7B beats Llama 3.2 Vision 11B on DocVQA 95.7 vs 88.4, MMMU 58.6 vs 50.7) is real. Qwen 2.5-VL is the most accurate local vision model you can run on a single consumer GPU. The 90-second latency on M2 is the trade. If you are captioning 5 images, that is 7.5 minutes. If you are captioning 500, that is 12.5 hours.

How to reproduce this

# 1. Pull the three models
ollama pull moondream
ollama pull gemma3:4b
ollama pull qwen2.5vl:7b

# 2. Set up a venv and install deps
python3 -m venv /tmp/vision-m2-venv
/tmp/vision-m2-venv/bin/pip install ollama sacrebleu requests Pillow

# 3. Build the test corpus (downloads 15 images from Wikimedia)
git clone https://github.com/Pitambarmahato/hardnumbers-experiments
cd hardnumbers-experiments/vision-models-m2
/tmp/vision-m2-venv/bin/python3 data/make_corpus.py

# 4. Run the benchmark
/tmp/vision-m2-venv/bin/python3 src/benchmark.py

# 5. Score captions with BLEU
/tmp/vision-m2-venv/bin/python3 src/score.py

The full per-image timings, the model outputs, the BLEU scores, and the per-model summary are all in results/benchmark.json and results/summary.json in the experiments repo.

What I did not test

• Larger images. All 15 photos are 1024 px-wide thumbnails. Real-world workflows often feed 4K photos or PDF pages. Vision models handle large images by tiling, and the relative ordering might change at higher resolutions.
• Document OCR. Qwen 2.5-VL 7B is specifically designed for OCR-heavy workloads (receipts, forms, charts). The 15 photos have minimal text. On a document OCR task, Qwen would almost certainly beat both other models by a large margin.
• Multi-image prompts. All three models support multiple images in a single prompt. I tested one image at a time. Multi-image inference has different memory and latency characteristics.
• VQA (visual question answering). The prompt was captioning. VQA is a different task with different reference style, and the ordering might change.
• Larger models. Llama 3.2 Vision 90B, Qwen 2.5-VL 72B, InternVL 78B — these need 50+ GB and are out of reach for a 24 GB M2.
• Gemma 3 12B and 27B. I tested only the 4B variant. The 12B and 27B variants might have a different speed/accuracy profile.
• Other backends. vLLM, llama.cpp directly, and MLX all implement vision inference differently. The 35-second Gemma 3 4B pre-roll might be specific to Ollama's Metal backend.

What this means for you

If you want the most accurate captioning on a 24 GB Mac and can tolerate 90 seconds per image, use Qwen 2.5-VL 7B. It is the published leader for a reason. The captions are detailed and accurate.

If you want the fastest captioning and can tolerate the occasional empty or partial output, use Moondream 2. The 2-second median is the only one in this benchmark that is usable for real-time UI. A re-issue fallback will fix the 5/15 empty cases.

If you want the published Google model, avoid Gemma 3 4B on Ollama for now. The 35-45 second pre-roll on every call makes it non-viable for any interactive use. Try it again on a different backend before giving up on it.

If you are picking a vision model for a new project, do not generalize from this benchmark. Run your own photos, your own prompt, your own evaluation. The published leaderboard predicts accuracy on standard benchmarks, not on your workload.

Reproduction footer

This article, the benchmark code, the 15-photo corpus, the per-image results, and the scoring scripts are all in the experiments repo. If you re-run on a different corpus or with a different prompt, please open a PR with the results — the goal of Hard Numbers is to make every claim falsifiable.

• Code: github.com/Pitambarmahato/hardnumbers-experiments/tree/main/vision-models-m2
• Results JSON: same repo, vision-models-m2/results/
• This article: hardnumbers.dev/articles/3-local-vision-models-moondream-vs-gemma-3-vs-qwen2-5-vl

FAQ

Which local vision model should I use on a 24GB Mac?

For captioning, Moondream 2 if you can tolerate the occasional empty output and want 2-second latency. Qwen 2.5-VL 7B if you want the most accurate captions and can wait 90 seconds per image. Avoid Gemma 3 4B via Ollama on M2 for now — the 35-45 second pre-roll per call makes it non-viable for interactive use. [Measured, Inferred]

Is Qwen 2.5-VL 7B the best local vision model in 2026?

On standard benchmarks (DocVQA, MMMU, ChartQA) yes. Qwen 2.5-VL 7B beats Llama 3.2 Vision 11B on almost every published benchmark despite being smaller. On a real workload on a 24 GB Mac, the accuracy wins are real but the 90-second latency per image is a serious trade. For document OCR, Qwen is the clear pick. [Documented, Measured]

Why is Gemma 3 4B so slow on M2?

The first Gemma 3 4B call took 2 seconds; every subsequent call took 35-45 seconds, with the first token arriving 35-43 seconds after the request. The model is generating tokens normally once it starts; the bottleneck is the pre-roll. I do not have a clean diagnosis — likely candidates are Ollama unloading the model between calls, slow image pre-processing on the Metal backend, or a Gemma 3-specific Ollama regression. Try a different backend (vLLM, llama.cpp, MLX) before giving up on it. [Measured, Inferred]

Does Moondream 2 work well for real-world use?

For a 1.8 B model, yes — when it produces output. Five of 15 outputs in the benchmark were empty strings or partial sentences (the "urn of flowers" pattern is a known streaming quirk). For interactive UI, a 2-second median is the only viable choice of the three. A re-issue fallback on empty/short output will fix most of the failures. [Measured]

What about LLaVA or Pixtral?

I did not test them. Llama 3.2 Vision is currently broken on Ollama v0.30.0+ (the "mllama" architecture is not supported). Pixtral 12B is not in the official Ollama library and needs a community GGUF. LLaVA 1.6 7B is the safe fallback — old, broadly compatible, well-documented. On published benchmarks it trails Qwen 2.5-VL 7B by 5-10 points on MMMU. I would expect it to land between Moondream and Qwen on this benchmark. [Documented, Inferred]

How does the BLEU score handle a single reference?

Single-reference BLEU is a style-matching metric as much as a correctness metric. With four or five references per image (the COCO Captions standard), the corpus BLEU scores for Gemma and Qwen would be higher because their longer prose would match some reference variant. The mean sentence BLEU4 ranking (Qwen 4.11 > Moondream 3.70 > Gemma 2.94) is a more reliable signal of caption quality than the corpus BLEU4 ranking (Moondream 3.68 > Qwen 2.37 > Gemma 1.40). [Documented]