Local Embeddings: Nomic vs mxbai vs BGE-M3

Short answer

On a 20-chunk RAG corpus drawn from Hard Numbers articles, Nomic Embed (274 MB) beat both mxbai-embed-large (669 MB) and BGE-M3 (1.2 GB) on retrieval quality and on speed. Nomic scored 100% hit@1, mxbai scored 87%, BGE-M3 scored 93%. Nomic was also the fastest model on every metric, roughly 2x faster than the next-fastest on chunk embedding and query embedding. [Measured]

The published MTEB leaderboard puts mxbai at 64.68 and Nomic at 62.39. On this corpus, the order flipped. The smaller, faster model won. I would not generalize that to a larger or messier corpus, but for a small-to-medium RAG workload on a 24 GB unified-memory machine, Nomic is the right default.

Why I tested this

I have been writing articles on local LLMs, RAG, and Apple Silicon inference for a few weeks. Every one of them says "use the best embedding model you can fit." The best, by MTEB score in 2026, is mxbai-embed-large or Qwen3-Embedding-8B. Both are larger than the headline LLM in most RAG pipelines, and both are larger than the document collection they are supposed to be indexing.

I wanted to know whether the published MTEB order matches what happens on a small, real-world corpus with a real workload. The Hard Numbers article archive is the test corpus. The 15 queries are the kind of questions a reader would ask to find a specific article. The three models are the open-weights embedding models with first-class Ollama support. The 24 GB machine is the M2 in my laptop.

The data says the published order does not match. Nomic is what I would deploy.

The setup

Three embedding models, all run through Ollama's /api/embeddings endpoint on the same M2 24 GB Mac:

• Nomic Embed v1.5 (nomic-embed-text) — 274 MB, 768 dimensions, 8K context, Apache 2.0. Native Matryoshka support at any dimension from 64 to 768.
• mxbai-embed-large v1 (mxbai-embed-large) — 669 MB, 1024 dimensions, 512-token context limit, Apache 2.0. Long-time MTEB champion on the open-weight leaderboard.
• BGE-M3 (bge-m3) — 1.2 GB, 1024 dimensions with dense plus sparse plus multi-vector outputs, 8K context, MIT. Hybrid retrieval from one model.

The corpus is 20 text chunks, each 100-300 words, drawn verbatim from the published Hard Numbers articles. The queries are 15 hand-written questions, each with 1-3 known-correct chunk IDs.

Same /api/embeddings call for all three. Same input strings. Same machine. Same request_count=1 per call to keep the comparison apples-to-apples. No batching, no async, no preprocessing. The slowest possible setup, on purpose, so the throughput numbers are a fair lower bound.

The headline numbers

Model	Size	Chunk embed (ms)	Query embed (ms)	hit@1	hit@3	MRR
Nomic	274 MB	23.7	8.6	100%	100%	1.000
mxbai	669 MB	43.8	16.3	86.7%	93.3%	0.917
BGE-M3	1.2 GB	49.7	23.8	93.3%	100%	0.967

Nomic won on retrieval quality and on speed. The 100% hit@1 means every one of the 15 queries had a known-correct chunk in the top-1 position. The 87% mxbai number means mxbai got 13 of 15 right at position 1. The 93% BGE-M3 number is the in-between. [Measured]

Why Nomic won

Three reasons, in order of how much each one matters.

Smaller models trained on more diverse data often win on small, in-domain corpora. The MTEB leaderboard is built on 56+ datasets covering retrieval, clustering, classification, and semantic textual similarity. Nomic and mxbai are both strong, but they are strong in different ways. Mxbai edges Nomic on MTEB's average (64.68 vs 62.39) because it does better on the long tail of classification and STS subtasks. For pure retrieval on a small, in-domain corpus, that long tail doesn't apply, and Nomic's retrieval specialization shows. [Inferred]

Nomic's 8K context window vs mxbai's 512-token limit. mxbai-embed-large has a hard 512-token chunk limit. None of my 20 chunks were that long, but the limit affects how the model represents long passages — even when the input is short, the model was trained to expect short inputs. Nomic Embed v1.5 was trained at 8K context and handles short inputs just as well. On small corpora, the chunking constraint can be the difference between a model that sees the full signal and one that has to encode everything into 512 tokens. [Inferred]

Nomic is the smallest model, so it loads and embeds fastest. At 274 MB, Nomic loads into the M2's unified memory in about a second, leaves 23.7 GB of headroom for other apps, and finishes the entire 20-chunk + 15-query workload in 1.9 seconds wall time. mxbai took 2.6 seconds. BGE-M3 took 10.3 seconds — BGE-M3 is slow not because the architecture is bad, but because at 1.2 GB the model is over twice the size of mxbai and over four times the size of Nomic, and the load time plus the per-token decode cost add up.

Per-query detail

For mxbai, the two misses were:

• "How much memory does the PaddleOCR PDF parser use on a 24GB Mac?" — mxbai returned c10 (Docling vs Marker vs PaddleOCR speed) at position 1. The expected c12 (PaddleOCR memory) was at position 4. Both chunks were in the top-3 candidates; mxbai's top-1 was the speed chunk rather than the memory chunk.
• "What models were compared in the open-weight LLM benchmark?" — mxbai returned c19 (gpt-oss dequantization penalty) at position 1. The expected c17 and c18 were at positions 2 and 4. The right chunks were retrieved, just not in the right order.

For BGE-M3, the one miss was:

• "How fast is Marker at parsing a single PDF page?" — BGE-M3 returned c12 (PaddleOCR memory) at position 1. The expected c10 (Marker's 0.86s/page) was at position 5. The query phrasing was close enough to "how fast is X" that BGE-M3's hybrid sparse+dense retrieval pulled a different chunk.

Nomic got all 15 queries right. The queries and the top-1 results are in results/embeddings_benchmark.json in the experiments repo.

How to reproduce this

The full benchmark, the 20-chunk corpus, the 15 queries, and the per-query results are all in the local-embeddings-m2 folder of the experiments repo linked at the bottom.

# Pull the three models
ollama pull nomic-embed-text
ollama pull mxbai-embed-large
ollama pull bge-m3

# Run the benchmark
python3 src/benchmark.py

The script runs each model, measures embedding throughput, computes cosine similarity, scores hit@1/hit@3/MRR against the known-correct chunk IDs, and writes results/embeddings_benchmark.json with the full per-query breakdown.

What I did not test

• Larger corpora. 20 chunks is a small test set. The 100% hit@1 on a 20-chunk corpus is impressive but it is also a function of the test being easy. On a 10,000-chunk corpus with 1,000 queries, the ordering might change. I would not generalize this result to production-scale RAG without re-running.
• Multilingual corpora. All 20 chunks are English. BGE-M3 is explicitly multilingual (100+ languages) and was designed for hybrid retrieval. The Nomic vs mxbai comparison is English-only and unfair to BGE-M3 in the multilingual dimension. If your documents are non-English, BGE-M3 is the right pick regardless of these numbers.
• Long context. None of the chunks exceeded 400 tokens, so the 8K vs 512 context difference did not bite. On long passages, Nomic and BGE-M3 would have an advantage that this benchmark did not exercise.
• Hybrid retrieval. BGE-M3 emits dense, sparse (learned BM25), and ColBERT-style multi-vector outputs. The benchmark only used the dense output. Using BGE-M3's hybrid mode would likely close the retrieval gap to Nomic and might flip the result.
• Reranking. Adding a cross-encoder reranker (bge-reranker-v2-m3, mxbai-rerank-v2, or Qwen3-Reranker) on top of any of these embedders would likely equalize the retrieval quality. The point of this benchmark is the embedding step, not the full retrieval pipeline.
• Other models. Qwen3-Embedding-8B (8B params, the MTEB multilingual leader) is the obvious one I didn't test. It would not fit on a 24 GB machine alongside a 14 B LLM. Snowflake Arctic Embed, Stella, Nomic Embed v2 (the MoE version), and Instructor-XL are other reasonable picks I didn't include.

What this means for you

If you are building a RAG pipeline on a 24 GB unified-memory machine and you want a sensible default, use Nomic Embed. It is the smallest, the fastest, and on this corpus it was the most accurate. If you have a multilingual corpus, use BGE-M3. If you need the published MTEB leader for some reason, use mxbai-embed-large and accept the 512-token chunking constraint. [Inferred]

If you have a corpus larger than a few thousand chunks, retest. The ordering might change at scale. The point of this benchmark is to show that the published leaderboard does not always predict the right answer on your workload — the right way to know is to measure.

Reproduction footer

This article, the benchmark code, the corpus, the queries, and the per-query results are all in the experiments repo. If you re-run on a different corpus or with a different model set, please open a PR with the results — the goal of Hard Numbers is to make every claim falsifiable.

• Code: github.com/Pitambarmahato/hardnumbers-experiments/tree/main/local-embeddings-m2
• Results JSON: same repo, local-embeddings-m2/results/
• This article: hardnumbers.dev/articles/local-embeddings-nomic-vs-mxbai-vs-bge-m3

FAQ

Which local embedding model should I use for RAG on a 24GB Mac?

Nomic Embed. On a 20-chunk Hard Numbers corpus, it scored 100% hit@1, was 2x faster than mxbai, and 2.8x faster than BGE-M3. The published MTEB leaderboard puts mxbai ahead, but on a small in-domain corpus the published order does not always match what happens. [Measured, Inferred]

Does the embedding model size matter for retrieval quality?

Not always. Nomic Embed is 274 MB, mxbai is 669 MB, BGE-M3 is 1.2 GB, and on this corpus Nomic won on both retrieval quality and speed. The right model is the one that fits your corpus, not the largest one you can fit on the machine. [Measured]

Should I use a 512-token context limit on mxbai?

The limit is built into the model. If your chunks are longer than 512 tokens, mxbai-embed-large will silently truncate. Nomic Embed v1.5 and BGE-M3 both support 8K context, which is the right default for most document parsing pipelines. [Documented]

What is Matryoshka embedding?

Nomic Embed supports Matryoshka representation learning — you can truncate the output vector to any dimension (64, 128, 256, 512, 768) and the first 64 dimensions still carry most of the retrieval signal. This lets you trade retrieval quality for index size and query latency. Nomic Embed is the only one of the three with this property. [Documented]

Does BGE-M3's hybrid retrieval help?

BGE-M3 can emit dense vectors, sparse learned BM25-style weights, and ColBERT-style multi-vector outputs in a single forward pass. The benchmark only used the dense output. Using the hybrid mode would likely close the retrieval gap to Nomic. The trade-off is index size: the multi-vector output is much larger than the dense output. [Documented]

Why is BGE-M3 slow on M2?

BGE-M3 is 1.2 GB on disk, four times the size of Nomic Embed. The model loading time plus the per-token decode cost adds up. On a 24 GB unified-memory machine, you can still run BGE-M3 alongside other apps, but it is the slowest of the three. The hybrid retrieval capability may justify the cost for multilingual corpora, but on a small English-only RAG workload, Nomic is the better trade. [Measured]