Four models share the top of our benchmark of 17 Ollama embedding models: embeddinggemma-2, qwen3-embedding:8b, embeddinggemma and qwen3-embedding:4b, each with an aggregate score of about 0.73. What separates them is cost. embeddinggemma, a 300M-parameter model, completed the nine-task suite in 423 seconds; qwen3-embedding:8b needed 3,064 seconds for a practically identical score.
We ran each model exactly as Ollama serves it, as a quantized build called through /api/embed, on a single NVIDIA L4 GPU, and scored it with the open-source mteb library. The suite covers English and Vietnamese retrieval for RAG, code retrieval, sentence similarity and intent classification. Two supplementary experiments test Vietnamese legal-domain fine-tuning and retrieval over page images.
Which model should you use?
Default:
embeddinggemma, the fastest model in the leading group.Vietnamese or code retrieval:
qwen3-embedding:4b, or:8bif runtime is not a concern.Image input:
embeddinggemma-2, the only image-capable embedding model in Ollama's catalogue.Avoid: English-only models for Vietnamese content, and
laya, which returns no usable embeddings.
Key results
- No clear winner at the top. The four leaders lie within 0.009 of each other, a gap a single run cannot resolve.
- Runtime differs sevenfold. Within the leading group, suite runtime ranges from 423 s to 3,064 s.
- Vietnamese separates the field. On SciFact-VN, multilingual models score 0.63–0.74, while English-only models score 0.11–0.42.
- Bigger is not better for code.
qwen3-embedding:4bleads two of the three code tasks; the 8B variant does not improve on it. - Fine-tuning pays off. 133 seconds of training lifted
paraphrase-multilingualfrom 0.500 to 0.785 nDCG@10 on Vietnamese legal retrieval.
Figure 1. Aggregate score of the 17 models. Dashed lines mark the boundaries of the four reading bands used in the report; colours denote model families.
How we tested
| Component | Configuration |
|---|---|
| Run date | 25 October 2026 |
| Hardware | Google Colab Pro, NVIDIA L4 (23 GB VRAM), 52 GB RAM |
| Software | Ollama 0.40.11, mteb 2.26.21 |
| Models | 17 Ollama embedding models; 153 results (17 models × 9 tasks) |
| Embedding call | /api/embed, 2,048-token context for every model, L2-normalised vectors |
| Prefixes | Query and document prefixes from each model card |
| Group | Tasks | What it measures | Metric |
|---|---|---|---|
| English RAG | NanoSciFact, NanoNFCorpus, NanoNQ | Document retrieval on NanoBEIR subsets | nDCG@10 |
| Vietnamese RAG | SciFact-VN | Retrieval on a Vietnamese translation of SciFact (VN-MTEB) | nDCG@10 |
| Code | CodeTransOceanDL, CodeTransOceanContest | Finding equivalent code across languages and frameworks | nDCG@10 |
| Code | CodeRAGProgrammingSolutions | Reranking candidate solutions | nDCG@10 |
| Text | STSBenchmark | Sentence similarity | Spearman |
| Text | Banking77Classification | Intent classification | Accuracy |
The aggregate is the unweighted mean of the four groups. Because Vietnamese RAG is a single task, SciFact-VN alone carries 25% of the aggregate. A score of 0.73 summarises this benchmark design; it is not an accuracy of 73%.
Similar quality, very different runtime
| Model | Aggregate | SciFact-VN | Suite runtime |
|---|---|---|---|
| embeddinggemma-2 | 0.735 | 0.712 | 521 s |
| qwen3-embedding:8b | 0.734 | 0.740 | 3,064 s |
| embeddinggemma | 0.733 | 0.705 | 423 s |
| qwen3-embedding:4b | 0.726 | 0.738 | 1,884 s |
| qwen3-embedding:0.6b | 0.678 | 0.674 | 577 s |
| snowflake-arctic-embed2 | 0.655 | 0.651 | 524 s |
| bge-m3 | 0.628 | 0.626 | 520 s |
| nomic-embed-text-v2-moe | 0.618 | 0.657 | 398 s |
| granite-embedding:278m | 0.599 | 0.571 | 311 s |
| bge-large | 0.592 | 0.420 | 500 s |
| mxbai-embed-large | 0.586 | 0.402 | 492 s |
| snowflake-arctic-embed:335m | 0.558 | 0.393 | 489 s |
| granite-embedding:30m | 0.551 | 0.374 | 267 s |
| nomic-embed-text | 0.545 | 0.390 | 346 s |
| all-minilm:33m | 0.534 | 0.236 | 455 s |
| all-minilm:22m | 0.489 | 0.106 | 333 s |
| paraphrase-multilingual | 0.456 | 0.393 | 319 s |
embeddinggemma matches qwen3-embedding:8b with 7.2 times less suite runtime. Within the Qwen3 family, runtime grows much faster than quality: the 0.6B model took 577 s (0.678), 4B took 1,884 s (0.726) and 8B took 3,064 s (0.734). Runtimes are suite totals that include model loading and, for some models, a shared GPU, so they show relative cost rather than per-query latency.
Figure 2. Aggregate score against total suite runtime (log scale). The dashed line links embeddinggemma and qwen3-embedding:8b.
Vietnamese: English scores do not transfer
Models built mainly for English degrade sharply on Vietnamese: all-minilm:22m scores 0.106, and mxbai-embed-large, bge-large and nomic-embed-text reach only 0.39–0.42. Multilingual models, including the Qwen3 and gemma families, nomic-embed-text-v2-moe, snowflake-arctic-embed2 and bge-m3, hold 0.63–0.74.
SciFact-VN is machine-translated from English, so it tests Vietnamese text but not the vocabulary, abbreviations and phrasing of native Vietnamese documents. A Vietnamese RAG system still needs its own evaluation set.
Figure 3. nDCG@10 on the four retrieval tasks. Bold marks the highest value in each column.
Code: the 4B model is enough
qwen3-embedding:4b scores 0.920 on CodeTransOceanContest and 0.768 on CodeRAGProgrammingSolutions, against 0.908 and 0.757 for the 8B variant. Both gemma models follow closely. CodeTransOcean measures retrieval of functionally equivalent code, not search from natural-language questions, so repository search still needs an evaluation built from real user queries.
Figure 4. nDCG@10 on the three code tasks.
Text: similarity and classification favour different models
mxbai-embed-large has the highest STS score (0.893), while embeddinggemma-2 leads Banking77 intent classification (0.923). The three best STS scores lie within 0.02 of each other; Banking77 separates models more clearly, with both gemma models above 0.91.
Figure 5. STSBenchmark (Spearman correlation) and Banking77 (classification accuracy).
Supplementary: fine-tuning on Vietnamese legal text
Ollama cannot fine-tune models, so we trained the corresponding Hugging Face checkpoints on 2,505 question–article pairs from Zalo AI Legal Text Retrieval and evaluated them on 788 test questions against a reduced corpus of 8,447 passages. Training used MultipleNegativesRankingLoss, batch size 32, learning rate 2e-5, three epochs and a 256-token limit.
| Hugging Face checkpoint | nDCG@10 before | nDCG@10 after | Training time |
|---|---|---|---|
| paraphrase-multilingual-mpnet-base-v2 | 0.500 | 0.785 | 133 s |
| granite-embedding-278m-multilingual | 0.741 | 0.790 | 93 s |
On SciFact-VN the same checkpoints moved by only +0.034 and −0.009, so the gain on legal text did not come at a visible cost elsewhere. These checkpoints are separate from the Ollama models, and their scores are not part of the main ranking.
Figure 6. nDCG@10 before and after fine-tuning, with SciFact-VN as an out-of-domain check.
Supplementary: retrieval over page images
embeddinggemma-2 is the only image-capable model in Ollama's embedding catalogue. Given text questions and page images, it reaches nDCG@5 of 0.84 on TabFQuAD, 0.89 on SyntheticDocQA-AI and 0.63 on ArxivQA, three tasks from the ViDoRe v1 benchmark. Images must be sent as input: [{"image": <base64>}]; in our checks, the alternative images field was silently ignored. The latest (740m) and 440m tags returned identical embeddings in our tests.
Figure 7. nDCG@5 on three ViDoRe v1 tasks. SigLIP2 and CLIP are Hugging Face reference runs with a different pipeline.
Author's picks by use case (personal opinion)
The picks below are my personal opinion as the author. They draw on the results above and weigh quality against runtime in this setup; they are not a universal ranking, so test the shortlist on your own data.
| Use case | My pick | Alternatives | Why |
|---|---|---|---|
| General RAG in English | embeddinggemma | embeddinggemma-2, qwen3-embedding:4b | Leading-group quality at the lowest runtime in that group (423 s) |
| RAG on Vietnamese documents | qwen3-embedding:4b | qwen3-embedding:8b, embeddinggemma-2 | SciFact-VN 0.738, level with 8B (0.740) at about 60% of its runtime |
| Mixed Vietnamese and English knowledge base | embeddinggemma-2 | qwen3-embedding:4b | Strong on both languages: NanoSciFact 0.882, SciFact-VN 0.712 |
| Code search and coding assistants | qwen3-embedding:4b | embeddinggemma, qwen3-embedding:8b | Best on CodeTransOceanContest (0.920) and CodeRAG (0.768) |
| Semantic similarity and deduplication | mxbai-embed-large | qwen3-embedding:8b, embeddinggemma | Highest STS score (0.893); English only |
| Intent classification and routing | embeddinggemma-2 | embeddinggemma | Banking77 accuracy of 0.923 and 0.913 |
| Scanned documents, slides and PDF pages as images | embeddinggemma-2 | — | The only image-capable model; ViDoRe nDCG@5 of 0.63–0.89 |
| Low-resource machines | embeddinggemma | granite-embedding:30m (English only) | 300M parameters with leading-group quality; granite-embedding:30m was the fastest model (267 s) |
| Fine-tuning for a Vietnamese domain | granite-embedding:278m | paraphrase-multilingual | Strong starting point (0.741 → 0.790); paraphrase-multilingual gained the most (+0.285) |
Limitations
- Nine tasks with one run each and no confidence intervals; treat differences below about 0.02 as noise.
- NanoBEIR subsets and a reduced legal corpus do not reflect production scale.
- GGUF quantization, the 2,048-token context and prompt prefixes can shift scores relative to the reference checkpoints.
embeddinggemma-2borrows its predecessor's prefixes, and Qwen3 uses one generic instruction. - Runtime was not measured on an isolated GPU; per-query latency and peak memory were not recorded.
- Fine-tuning
mxbai-embed-largecollapsed in every attempt, and the Hugging Face versions ofnomic-embed-textandembeddinggemmacould not be fine-tuned in our environment. This does not mean these models cannot be fine-tuned.
Applying the results
Use these results to shortlist two or three candidates, then test them on your own queries and documents under identical conditions, measuring latency, memory and cost alongside retrieval quality. For Vietnamese RAG, include proper names, abbreviations, diacritics and domain terms; for code search, include natural-language queries; for fine-tuning, keep an independent test set and check out-of-domain data.
Sources and reproducibility
All values come from our benchmark run of 25 October 2026. The full report, with methods, all figures and per-task scores for the 17 models, is available as a PDF.
Benchmark tasks are loaded through mteb: NanoSciFactRetrieval, NanoNFCorpusRetrieval, NanoNQRetrieval, SciFact-VN, CodeTransOceanDL, CodeTransOceanContest, CodeRAGProgrammingSolutions, STSBenchmark, Banking77Classification and three ViDoRe v1 tasks. The fine-tuning data is GreenNode/zalo-ai-legal-text-retrieval-vn on Hugging Face.
- MTEB evaluation library
- Ollama embeddings documentation
- EmbeddingGemma on Ollama
- EmbeddingGemma 2 on Ollama
- Qwen3 Embedding on Ollama
- Zalo AI Legal Text Retrieval, GreenNode version
The article thumbnail is an AI-generated editorial illustration, not a plot of results. Model pages describe the models; all scores in this article are our own measurements, not vendor claims.