Update the model card from turintech/artemis-models

README.md

@@ -2,44 +2,70 @@
 license: apache-2.0
 base_model: google/embeddinggemma-2
 base_model_relation: quantized
+library_name: gguf
 pipeline_tag: sentence-similarity
 tags:
-- gguf
+- turintech
+- artemis
+- embeddinggemma
 - embeddings
-
+- retrieval
+- gguf
+- llama.cpp
+- quantized
 ---
 
-<!-- opening:start -->
-<p><a href="https://turintech.ai"><img src="https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/assets/turintech-logo.png" width="140" alt="TurinTech"></a><br><em>TurinTech models, built with Artemis</em></p>
+<!-- v2-header:start -->
+<div align="center">
+
+<a href="https://www.turintech.ai"><img src="https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/assets/turintech-logo-v2.png" width="360" alt="TurinTech"></a>
 
-Google's EmbeddingGemma 2 for search and retrieval on a CPU: it turns text into 768-dimensional vectors through llama.cpp's `/v1/embeddings` endpoint.
+<em>TurinTech models, built with Artemis</em>
 
-- 156.935 MB download for the default file, `embeddinggemma-2-Artemis-Q4_K_S.gguf`.
-- Tested on a Linux CPU with four threads and a 512-token input cap.
-- nDCG@10 0.859481, against 0.857125 for Unsloth's Q4 (175.674 MB), from a file 10.67% smaller. This development panel also chose the files.
-<!-- opening:end -->
+### EmbeddingGemma 2 for search and retrieval on a CPU, compressed with Artemis
+
+<div style="display: flex; flex-wrap: wrap; justify-content: center; align-items: center; gap: 6px;">
+<a href="https://www.turintech.ai"><img src="https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/assets/buttons/artemis.png" width="84" alt="Artemis"></a>
+<a href="https://huggingface.co/turintech"><img src="https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/assets/buttons/other-models.png" width="122" alt="Other TurinTech models on Hugging Face"></a>
+<a href="https://ollama.com/turintech/embeddinggemma-2-artemis"><img src="https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/assets/buttons/ollama.png" width="78" alt="Run it with Ollama"></a>
+<a href="https://lmstudio.ai/turintech/embeddinggemma-2-artemis"><img src="https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/assets/buttons/lm-studio.png" width="99" alt="Run it in LM Studio"></a>
+<a href="https://www.linkedin.com/company/turintechai"><img src="https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/assets/buttons/turintech.png" width="98" alt="TurinTech on LinkedIn"></a>
+</div>
+
+| Download | Peak CPU memory | Recall@10, BEIR test |
+| :---: | :---: | :---: |
+| **156.935 MB** | **0.339 GiB** | **0.6588** (Unsloth UD-Q4_K_XL 0.6427) |
+
+</div>
+<!-- v2-header:end -->
 
 ## Run it
 
-An embedding model: llama.cpp serves its vectors on `/v1/embeddings`, not chat. The commands below select the exact compact file. The local files passed the CPU interface checks described below.
+An embedding model: it turns a query or a document into a 768-dimensional vector for search and retrieval, served on
+llama-server's `/v1/embeddings` endpoint. It does not chat.
 
 ```bash
-set -e
-hf download turintech/embeddinggemma-2-Artemis-GGUF \
-  embeddinggemma-2-Artemis-Q4_K_S.gguf --local-dir ./embeddinggemma-2-artemis
-cd ./embeddinggemma-2-artemis
-printf '%s  %s\n' f76351ba13988274750620577c88911b195a15321c37fc991bed485ed8f81f45 embeddinggemma-2-Artemis-Q4_K_S.gguf | sha256sum -c -
+hf download turintech/embeddinggemma-2-Artemis-GGUF embeddinggemma-2-Artemis-Q4_K_S.gguf --local-dir .
+llama-server -m embeddinggemma-2-Artemis-Q4_K_S.gguf --embeddings --pooling mean --embd-normalize 2 \
+  -ngl 0 -t 4 -tb 4 -c 512 -b 512 -ub 512 -np 1 --flash-attn off --host 127.0.0.1 --port 8080 --no-warmup
 ```
 
-<details>
-<summary>Runtime build, server, requests and Ollama</summary>
+<!-- v2-apps:start -->
+Ollama: [`turintech/embeddinggemma-2-artemis`](https://ollama.com/turintech/embeddinggemma-2-artemis) on `/api/embed`, as in Run it &nbsp;·&nbsp; LM Studio: [`turintech/embeddinggemma-2-artemis`](https://lmstudio.ai/turintech/embeddinggemma-2-artemis)
+<!-- v2-apps:end -->
 
-<details>
-<summary>Build the pinned CPU runtime</summary>
+`llama-server` here is llama.cpp at commit `f3363593`, built for the CPU as in the folded section below. Other
+llama.cpp builds and GPUs have not been checked with these files; Ollama and LM Studio were checked as described there.
+Queries take the prefix `task: search result | query: ` and documents the form `title: ... | text: ...`; every result
+below was measured with them, mean pooling and inputs of at most 512 tokens.
 
-On Linux x86-64, install Git, CMake, a C/C++ compiler and OpenMP development support using your system's package manager. The source commit and CMake options below match our successful CPU build. This is a source-build path, not a prebuilt binary download; your compiler and CPU can produce different binary bytes. Other platforms have not been checked here. See the [upstream build instructions at this commit](https://github.com/ggml-org/llama.cpp/blob/f3363593f457169cdfbdfe22ea399b1cc139ba18/docs/build.md) for dependencies.
+<details>
+<summary>Build the runtime, embed with Python, check the download, Ollama and LM Studio</summary>
 
-Run in a directory where `llama-cpp-embeddinggemma2` does not already exist:
+On Linux x86-64, install Git, CMake, a C/C++ compiler and OpenMP development support with the system package manager
+([upstream build instructions at this commit](https://github.com/ggml-org/llama.cpp/blob/f3363593f457169cdfbdfe22ea399b1cc139ba18/docs/build.md)).
+These are the source commit and CMake options of our CPU build; another compiler or CPU can give different binaries,
+and other platforms have not been checked. Run it in a directory without a `llama-cpp-embeddinggemma2` folder:
 
 ```bash
 set -e
@@ -56,25 +82,18 @@ export PATH="$PWD/llama-cpp-embeddinggemma2/build/bin:$PATH"
 llama-server --version
 ```
 
-Keep the build directory and its libraries together. `GGML_NATIVE=ON` targets the build machine's CPU; do not assume its binaries run on a different CPU.
+Keep the build directory and its libraries together. `GGML_NATIVE=ON` targets the CPU of the build machine, so the
+binaries may not run on another CPU.
 
-</details>
-
-With that build on PATH and the current directory containing the downloaded GGUF, run:
+Check the download against its checksum:
 
 ```bash
-llama-server -m embeddinggemma-2-Artemis-Q4_K_S.gguf \
-  --embeddings --pooling mean --embd-normalize 2 \
-  -ngl 0 -t 4 -tb 4 -c 512 -b 512 -ub 512 -np 1 \
-  --flash-attn off --host 127.0.0.1 --port 8080 --no-warmup
+printf '%s  %s\n' f76351ba13988274750620577c88911b195a15321c37fc991bed485ed8f81f45 embeddinggemma-2-Artemis-Q4_K_S.gguf | sha256sum -c -
 ```
 
-Use `embeddinggemma-2-Artemis-Q5_K_S.gguf` for the larger alternative. Download the larger file explicitly with the same `hf download` command and its exact filename. Automatic Hugging Face import defaults remain unverified. These commands follow the repository head. Verify the downloaded file against the checksum below.
-
-<details>
-<summary>Embed a query and document with Python</summary>
-
-The prefixes, special tokens and 512-token cap match the measured path. Run this in another terminal:
+For the larger file, download `embeddinggemma-2-Artemis-Q5_K_S.gguf` by its exact name with the same `hf download`
+command; automatic Hugging Face imports have not been checked. With the server running, embed a query and a document
+from another terminal:
 
 ```python
 import json
@@ -101,22 +120,15 @@ for text in texts:
     print(len(vector))  # 768 for each input
 ```
 
-Both final files passed this two-input CPU interface check with llama.cpp `f3363593f457169cdfbdfe22ea399b1cc139ba18`. It is an embedding endpoint, not a chat model. This check does not establish performance or support in other runtimes.
-
-</details>
-
-### Ollama
+Both files passed this two-input check on the CPU build above. The snippet keeps the first 512 tokens of a long text;
+the test run kept the first 511 and the end token, as sentence-transformers does.
 
-[Native Ollama packages](https://ollama.com/turintech/embeddinggemma-2-artemis) are available. Pull the exact recommended variant:
+Ollama (0.40.4 tested) serves the package on `/api/embed`. `latest` and `q4_k_s` hold the default file, `q5_k_s` the
+182.04 MB file; both files passed a public pull, a full checksum and a two-input CPU embedding check. The retrieval
+numbers on this card come from llama.cpp, not Ollama.
 
 ```bash
 ollama pull turintech/embeddinggemma-2-artemis:q4_k_s
-```
-
-<details>
-<summary>Embed a query and document with Ollama</summary>
-
-```bash
 curl http://localhost:11434/api/embed -H 'Content-Type: application/json' -d '{
   "model": "turintech/embeddinggemma-2-artemis:q4_k_s",
   "input": [
@@ -128,96 +140,114 @@ curl http://localhost:11434/api/embed -H 'Content-Type: application/json' -d '{
 }'
 ```
 
-Keep the query/document prefixes and replace the text after them. This request uses CPU execution and rejects inputs exceeding the context instead of silently truncating them.
-
-</details>
+This request runs on the CPU and rejects an input longer than the context instead of cutting it. Ollama lists both
+tags as `Q4_K_M`, a label the files inherit from their headers; the tags and checksums identify the files.
 
-`latest` and `q4_k_s` select the compact file; `q5_k_s` selects the 182.04 MB alternative. Both current files passed public-pull, full-checksum and two-input CPU embedding checks in Ollama 0.40.4. The new larger file was checked after its tag was updated; the compact file and default alias are unchanged. No chat, GPU, speed or retrieval-quality claim through Ollama is made.
+LM Studio (0.4.26 tested, CPU engine 2.64.0) loaded the Hub entry
+[turintech/embeddinggemma-2-artemis](https://lmstudio.ai/turintech/embeddinggemma-2-artemis) with the default file on
+25 October 2026, and its `/v1/embeddings` on port 1234 returned 768-dimensional vectors:
 
-The existing Ollama packages display the inherited broad `Q4_K_M` metadata label, including in Ollama's runtime listing. Their payloads differ: the exact tags and file checksums identify the intended mixed-quantization artifacts.
+```bash
+lms get turintech/embeddinggemma-2-artemis@q4_k_s --gguf
+```
 
 </details>
 
-<!-- choose:start -->
-## Choose this model
-
-- Best for search and retrieval where a small download matters.
-- Consider another file when you want the best measured retrieval: our Q5_K_S (182.04 MB) has higher nDCG and Recall than Unsloth Q4. The default's Recall@10 is 0.914856, against 0.920933 for Unsloth's Q4.
-<!-- choose:end -->
-
-## Files
-
-<!-- files:start -->
-Every file we publish for this model. Sizes are download sizes, not memory.
-
-| File | Size | Use it for | Ollama tag |
-| --- | --- | --- | --- |
-| [`embeddinggemma-2-Artemis-Q4_K_S.gguf`](https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/embeddinggemma-2-Artemis-Q4_K_S.gguf) | 157 MB | Default: smaller download; measured Recall below Unsloth Q4 | `q4_k_s` (latest) |
-| [`embeddinggemma-2-Artemis-Q5_K_S.gguf`](https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/embeddinggemma-2-Artemis-Q5_K_S.gguf) | 182 MB | Larger file with higher observed nDCG and Recall on the exposed development panel | `q5_k_s` |
-
-- With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: `-hf turintech/embeddinggemma-2-Artemis-GGUF:Q4_K_S`.
-- The Ollama tag pulls our package, which holds the same file: `ollama pull turintech/embeddinggemma-2-artemis:<tag>`.
-- LM Studio's Hub entry [`turintech/embeddinggemma-2-artemis`](https://lmstudio.ai/turintech/embeddinggemma-2-artemis) offers every model file above.
-
-Smaller weights do not shrink the embedding index.
-<!-- files:end -->
-
-<!-- results:start -->
+<!-- v2-results:start -->
 ## Results
 
-Our two files against Unsloth's Q4 file and the original, on 96 queries and 1,024 documents from SciFact and FiQA. This development panel was used to choose the files.
+![nDCG@10 on 948 BEIR SciFact and FiQA test queries for our two files, Unsloth's UD-Q4_K_XL and the FP32 original, with download sizes.](assets/comparison.png)
 
 | File | Size | nDCG@10 | Recall@10 |
 | --- | --- | --- | --- |
-| **Ours, Q4_K_S (default)** | 156.935 MB | 0.859481 | 0.914856 |
-| **Ours, Q5_K_S** | 182.036 MB | 0.872079 | 0.928224 |
-| Unsloth Q4 | 175.674 MB | 0.857125 | 0.920933 |
-| Original FP32 | n/a | 0.856404 | 0.928224 |
-
-The default is the smallest file and gives up a little Recall@10 against Unsloth's Q4. Peak RAM and GPU compatibility are not established yet. [How it was tested](#measured-retrieval-trade-offs).
-<!-- results:end -->
+| [**TurinTech Q4_K_S**](https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/embeddinggemma-2-Artemis-Q4_K_S.gguf) | 156.935 MB | 0.5754 | **0.6588** |
+| [**TurinTech Q5_K_S**](https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/embeddinggemma-2-Artemis-Q5_K_S.gguf) | 182.036 MB | 0.5790 | 0.6538 |
+| Unsloth UD-Q4_K_XL | 175.674 MB | 0.5738 | 0.6427 |
+| FP32 original | n/a | **0.5833** | 0.6541 |
 
-<details>
-<summary>Measured retrieval trade-offs</summary>
+Equal-weight means of SciFact (300 queries, 5,183 documents) and FiQA (648 queries, 57,638 documents): BEIR test queries, each ranked against its dataset's full corpus. The records of how the files were chosen do not mention these queries ([How it was tested](#how-it-was-tested)); bold marks the best in each column, not a tested difference. One gap holds after correcting for four planned tests: Recall@10 of our Q4_K_S over Unsloth's UD-Q4_K_XL (0.6588 against 0.6427, Bonferroni p 0.043). The default's nDCG@10 did not separate from Unsloth's file (0.5754 against 0.5738) or from the original (0.5833), and the plan did not test our Q5_K_S. [Full results, memory and method](RESULTS.md).
+<!-- v2-results:end -->
 
-## Measured retrieval trade-offs
+<!-- v2-files:start -->
+## Files
 
-The recommended compact candidate is 156.94 MB, 10.67% smaller than the measured Unsloth Q4 file. On our exposed development panel, nDCG@10 was 0.236 percentage points higher and Recall@10 was 0.608 points lower. It offers a smaller download with a measured retrieval trade-off.
+| File | Size | Peak CPU memory | Use it for |
+| --- | --- | --- | --- |
+| [`embeddinggemma-2-Artemis-Q4_K_S.gguf`](https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/embeddinggemma-2-Artemis-Q4_K_S.gguf) | 157 MB | 0.339 GiB | Default: the smaller download, Ollama tag `q4_k_s` (latest) |
+| [`embeddinggemma-2-Artemis-Q5_K_S.gguf`](https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/main/embeddinggemma-2-Artemis-Q5_K_S.gguf) | 182 MB | 0.359 GiB | Larger file with higher measured nDCG@10 than the default, Ollama tag `q5_k_s` |
 
-A 182.04 MB alternative has higher measured nDCG and Recall than Unsloth Q4 on the same panel, at a larger file size. Neither candidate has a confirmed speed or runtime-memory advantage. Both produce 768-dimensional text embeddings; smaller weights do not shrink the embedding index.
+Text only. Smaller weights do not shrink the embedding index, which stores 768 values per text. The memory figures come from a 2,048-token context and batch; the 512-token settings in Run it were not measured. Peak CPU memory measured on AMD Ryzen 9 5900X, CPU only, 12 threads, llama.cpp f3363593 CPU build, -c 2048 -b 2048 -ub 2048 -np 1 --flash-attn off --no-warmup, peak resident memory over the 300 SciFact test queries in requests of up to 16, the memory-mapped file included.
+<!-- v2-files:end -->
 
-96 queries and 1,024 documents across SciFact and FiQA; equal-domain averages, 512-token cap, mean pooling and normalized 768-dimensional vectors. This development panel was used to select candidates. It is not an independent test or full MTEB evaluation.
+<!-- v2-pick:start -->
+## Pick this model
 
-| Candidate | File MB | nDCG@10 | Recall@10 |
-|---|---:|---:|---:|
-| Original FP32 reference | n/a | .856404 | **.928224** |
-| Unsloth Q4 | 175.674 | .857125 | .920933 |
-| Artemis compact, recommended for size | **156.935** | .859481 | .914856 |
-| Artemis larger alternative | 182.036 | **.872079** | **.928224** |
+- **Good for** search and retrieval on a CPU in a 156.935 MB download. On 948 test queries its Recall@10 was higher than that of Unsloth's 175.674 MB UD-Q4_K_XL (0.6588 against 0.6427), and its nDCG@10 did not separate from that file or the original.
+- **Pick another file** when nDCG@10 matters more than download size: our Q5_K_S (182.036 MB) scored nDCG@10 0.5790 against the default's 0.5754 but Recall@10 0.6538 against 0.6588, and no planned test covers either gap.
+<!-- v2-pick:end -->
 
-Bold marks the best displayed value in each column, not statistical significance. The larger candidate's measured gains over Unsloth Q4 are 1.50 percentage points in nDCG and 0.73 points in Recall; it is 3.62% larger. Small sampled-panel differences do not establish broad superiority. The compact candidate had one timing-comparable observation; that does not establish a reliable speed advantage. The larger candidate had one timing-comparable observation: query latency was 6.94% higher than the paired Unsloth control, while document throughput was 1.64% higher. This is not a speedup claim. No independent confirmation has been completed.
+## How it was made
 
-<details>
-<summary>Quantization labels and file size</summary>
+Built with [Artemis](https://www.turintech.ai/product/overview), TurinTech's optimisation platform, from Google's
+[EmbeddingGemma 2](https://huggingface.co/google/embeddinggemma-2) at revision `914f7f8`. Each file is a per-tensor mix
+of standard llama.cpp quant types, so the llama.cpp build above, Ollama and LM Studio load it as it is. Its label is a
+size class: the Q4_K_S file is smaller than a stock Q4_K_S file of this model, the Q5_K_S file smaller than a stock
+Q5_K_S. The tensor types and bits per weight are on the [results page](RESULTS.md). Text only.
 
-`Q4_K_S` and `Q5_K_S` are size classes in a measured standard K-quant ladder made from the same BF16 parent and quantizer build. They do not describe stock tensor recipes or rank quality. The compact file fits below the 178,164,000-byte stock Q4_K_S reference; the larger file fits below the 210,669,856-byte stock Q5_K_S reference. Intermediate K classes were checked; IQ formats were not part of this naming ladder.
+## How it was tested
 
-The compact file mixes Q3_K, Q4_K, BF16 and F32; the larger file mixes Q4_K, Q5_K, BF16 and F32. Whole-file bits per weight are 4.632730 and 5.373691, respectively; tensor-payload values are 4.166577 and 4.907537. Both files include about 15.79 MB of metadata and alignment. Their inherited `general.file_type` headers still read Q4_K_M; the exact filenames select the intended files.
+Every file in the Results table ran on one AMD Ryzen 9 5900X with no GPU (12 threads) on 27 and 28 October 2026: the
+GGUF files through llama-server at llama.cpp `f3363593`, the FP32 original through sentence-transformers, with the
+same prompts, 512-token cap, mean pooling and normalisation. The queries are all 948 BEIR test queries of SciFact and
+FiQA that have a relevance judgment, each ranked against the full corpus of its dataset. Before scoring, a BF16
+conversion of the original on the same llama-server gave the same vectors as the FP32 original on 128 of the queries,
+to a worst cosine error of 8.3e-06.
 
-</details>
+### Measured retrieval trade-offs
 
-</details>
+The files were chosen on a development panel of 96 queries from the train and dev relevance judgments of the same two
+datasets, with 1,024 of their documents. The records of how the files were chosen mention only that panel; they do not
+mention the test queries. On the development panel the default had a little lower Recall@10 than Unsloth's file
+(0.914856 against 0.920933); on the test queries it was higher (0.6588 against 0.6427), the one gap that holds after a
+Bonferroni correction over the four tests the run planned. nDCG@10 did not separate from Unsloth's file or from the
+original. The [results page](RESULTS.md) has the method, the scores per dataset, the paired tests, peak memory and the
+development panel.
 
 <!-- closeness:start -->
 ## Retrieval quality
 
-The retrieval numbers from the tables above, in one fixed layout with the protocol behind each table. nDCG@10 and Recall@10 score the ranking against the panel's relevance labels; higher is better.
+The retrieval numbers from the tables on the [results page](RESULTS.md), in one fixed layout with the protocol behind each table. nDCG@10 and Recall@10 score the ranking against the panel's relevance labels; higher is better.
 
 <details>
 <summary>Protocols and standard tables</summary>
 
 n/a means the card's tables or our run records do not have that value.
 
+### Protocol `cpu-beir-test-512-20261027`
+
+AMD Ryzen 9 5900X, CPU only, 27 and 28 October 2026.
+
+- Reference: the original google/embeddinggemma-2 FP32 checkpoint (revision 914f7f89142e33e77833254d9c9b90c3cef7303b) through sentence-transformers 6.2.0 and PyTorch 2.14.1 on the same CPU, mean pooling, L2-normalised; it is not a GGUF. A BF16 GGUF conversion served by the same llama-server matched its vectors on 128 test queries (mean cosine error 2.9e-06, worst 8.3e-06)
+- Panel `test`: BEIR SciFact and FiQA test queries, full corpora, equal-domain average; dataset: BEIR SciFact (BeIR/scifact at b3b5335604bf5ee3c4447671af975ea25143d4f5, judgments BeIR/scifact-qrels at 2938d17dc3b09882fdb8c12bbbe2e2dc0e75a029) and FiQA (BeIR/fiqa at 979c07a7cb5ccc6ca009792241fa1250b98055dd, judgments BeIR/fiqa-qrels at 252958f2d646e22cab6d0c72dd3f0d5de6d0655a); split: test relevance judgments (test.tsv); every query with at least one positive judgment; queries: 948 (SciFact 300, FiQA 648); documents: 62,821, the full corpora (SciFact 5,183, FiQA 57,638); text SHA-256: 37688fe561f82a27b54e556d1e3b41b5d340df3ead021039d7c9681cb052c41f (SHA-256 of the panel file; content digest 904fd90350ab83bcd1edc11aa893353f3352bbbc59583b24385d0ab32f54e4eb)
+- Tool: llama-server `/v1/embeddings` for the GGUF files, each text sent as the server's own token ids; cosine ranking within each dataset (ties by ascending document id), linear-gain nDCG@10 and Recall@10, averaged equally over SciFact and FiQA
+- llama.cpp build: commit f3363593f457169cdfbdfe22ea399b1cc139ba18, CPU build from source (its version line reads build 1, commit f336359)
+- Command: `llama-server -m <file.gguf> --embeddings --pooling mean --embd-normalize 2 -ngl 0 -t 12` (the flags the run record lists), 16 texts per request
+- Hardware: AMD Ryzen 9 5900X 12-Core Processor, 12 threads, no GPU
+- Settings: 512-token cap (a longer text keeps its first 511 tokens and the end token, as sentence-transformers cuts it; 2,890 of the 63,769 texts reached the cap), mean pooling including prompts, query prefix `task: search result | query: `, documents as `title: <title, or none> | text: <text>`
+- Measured by: TurinTech measured every GGUF file in this table, including Unsloth's UD-Q4_K_XL, the same way on the same machine, each once, and the original through sentence-transformers on that machine; the GGUF files measured are the published ones
+- Selection: the records of how the files were chosen do not mention these test queries; the files were chosen on the development panel of protocol cpu-dev-panel-512-20261016, whose documents come from the same two corpora
+- Source, on the results page: the table columns "nDCG@10", "Recall@10"
+- Comparison files: Unsloth: `unsloth/embeddinggemma-2-GGUF` at `031f0d4b3553`, `embeddinggemma-2-UD-Q4_K_XL.gguf`, SHA-256 ea905fd08e8061db77a0031cbb5096d459d7cfd2bdd0ad2ba66e483f16719493
+
+Protocol `cpu-beir-test-512-20261027`, panel `test` (BEIR SciFact and FiQA test queries, full corpora, equal-domain average):
+
+| File | Repository | Measured by | Size (MB) | Size (bytes) | nDCG@10 | Recall@10 |
+| --- | --- | --- | --- | --- | --- | --- |
+| FP32 original | `google/embeddinggemma-2` at `914f7f89142e` | reference | n/a | n/a | 0.5833 | 0.6541 |
+| `embeddinggemma-2-Artemis-Q4_K_S.gguf` | `turintech/embeddinggemma-2-Artemis-GGUF` at `8748160053b5` | TurinTech | 156.935 | 156935264 | 0.5754 | 0.6588 |
+| `embeddinggemma-2-Artemis-Q5_K_S.gguf` | `turintech/embeddinggemma-2-Artemis-GGUF` at `8748160053b5` | TurinTech | 182.036 | 182035552 | 0.5790 | 0.6538 |
+| `embeddinggemma-2-UD-Q4_K_XL.gguf` | `unsloth/embeddinggemma-2-GGUF` at `031f0d4b3553` | TurinTech | 175.674 | 175673856 | 0.5738 | 0.6427 |
+
 ### Protocol `cpu-dev-panel-512-20261016`
 
 Four CPU cores, 16 October 2026.
@@ -226,46 +256,50 @@ Four CPU cores, 16 October 2026.
 - Panel `dev`: BEIR SciFact and FiQA development panel, equal-domain average; dataset: BEIR SciFact and FiQA; split: development queries from the train and dev relevance judgments; queries: 96 (48 per domain); documents: 1,024 (512 per domain); text SHA-256: 097270111c4e6a70cdf602a8416897bdb393c0bebdddf1ec7c6fcf29b2487cb1 (SHA-256 of the panel file)
 - Tool: llama-server `/v1/embeddings` for the GGUF files; nDCG@10 and Recall@10 from dot-product ranking of the normalised 768-dimensional vectors, averaged equally over the SciFact and FiQA domains
 - llama.cpp build: commit f3363593f457169cdfbdfe22ea399b1cc139ba18
-- Command: `llama-server -m <file.gguf> --embeddings --pooling mean --embd-normalize 2 -ngl 0 -t 4 -c 2048 -b 2048 -ub 2048 -np 1 --flash-attn off`, inputs capped at 512 tokens by our harness (the card's 512-token run command is for interface checks, not this measurement)
+- Command: `llama-server -m <file.gguf> --embeddings --pooling mean --embd-normalize 2 -ngl 0 -t 4 -c 2048 -b 2048 -ub 2048 -np 1 --flash-attn off`, inputs capped at 512 tokens by our harness
 - Hardware: four CPU cores, no GPU
 - Settings: 512-token input cap, mean pooling including prompts, query prefix `task: search result | query: `, documents as `title: ... | text: ...`
-- Measured by: TurinTech measured every file in this table, including Unsloth's UD-Q4_K_XL, the same way on the same machine. Our numbers were measured on the raw quantizer outputs; the published files change only header metadata, with identical tensors
-- Selection: this development panel was used to choose the files; it is not an independent test or a full MTEB evaluation
-- Source on this card: the table columns "nDCG@10", "Recall@10"
+- Measured by: TurinTech measured every GGUF file in this table, including Unsloth's UD-Q4_K_XL, the same way on the same machine. Our numbers were measured on the raw quantizer outputs; the published files change only header metadata, with identical tensors
+- Selection: this development panel was used to choose the files, so it favours them; it is not an independent test or a full MTEB evaluation
+- Source, on the results page: the table columns "nDCG@10, development panel", "Recall@10, development panel"
 - Comparison files: Unsloth: `unsloth/embeddinggemma-2-GGUF` at `031f0d4b3553`, `embeddinggemma-2-UD-Q4_K_XL.gguf`, SHA-256 ea905fd08e8061db77a0031cbb5096d459d7cfd2bdd0ad2ba66e483f16719493
 
 Protocol `cpu-dev-panel-512-20261016`, panel `dev` (BEIR SciFact and FiQA development panel, equal-domain average):
 
 | File | Repository | Measured by | Size (MB) | Size (bytes) | nDCG@10 | Recall@10 |
 | --- | --- | --- | --- | --- | --- | --- |
-| Original FP32 reference | `google/embeddinggemma-2` at `914f7f89142e` | reference | n/a | n/a | 0.856404 | 0.928224 |
-| `embeddinggemma-2-UD-Q4_K_XL.gguf` | `unsloth/embeddinggemma-2-GGUF` at `031f0d4b3553` | TurinTech | 175.674 | 175673856 | 0.857125 | 0.920933 |
+| FP32 original | `google/embeddinggemma-2` at `914f7f89142e` | reference | n/a | n/a | 0.856404 | 0.928224 |
 | `embeddinggemma-2-Artemis-Q4_K_S.gguf` | `turintech/embeddinggemma-2-Artemis-GGUF` at `2f1be8eeab0f` | TurinTech | 156.935 | 156935264 | 0.859481 | 0.914856 |
 | `embeddinggemma-2-Artemis-Q5_K_S.gguf` | `turintech/embeddinggemma-2-Artemis-GGUF` at `2f1be8eeab0f` | TurinTech | 182.036 | 182035552 | 0.872079 | 0.928224 |
+| `embeddinggemma-2-UD-Q4_K_XL.gguf` | `unsloth/embeddinggemma-2-GGUF` at `031f0d4b3553` | TurinTech | 175.674 | 175673856 | 0.857125 | 0.920933 |
 
 </details>
 <!-- closeness:end -->
 
-## Artifact checks and sources
-
-Both raw files rebuilt to their measured SHA-256 identities. Each cleaned copy preserves all 413 tensor payloads and all inference metadata; only the display name and modality tags changed. Both final files returned finite, normalized vectors through the CPU endpoint. No GPU or multimodal result is claimed; the separate Ollama CPU interface checks are described above. MB are decimal file sizes, not memory requirements.
+<!-- v2-checksums:start -->
+<details>
+<summary>File checksums</summary>
 
-| File | Bytes | SHA-256 |
-|---|---:|---|
+| File | Bytes | sha256 |
+| --- | --- | --- |
 | `embeddinggemma-2-Artemis-Q4_K_S.gguf` | 156,935,264 | `f76351ba13988274750620577c88911b195a15321c37fc991bed485ed8f81f45` |
 | `embeddinggemma-2-Artemis-Q5_K_S.gguf` | 182,035,552 | `476a432281b3c9c8f16b5286bbe1aa482357e4b15fe69c3f8008ade36fd8f381` |
 
-Original: `google/embeddinggemma-2` at `914f7f89142e33e77833254d9c9b90c3cef7303b`.
-Comparator: `unsloth/embeddinggemma-2-GGUF` at `031f0d4b35536f69ab3509d4893c923264fcf253`, file `embeddinggemma-2-UD-Q4_K_XL.gguf` (175,673,856 bytes).
+</details>
+<!-- v2-checksums:end -->
 
-The earlier [151 MB](previous-compact-151mb.md) and [154 MB](previous-compact-154mb.md) candidates remain in the dated research record. This compact candidate has higher measured nDCG and Recall than both, at a slightly larger file size.
+<!-- v2-footer:start -->
+---
 
-## Changes
+<div align="center">
 
-16 October 2026: updated the larger Q5_K_S file from 201.17 MB to 182.04 MB, with higher observed nDCG and Recall on the same development panel. The compact default is unchanged. The [previous larger file](https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/42212da2548a4b9ae25db9ca429270da6d48119b/embeddinggemma-2-Artemis-Q5_K_S.gguf) and [previous card](https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/blob/42212da2548a4b9ae25db9ca429270da6d48119b/README.md) remain available at their original revision.
+[Artemis](https://www.turintech.ai/product/overview) is TurinTech's optimisation platform: teams use it to make their own models smaller and faster, or to optimise their code.
 
-These files keep improving.
+[All our models](https://www.turintech.ai/models) · [LinkedIn](https://www.linkedin.com/company/turintechai) · [Questions and requests](https://huggingface.co/spaces/turintech/README/discussions/1)
+
+<sub>Apache 2.0, as released by Google ([licence](https://www.apache.org/licenses/LICENSE-2.0)); see `LICENSE` and `NOTICE`. An independent TurinTech release, not affiliated with or endorsed by Google. When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.</sub>
 
-## About and licence
+These files keep improving.
 
-Original model by Google; public GGUF comparison by Unsloth. Both candidates use Artemis compression recipes. Apache License 2.0 applies; see [LICENSE](LICENSE) and [NOTICE](NOTICE).
+</div>
+<!-- v2-footer:end -->

RESULTS.md

@@ -0,0 +1,143 @@
+# EmbeddingGemma 2 Artemis GGUF: full results and method
+
+Everything behind the numbers on the [model card](README.md): the BEIR test queries the card reports, with the
+paired tests, peak memory and the encoder check, then the development panel the files were chosen on.
+
+## BEIR test queries, AMD Ryzen 9 5900X, 27 and 28 October 2026
+
+The numbers on the card come from this run.
+
+AMD Ryzen 9 5900X 12-Core Processor, CPU only, 12 threads. The queries are every BEIR test query of SciFact and FiQA
+with at least one positive relevance judgment, 948 in all (SciFact 300, FiQA 648), and each is ranked against the full
+corpus of its dataset (SciFact 5,183 documents, FiQA 57,638). The datasets are pinned to `BeIR/scifact` at
+`b3b5335604bf5ee3c4447671af975ea25143d4f5` with the `test.tsv` judgments of `BeIR/scifact-qrels` at
+`2938d17dc3b09882fdb8c12bbbe2e2dc0e75a029`, and `BeIR/fiqa` at `979c07a7cb5ccc6ca009792241fa1250b98055dd` with those of
+`BeIR/fiqa-qrels` at `252958f2d646e22cab6d0c72dd3f0d5de6d0655a`. Texts take the prompts in Run it, `task: search result
+| query: <text>` for a query and `title: <title, or none> | text: <text>` for a document. Each text is tokenized with
+its special tokens and capped at 512 tokens; a longer text keeps its first 511 tokens and the end token, as
+sentence-transformers cuts it (2,890 of the 63,769 texts reached the cap). Vectors are mean-pooled over every token,
+prompt included, and L2-normalised to 768 dimensions. Each query ranks its own dataset's documents by cosine
+similarity (ties by ascending document id). nDCG@10 (linear gain) and Recall@10 are averaged over the queries of each
+dataset, and the card's numbers are the mean of the two datasets with equal weight.
+
+The GGUF files ran on llama-server at llama.cpp `f3363593f457169cdfbdfe22ea399b1cc139ba18` (the commit Run it
+builds), `llama-server -m <file> --embeddings --pooling mean --embd-normalize 2 -ngl 0 -t 12`, with each text sent as
+the server's own token ids, 16 texts per request. The original, `google/embeddinggemma-2` at
+`914f7f89142e33e77833254d9c9b90c3cef7303b` (`model.safetensors` SHA-256
+`197a32965d4b1105faf060417baa899e193fb73cd401f42ec9295234d5553d79`), ran at FP32 through sentence-transformers 6.2.0
+and PyTorch 2.14.1 on the same CPU with `max_seq_length` 512. The files measured are the published ones (SHA-256 in
+the card's checksums) and Unsloth's `embeddinggemma-2-UD-Q4_K_XL.gguf` from `unsloth/embeddinggemma-2-GGUF` at
+`031f0d4b35536f69ab3509d4893c923264fcf253` (SHA-256 `ea905fd08e8061db77a0031cbb5096d459d7cfd2bdd0ad2ba66e483f16719493`).
+TurinTech measured every file the same way on the same machine, each once.
+
+The files were chosen on the development panel further down, 96 queries from the train and dev relevance judgments of
+the same two datasets with 1,024 of their documents. The records of how the files were chosen mention only that panel;
+they do not mention these test queries. Both panels draw their documents from the same two corpora.
+
+### Results
+
+| File | Size | nDCG@10 | Recall@10 | SciFact nDCG@10 | SciFact Recall@10 | FiQA nDCG@10 | FiQA Recall@10 |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| TurinTech Q4_K_S | 156.935 MB | 0.5754 | 0.6588 | 0.8372 | 0.9413 | 0.3137 | 0.3763 |
+| TurinTech Q5_K_S | 182.036 MB | 0.5790 | 0.6538 | 0.8455 | 0.9410 | 0.3125 | 0.3667 |
+| Unsloth UD-Q4_K_XL | 175.674 MB | 0.5738 | 0.6427 | 0.8416 | 0.9337 | 0.3061 | 0.3518 |
+| FP32 original | n/a | 0.5833 | 0.6541 | 0.8544 | 0.9403 | 0.3122 | 0.3680 |
+
+Sizes are download sizes in decimal MB; n/a for the original, which ran as a safetensors checkpoint. No bold here: the
+paired tests below decide what differs. The two datasets pull in different directions: on SciFact our Q4_K_S ranked a
+little below Unsloth's file (nDCG@10 0.8372 against 0.8416), on FiQA a little above (0.3137 against 0.3061).
+
+### Paired tests
+
+A paired bootstrap over queries: 10,000 resamples with seed 20261027, queries drawn with replacement within each
+dataset (dataset sizes kept), the equal-weight mean recomputed each time. The interval is the 2.5th to 97.5th
+percentile of the resampled differences; p is two-sided (twice the smaller share of resampled differences on either
+side of zero, at most 1). The plan for the run named two comparisons, our default against Unsloth's file and against
+the original, on two metrics: four tests. It set no correction for multiple tests; this page applies Bonferroni over
+the four, so a difference is stated only when the corrected p is below 0.05 (p below 0.0125).
+
+| Comparison | Metric | First | Second | Difference | 95% interval | p | Bonferroni p (x4) | Queries first better / second better |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| TurinTech Q4_K_S vs Unsloth UD-Q4_K_XL | nDCG@10 | 0.5754 | 0.5738 | +0.0016 | -0.0072 to +0.0105 | 0.716 (not significant) | 1.00 (not significant) | 155 / 139 |
+| TurinTech Q4_K_S vs Unsloth UD-Q4_K_XL | Recall@10 | 0.6588 | 0.6427 | +0.0161 | +0.0038 to +0.0287 | 0.0108 | 0.043 | 64 / 34 |
+| TurinTech Q4_K_S vs FP32 original | nDCG@10 | 0.5754 | 0.5833 | -0.0079 | -0.0165 to +0.0005 | 0.067 (not significant) | 0.27 (not significant) | 143 / 150 |
+| TurinTech Q4_K_S vs FP32 original | Recall@10 | 0.6588 | 0.6541 | +0.0047 | -0.0062 to +0.0155 | 0.399 (not significant) | 1.00 (not significant) | 43 / 39 |
+
+One difference holds: Recall@10 of our Q4_K_S over Unsloth's UD-Q4_K_XL. Our Q5_K_S was not in the plan, so its gaps
+to the other files are descriptive only.
+
+### Peak CPU memory
+
+| File | Peak CPU memory, GiB | Peak CPU memory, MiB |
+| --- | --- | --- |
+| TurinTech Q4_K_S | 0.339 | 346.8 |
+| TurinTech Q5_K_S | 0.359 | 367.4 |
+| Unsloth UD-Q4_K_XL | 0.362 | 371.0 |
+| BF16 conversion of the original | 0.657 | 672.8 |
+
+A fresh llama-server per file, `llama-server -m <file> --embeddings --pooling mean --embd-normalize 2 -ngl 0 -t 12 -tb
+12 -c 2048 -b 2048 -ub 2048 -np 1 --flash-attn off --no-warmup` at llama.cpp `f3363593`, embedded all 300 SciFact test
+queries in requests of up to 16; the figure is the server process's peak resident memory (VmHWM) at the end. The
+memory-mapped model file counts. Run it uses a 512-token context and batch, which was not measured. The BF16 row is
+the original converted to a BF16 GGUF with the converter of the same llama.cpp commit; it is there for scale and is not
+a published file.
+
+### Encoder check
+
+Before scoring, the first 64 test queries of each dataset (128 in all) were encoded twice with the same prompts,
+pooling and normalisation: by the FP32 original through sentence-transformers, and by that BF16 conversion through
+llama-server. The mean cosine error between the two was 2.9e-06 and the worst 8.3e-06 (SciFact query 5), inside the
+limits of 0.001 and 0.01. The check compared only these 128 short queries (3,003 tokens, none at the cap);
+documents, capped texts and the retrieval ranking of the BF16 conversion were not compared. In the scoring runs all
+four arms counted the same 11,513,932 tokens, with the same 2,890 texts at the cap.
+
+## Development panel, four CPU cores, 16 October 2026
+
+The files were chosen on this panel, so it favours them. It is not an independent test or a full MTEB evaluation.
+
+96 queries and 1,024 documents, 48 queries and 512 documents per dataset from SciFact and FiQA, the queries from the
+train and dev relevance judgments (panel file SHA-256 `097270111c4e6a70cdf602a8416897bdb393c0bebdddf1ec7c6fcf29b2487cb1`;
+the dataset revisions were not recorded). Four CPU cores, no GPU, CPU model not recorded; llama.cpp `f3363593`,
+`llama-server -m <file> --embeddings --pooling mean --embd-normalize 2 -ngl 0 -t 4 -c 2048 -b 2048 -ub 2048 -np 1
+--flash-attn off`, inputs capped at 512 tokens by the harness, the same prompts and pooling. Equal-domain averages. The
+original is the same FP32 checkpoint, mean pooling, L2-normalised. Our numbers here were measured on the raw quantizer
+outputs; the published files change only header metadata, with the same tensors.
+
+| File | Size | nDCG@10, development panel | Recall@10, development panel |
+| --- | --- | --- | --- |
+| TurinTech Q4_K_S | 156.935 MB | 0.859481 | 0.914856 |
+| TurinTech Q5_K_S | 182.036 MB | 0.872079 | 0.928224 |
+| Unsloth UD-Q4_K_XL | 175.674 MB | 0.857125 | 0.920933 |
+| FP32 original | n/a | 0.856404 | 0.928224 |
+
+On this panel the default, 10.67% smaller than Unsloth's file, had nDCG@10 0.236 points higher and Recall@10 0.608
+points lower (in percentage points). Our Q5_K_S, 3.62% larger than Unsloth's file, was 1.50 points higher in nDCG@10
+and 0.73 points higher in Recall@10. No test of these gaps was run. Timing was observed once per file during the
+choice: the Q5_K_S file's query latency was 6.94% higher than the paired Unsloth control and its document throughput
+1.64% higher. A single observation is not a speed result, and the card makes no speed claim.
+
+## Quantization labels and file size
+
+`Q4_K_S` and `Q5_K_S` are size classes in a measured standard K-quant ladder made from the same BF16 parent and
+quantizer build. They do not describe stock tensor recipes or rank quality. The compact file fits below the
+178,164,000-byte stock Q4_K_S reference; the larger file fits below the 210,669,856-byte stock Q5_K_S reference.
+Intermediate K classes were checked; IQ formats were not part of this naming ladder.
+
+The compact file mixes Q3_K, Q4_K, BF16 and F32; the larger file mixes Q4_K, Q5_K, BF16 and F32. Whole-file bits per
+weight are 4.632730 and 5.373691; tensor-payload values are 4.166577 and 4.907537. Both files include about 15.79 MB of
+metadata and alignment. Their inherited `general.file_type` headers still read Q4_K_M; the exact file names select the
+intended files, and the Ollama packages show the same inherited Q4_K_M label.
+
+## Files and earlier versions
+
+Both raw files rebuilt to their measured SHA-256 identities. Each published copy keeps all 413 tensor payloads and
+all inference metadata; only the display name and modality tags changed. Both returned finite, normalized vectors
+through the CPU endpoint.
+
+16 October 2026: the larger Q5_K_S file went from 201.17 MB to 182.04 MB, with higher nDCG@10 and Recall@10 on the
+development panel; the compact default did not change. The
+[previous larger file](https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/resolve/42212da2548a4b9ae25db9ca429270da6d48119b/embeddinggemma-2-Artemis-Q5_K_S.gguf)
+and [previous card](https://huggingface.co/turintech/embeddinggemma-2-Artemis-GGUF/blob/42212da2548a4b9ae25db9ca429270da6d48119b/README.md)
+remain at their original revision. The earlier [151 MB](previous-compact-151mb.md) and
+[154 MB](previous-compact-154mb.md) compact candidates remain in the research record; on the development panel the
+current default has higher nDCG@10 and Recall@10 than both, at a slightly larger size.