---
language:
- pt
license: gemma
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- embeddings
- text-embeddings
- text-embeddings-inference
- retrieval
- information-retrieval
- semantic-search
- rag
- reranking
- clustering
- sts
- portuguese
- pt-br
- brazilian-portuguese
- mteb
- vocabulary-trimming
- knowledge-distillation
- model-soup
base_model:
- google/embeddinggemma-300m
- tardellirs/embeddinggemma-pt-br
datasets:
- HuggingFaceFW/fineweb-2
- celsowm/jurisprudencias_stj
- celsowm/leis_ordinarias_1988_2024
- celsowm/codigo_tributario_lei_5172_1966
- bancodobrasil/bbrc_brazilian_banking_regulation_corpora
- rodrigedilson/bacen2024
- carolina-c4ai/corpus-carolina
---
# 🐦 Colibri — Brazilian-Portuguese Embeddings
**A tiny embedding model that punches far above its weight.**
*Named after the colibri (hummingbird) — the smallest bird, yet it out-flies much larger ones.*
`~157M params` · `768-dim` · `plain SentenceTransformer (no adapters)` · `runs on a cheap CPU VPS`
---
Colibri is a compact **Brazilian-Portuguese text-embedding model** derived from
[`google/embeddinggemma-300m`](https://huggingface.co/google/embeddinggemma-300m). Despite being **half the size**,
it **matches or beats much larger multilingual embedders on MTEB(por)** — including 7B and 27B models — and is
**designed small on purpose so it runs comfortably on a simple CPU VPS**, with no GPU required.
**Use cases:** Portuguese semantic search, retrieval-augmented generation (**RAG**), retrieval, reranking,
clustering, semantic textual similarity (STS) and classification — for Brazilian-Portuguese text across legal,
tax, financial, medical, academic and general domains.
## ✨ Highlights
- 🏆 **Beats bigger models** on MTEB(por) — including `embeddinggemma-300m`, and 7B / 27B multilingual embedders.
- 🪶 **Half the footprint** of `embeddinggemma-300m`: ~607 MB vs ~1.2 GB, and less RAM.
- 💻 **Runs on a $-few/month CPU VPS** — benchmarked on 4 vCPU / 16 GB (no GPU).
- 🔌 **Plain `SentenceTransformer`** — no adapters, no LoRA at inference. Drop-in.
- 📐 **Matryoshka dimensions** (768 / 512 / 256 / 128) + an **fp16** branch + a **fast ONNX** path.
---
## 🥇 Quality — MTEB(por)
Evaluated on **[MTEB(por)](https://huggingface.co/spaces/mteb-pt/leaderboard)** (22 native Brazilian-Portuguese tasks:
retrieval, reranking, STS, classification, clustering, pair-classification). Score = mean over the 22 tasks.
| Model | Params | MTEB(por) |
|---|---:|:---:|
| 🐦 **Colibri** (this model) | **~157M** | **0.6501** |
| google/embeddinggemma-300m | 300M | 0.6490 |
| Linq-AI-Research/Linq-Embed-Mistral | 7B | 0.6473 |
| openai/text-embedding-3-large | – | 0.6449 |
| intfloat/multilingual-e5-large-instruct | 560M | 0.6409 |
| Salesforce/SFR-Embedding-2_R | 7B | 0.6397 |
| Alibaba-NLP/gte-Qwen2-7B-instruct | 7B | 0.6392 |
| microsoft/harrier-oss-v1-27b | 27B | 0.6390 |
| BAAI/bge-m3 | 568M | 0.6157 |
| intfloat/multilingual-e5-large | 560M | 0.5909 |
> A ~157M-parameter model outperforming 7B–27B multilingual embedders on Brazilian Portuguese.
**Size ↔ quality frontier.** Colibri sits on the open-model Pareto frontier for MTEB(por) — it **dominates its own
base** `embeddinggemma-300m` (half the size, higher score) and matches models up to ~10× larger:
---
## 💻 Built for a simple CPU VPS
Colibri was designed to be **small enough for cheap, GPU-less hosting**. Benchmarked head-to-head against
`google/embeddinggemma-300m` on a **[Verda](https://verda.com/?utm_content=mteb-pt) `CPU.4V.16G` (4 vCPU, 16 GB RAM)** —
a typical small VPS — encoding Brazilian-Portuguese sentences:
| Precision | Model | Model size | Peak RAM | Latency (p50) | Throughput |
|---|---|---:|---:|---:|---:|
| **fp32** | 🐦 **Colibri** | **607 MB** | **969 MB** | 76 ms | 37 sent/s |
| fp32 | embeddinggemma-300m | 1211 MB | 1253 MB | 78 ms | 36 sent/s |
| **ONNX** | 🐦 **Colibri** | ~600 MB | 2.9 GB | **33 ms** | **49 sent/s** |
| ONNX | embeddinggemma-300m | ~1200 MB | 5.3 GB | 38 ms | 41 sent/s |
**Takeaways**
- **Half the size and less RAM** than `embeddinggemma-300m` — fits a smaller, cheaper VPS.
- **Same encode latency** at equal precision (the vocabulary trim shrinks the token-embedding matrix, not the
transformer compute) — you lose nothing in speed.
- The **ONNX** backend roughly **doubles throughput** (33 ms vs 76 ms) — best when latency matters; fp32 is the
most RAM-frugal.
*Benchmark hardware: [Verda](https://verda.com/?utm_content=mteb-pt) `CPU.4V.16G`. Numbers are indicative
single-node CPU measurements.*
---
## 🚀 Usage
```python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("tardellirs/colibri-embed-ptbr")
query = "Como declarar imposto de renda de aluguel?"
docs = [
"Rendimentos de aluguel devem ser informados na ficha de Rendimentos Tributáveis...",
"O Pix é um meio de pagamento instantâneo criado pelo Banco Central...",
]
q = model.encode(query, prompt_name="query") # "task: search result | query: "
d = model.encode(docs, prompt_name="document") # "title: none | text: "
print(model.similarity(q, d))
```
**Faster on CPU — ONNX backend** (≈2× throughput, no code changes to your pipeline):
```python
model = SentenceTransformer("tardellirs/colibri-embed-ptbr", backend="onnx")
```
Prompts follow EmbeddingGemma conventions (query / document / classification / clustering / similarity).
---
## 🎛️ Variants: precision & dimension
**Precision** — a half-precision branch (same quality, half the size; best for GPU/storage — on CPU it's not faster):
| Variant | Load | Size |
|---|---|---|
| fp32 (default) | `SentenceTransformer("tardellirs/colibri-embed-ptbr")` | ~607 MB |
| fp16 | `SentenceTransformer("tardellirs/colibri-embed-ptbr", revision="fp16")` | ~304 MB |
**Matryoshka dimensions** — truncate the 768-dim output to **512 / 256 / 128** for smaller vectors (cheaper index,
faster search) with a graceful quality trade-off:
```python
model = SentenceTransformer("tardellirs/colibri-embed-ptbr", truncate_dim=256)
```
*(The MTEB(por) score above is for the full 768-dim embeddings.)*
---
## 🔧 How it was built
A three-stage pipeline: **vocabulary trimming → knowledge distillation → model soup**.
**1. Vocabulary trimming.** [`tardellirs/embeddinggemma-pt-br`](https://huggingface.co/tardellirs/embeddinggemma-pt-br)
trims `embeddinggemma-300m`'s 262k multilingual vocabulary down to a **~64k Brazilian-Portuguese vocabulary**,
cutting the model from ~300M to **~157M effective parameters** with negligible quality loss.
Code: **[github.com/tardellirs/embedding-vocab-trimmer](https://github.com/tardellirs/embedding-vocab-trimmer)**.
**2. Knowledge distillation.** Relational (similarity-preserving) KD from **two complementary teachers** —
[`Qwen/Qwen3-Embedding-4B`](https://huggingface.co/Qwen/Qwen3-Embedding-4B) (strong at clustering) and
[`Qwen/Qwen3-Embedding-8B`](https://huggingface.co/Qwen/Qwen3-Embedding-8B) (strong at retrieval / reranking). The
student learns to reproduce the **average of the two teachers' pairwise-similarity matrices** (dimension-agnostic;
preserves STS). Trained on a diverse **~100k-passage corpus of native Brazilian Portuguese** spanning legal, tax,
financial, medical, academic, technical, encyclopedic and general-web domains:
| Domain | Sources |
|---|---|
| General web | [HuggingFaceFW/fineweb-2](https://huggingface.co/datasets/HuggingFaceFW/fineweb-2) (pt) |
| Legal | [jurisprudencias_stj](https://huggingface.co/datasets/celsowm/jurisprudencias_stj), [leis_ordinarias_1988_2024](https://huggingface.co/datasets/celsowm/leis_ordinarias_1988_2024) |
| Tax | [codigo_tributario](https://huggingface.co/datasets/celsowm/codigo_tributario_lei_5172_1966) |
| Financial regulation | [bbrc](https://huggingface.co/datasets/bancodobrasil/bbrc_brazilian_banking_regulation_corpora), [bacen2024](https://huggingface.co/datasets/rodrigedilson/bacen2024) |
| Academic | Brazilian scientific abstracts, [corpus-carolina](https://huggingface.co/datasets/carolina-c4ai/corpus-carolina) |
| Medical | Brazilian clinical / health texts |
| Technical | Stack Overflow em Português |
| Encyclopedic | Wikipedia-PT |
> **Evaluation integrity:** the model is trained only on training / non-evaluation splits — **every MTEB(por)
> evaluation example is held out**, so the scores reflect generalization, not memorization.
**3. Model soup + merge.** Distillation checkpoints were combined (model soup / weight averaging) and linearly
**merged with the trimmed base** (θ = 0.35·base + 0.65·soup), with the mixing weight chosen on held-out validation.
Both operations yield ordinary weights, so the published model is a single standalone encoder.
---
## 🔗 Links
- Base: [google/embeddinggemma-300m](https://huggingface.co/google/embeddinggemma-300m) · Trimmed base: [tardellirs/embeddinggemma-pt-br](https://huggingface.co/tardellirs/embeddinggemma-pt-br)
- Trimming code: [github.com/tardellirs/embedding-vocab-trimmer](https://github.com/tardellirs/embedding-vocab-trimmer)
- Distillation teachers: [Qwen/Qwen3-Embedding-4B](https://huggingface.co/Qwen/Qwen3-Embedding-4B) · [Qwen/Qwen3-Embedding-8B](https://huggingface.co/Qwen/Qwen3-Embedding-8B)
- Benchmark: [MTEB(por)](https://huggingface.co/spaces/mteb-pt/leaderboard) · CPU benchmark hardware: [Verda](https://verda.com/?utm_content=mteb-pt)
---
## 🙏 Acknowledgments
We gratefully acknowledge **[Verda](https://verda.com/?utm_content=mteb-pt)** for the GPU compute credits that
supported this work.
Built with vocabulary trimming · relational knowledge distillation · model soup.