language:
- pt
license: gemma
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- embeddings
- text-embeddings
- text-embeddings-inference
- retrieval
- information-retrieval
- semantic-search
- rag
- reranking
- clustering
- sts
- portuguese
- pt-br
- brazilian-portuguese
- mteb
- vocabulary-trimming
- knowledge-distillation
- model-soup
base_model:
- google/embeddinggemma-300m
- tardellirs/embeddinggemma-pt-br
datasets:
- HuggingFaceFW/fineweb-2
- celsowm/jurisprudencias_stj
- celsowm/leis_ordinarias_1988_2024
- celsowm/codigo_tributario_lei_5172_1966
- bancodobrasil/bbrc_brazilian_banking_regulation_corpora
- rodrigedilson/bacen2024
- carolina-c4ai/corpus-carolina
🐦 Colibri — Brazilian-Portuguese Embeddings
A tiny embedding model that punches far above its weight.
Named after the colibri (hummingbird) — the smallest bird, yet it out-flies much larger ones.
~157M params · 768-dim · plain SentenceTransformer (no adapters) · runs on a cheap CPU VPS
Colibri is a compact Brazilian-Portuguese text-embedding model derived from
google/embeddinggemma-300m. Despite being half the size,
it matches or beats much larger multilingual embedders on MTEB-BR — including 7B and 27B models — and is
designed small on purpose so it runs comfortably on a simple CPU VPS, with no GPU required.
Use cases: Portuguese semantic search, retrieval-augmented generation (RAG), retrieval, reranking, clustering, semantic textual similarity (STS) and classification — for Brazilian-Portuguese text across legal, tax, financial, medical, academic and general domains.
✨ Highlights
- 🏆 Beats bigger models on MTEB-BR — including
embeddinggemma-300m, and 7B / 27B multilingual embedders. - 🪶 Half the footprint of
embeddinggemma-300m: ~607 MB vs ~1.2 GB, and less RAM. - 💻 Runs on a $-few/month CPU VPS — benchmarked on 4 vCPU / 16 GB (no GPU).
- 🔌 Plain
SentenceTransformer— no adapters, no LoRA at inference. Drop-in. - 📐 Matryoshka dimensions (768 / 512 / 256 / 128) + an fp16 branch + a fast ONNX path.
🥇 Quality — MTEB-BR
Evaluated on MTEB-BR (22 native Brazilian-Portuguese tasks: retrieval, reranking, STS, classification, clustering, pair-classification). Score = mean over the 22 tasks.
| Model | Params | MTEB-BR |
|---|---|---|
| 🐦 Colibri (this model) | ~157M | 0.6501 |
| google/embeddinggemma-300m | 300M | 0.6490 |
| Linq-AI-Research/Linq-Embed-Mistral | 7B | 0.6473 |
| openai/text-embedding-3-large | – | 0.6449 |
| intfloat/multilingual-e5-large-instruct | 560M | 0.6409 |
| Salesforce/SFR-Embedding-2_R | 7B | 0.6397 |
| Alibaba-NLP/gte-Qwen2-7B-instruct | 7B | 0.6392 |
| microsoft/harrier-oss-v1-27b | 27B | 0.6390 |
| BAAI/bge-m3 | 568M | 0.6157 |
| intfloat/multilingual-e5-large | 560M | 0.5909 |
A ~157M-parameter model outperforming 7B–27B multilingual embedders on Brazilian Portuguese.
Size ↔ quality frontier. Colibri sits on the open-model Pareto frontier for MTEB-BR — it dominates its own
base embeddinggemma-300m (half the size, higher score) and matches models up to ~10× larger:
💻 Built for a simple CPU VPS
Colibri was designed to be small enough for cheap, GPU-less hosting. Benchmarked head-to-head against
google/embeddinggemma-300m on a Verda CPU.4V.16G (4 vCPU, 16 GB RAM) —
a typical small VPS — encoding Brazilian-Portuguese sentences:
| Precision | Model | Model size | Peak RAM | Latency (p50) | Throughput |
|---|---|---|---|---|---|
| fp32 | 🐦 Colibri | 607 MB | 969 MB | 76 ms | 37 sent/s |
| fp32 | embeddinggemma-300m | 1211 MB | 1253 MB | 78 ms | 36 sent/s |
| ONNX | 🐦 Colibri | ~600 MB | 2.9 GB | 33 ms | 49 sent/s |
| ONNX | embeddinggemma-300m | ~1200 MB | 5.3 GB | 38 ms | 41 sent/s |
Takeaways
- Half the size and less RAM than
embeddinggemma-300m— fits a smaller, cheaper VPS. - Same encode latency at equal precision (the vocabulary trim shrinks the token-embedding matrix, not the transformer compute) — you lose nothing in speed.
- The ONNX backend roughly doubles throughput (33 ms vs 76 ms) — best when latency matters; fp32 is the most RAM-frugal.
Benchmark hardware: Verda CPU.4V.16G. Numbers are indicative
single-node CPU measurements.
🚀 Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("tardellirs/colibri-embed-ptbr")
query = "Como declarar imposto de renda de aluguel?"
docs = [
"Rendimentos de aluguel devem ser informados na ficha de Rendimentos Tributáveis...",
"O Pix é um meio de pagamento instantâneo criado pelo Banco Central...",
]
q = model.encode(query, prompt_name="query") # "task: search result | query: "
d = model.encode(docs, prompt_name="document") # "title: none | text: "
print(model.similarity(q, d))
Faster on CPU — ONNX backend (≈2× throughput, no code changes to your pipeline):
model = SentenceTransformer("tardellirs/colibri-embed-ptbr", backend="onnx")
Prompts follow EmbeddingGemma conventions (query / document / classification / clustering / similarity).
🎛️ Variants: precision & dimension
Precision — a half-precision branch (same quality, half the size; best for GPU/storage — on CPU it's not faster):
| Variant | Load | Size |
|---|---|---|
| fp32 (default) | SentenceTransformer("tardellirs/colibri-embed-ptbr") |
~607 MB |
| fp16 | SentenceTransformer("tardellirs/colibri-embed-ptbr", revision="fp16") |
~304 MB |
Matryoshka dimensions — truncate the 768-dim output to 512 / 256 / 128 for smaller vectors (cheaper index, faster search) with a graceful quality trade-off:
model = SentenceTransformer("tardellirs/colibri-embed-ptbr", truncate_dim=256)
(The MTEB-BR score above is for the full 768-dim embeddings.)
🔧 How it was built
A three-stage pipeline: vocabulary trimming → knowledge distillation → model soup.
1. Vocabulary trimming. tardellirs/embeddinggemma-pt-br
trims embeddinggemma-300m's 262k multilingual vocabulary down to a ~64k Brazilian-Portuguese vocabulary,
cutting the model from 300M to **157M effective parameters** with negligible quality loss.
Code: github.com/tardellirs/embedding-vocab-trimmer.
2. Knowledge distillation. Relational (similarity-preserving) KD from two complementary teachers —
Qwen/Qwen3-Embedding-4B (strong at clustering) and
Qwen/Qwen3-Embedding-8B (strong at retrieval / reranking). The
student learns to reproduce the average of the two teachers' pairwise-similarity matrices (dimension-agnostic;
preserves STS). Trained on a diverse ~100k-passage corpus of native Brazilian Portuguese spanning legal, tax,
financial, medical, academic, technical, encyclopedic and general-web domains:
| Domain | Sources |
|---|---|
| General web | HuggingFaceFW/fineweb-2 (pt) |
| Legal | jurisprudencias_stj, leis_ordinarias_1988_2024 |
| Tax | codigo_tributario |
| Financial regulation | bbrc, bacen2024 |
| Academic | Brazilian scientific abstracts, corpus-carolina |
| Medical | Brazilian clinical / health texts |
| Technical | Stack Overflow em Português |
| Encyclopedic | Wikipedia-PT |
Evaluation integrity: the model is trained only on training / non-evaluation splits — every MTEB-BR evaluation example is held out, so the scores reflect generalization, not memorization.
3. Model soup + merge. Distillation checkpoints were combined (model soup / weight averaging) and linearly merged with the trimmed base, with the mixing weight chosen on held-out validation. Both operations yield ordinary weights, so the published model is a single standalone encoder.
🔗 Links
- Base: google/embeddinggemma-300m · Trimmed base: tardellirs/embeddinggemma-pt-br
- Trimming code: github.com/tardellirs/embedding-vocab-trimmer
- Distillation teachers: Qwen/Qwen3-Embedding-4B · Qwen/Qwen3-Embedding-8B
- Benchmark: MTEB-BR · CPU benchmark hardware: Verda
🙏 Acknowledgments
We gratefully acknowledge Verda for the GPU compute credits that supported this work.