--- base_model: VAGOsolutions/SauerkrautLM-GLiNEREL-314m license: other license_name: lfm1.0 license_link: LICENSE pipeline_tag: token-classification library_name: glinerel-fast language: - en - de - fr - it - es tags: - glinerel - sauerkrautlm - named-entity-recognition - relation-extraction - information-extraction - pii-detection - pruning - knowledge-distillation --- # SauerkrautLM-GLiNEREL-128m Joint named-entity recognition, PII detection and relation extraction in one encoder pass. **It turns a 33-page PDF into a knowledge graph in 0.36 s on a GPU, where a 4B generative model needs 17.6 s for a single paragraph.** Labels are free text, chosen at inference time. English, German, French, Italian, Spanish. **[Try it in the demo →](https://huggingface.co/spaces/VAGOsolutions/SauerkrautLM-GLiNEREL-Knowledge-Graph)** | | | |---|---| | Parameters | 127.9M (256 MB) | | Origin | pruned + distilled from [SauerkrautLM-GLiNEREL-314m](https://huggingface.co/VAGOsolutions/SauerkrautLM-GLiNEREL-314m): 41% of the parameters, 98% of the score | | Overall score (4 relation benchmarks) | 65.2 | | Latency per text | 2.73 ms NVIDIA H100 (fp16) · 37.9 ms CPU (Apple M3 Pro, 4.1× faster than the reference) | | License | [LFM Open License v1.0](./LICENSE) | | Demo | [SauerkrautLM-GLiNEREL Interactive Knowledge Graph](https://huggingface.co/spaces/VAGOsolutions/SauerkrautLM-GLiNEREL-Knowledge-Graph): PDF → knowledge graph, text extraction, benchmarks | ![A 33-page PDF becomes an interactive knowledge graph in the demo: SauerkrautLM-GLiNEREL-128m on the German Wikipedia article Siemens in 0.36 s on ZeroGPU and 2.73 ms per text on an NVIDIA H100; a 4B generative model needs 17.6 s for one paragraph.](./SauerkrautLM-GLiNEREL-128m-pdf-graph.png) **Try it:** [SauerkrautLM-GLiNEREL Interactive Knowledge Graph](https://huggingface.co/spaces/VAGOsolutions/SauerkrautLM-GLiNEREL-Knowledge-Graph): upload a PDF, explore the graph and export it as JSON, CSV, GraphML or Cypher (Neo4j). ![The SauerkrautLM-GLiNEREL family on the main benchmarks: overall score, NER and relation F1, PII, multilingual NER, German relations, GPU latency on an NVIDIA H100, CPU latency on an Apple M3 Pro and model size.](./SauerkrautLM-GLiNEREL-128m-overview.png) ## Quickstart: glinerel-fast [`glinerel-fast`](https://pypi.org/project/glinerel-fast/) is our new inference library for SauerkrautLM-GLiNEREL, written for CPUs: about 1,600 lines of plain PyTorch, no `transformers`; also runs on NVIDIA and Apple GPUs. ```bash pip install -U glinerel-fast # private repo: hf auth login ``` ```python from glinerel_fast import GLiNERel model = GLiNERel.from_pretrained("VAGOsolutions/SauerkrautLM-GLiNEREL-128m") # device="cuda", precision="bf16" on GPUs result = model.extract( "Marie Curie was born in Warsaw. She married Pierre Curie, a professor at the University of Paris.", entities=["person", "location", "organization"], relations=["born in", "spouse", "works for"], ) # {"entities": [{label, text, start, end, score}, ...], "relations": [{label, head, tail, score}, ...]} model.redact("Call Laura Bianchi at +39 347 555 0192.", ["first name", "last name", "phone number"]) # 'Call [FIRST_NAME] [LAST_NAME] at [PHONE_NUMBER].' ``` ### What makes it fast - **Packed batches:** the tokens of all texts are concatenated without padding; only attention sees a padded view. - **Fused matrices:** q/k/v and the SwiGLU gate/up projections are one matrix product each; the LFM2 short convolutions run as three shifted multiply-adds that never cross text boundaries. - **No 4-D intermediate in span scoring:** the FiLM layer is rewritten algebraically, so the texts × labels × spans × width tensor is never built. - **Cache-sized CPU batches:** texts sorted by length, at most ~2,048 tokens per batch so activations stay in the CPU cache; tokenized words and label prompts are cached. - **Inference-only weights:** training heads are dropped at load time, no `transformers` model is built. - **GPU (0.1.1):** span selection on the GPU, an asynchronous pipeline and CUDA graphs for small batches: about 3 ms per text on an H100. - **Same predictions:** in fp32 the output matches the reference implementation exactly (agreement F1 1.000). ![glinerel-fast against the reference implementation on an Apple M3 Pro: time per text on CPU and Apple GPU, one text and batches of 32, and loading time.](./SauerkrautLM-GLiNEREL-128m-glinerel-fast.png) ## Pruning Structured width pruning of SauerkrautLM-GLiNEREL-314m (narrower residual stream and feed-forward layers, factorized token embedding; all 14 layers and all attention heads kept), followed by knowledge distillation from SauerkrautLM-GLiNEREL-314m. ![Architecture of the SauerkrautLM-GLiNEREL family: residual width, feed-forward width, token embedding, layers, heads, parameters and overall score of 314m, 128m and 72m.](./SauerkrautLM-GLiNEREL-128m-pruning.png) ## Speed ![glinerel-fast speed of the three SauerkrautLM-GLiNEREL sizes: CPU latency and throughput with and without torch.compile.](./SauerkrautLM-GLiNEREL-128m-speed-compile.png) ![Time to process one German document of 1,065 characters, with the model warm and the median of 3 calls. On a laptop CPU, SauerkrautLM-GLiNEREL-128m takes 402 ms, gliner2.5-multi-v1 takes 1.3 s, and the 4B generative models take 13.5 and 14.5 minutes. Bars are linear up to 1.5 s, with an axis break for the generative models.](./SauerkrautLM-GLiNEREL-128m-speed.png) *One German document (1,065 characters), against gliner2.5 and 4B generative models.* ## Results ![Zero-shot NER and relation F1 on the dev splits of CoNLL04, CrossRE, DocRED and FewRel. SauerkrautLM-GLiNEREL-128m has the best value in every row. Its mean relation F1 (53.0) is 1.9× that of Qwen3.8-27B without reasoning (27.5). The LLM rows were scored by a separate scorer that follows the same protocol.](./SauerkrautLM-GLiNEREL-128m-relation-benchmarks.png) *Development splits, zero-shot, against gliner2.5, gliner-relex, GPT-6 Luna and Qwen3.8-27B.* ![Official test splits (CoNLL04, CrossRE) and multilingual NER and PII F1 of SauerkrautLM-GLiNEREL-314m, 128m and 72m.](./SauerkrautLM-GLiNEREL-128m-test-multilingual.png) ![Zero-shot strict relation F1 on the German RED^FM test split (285 documents, 32 relation types), with 95% document-level bootstrap intervals. All three SauerkrautLM-GLiNEREL sizes score 2.9 to 3.8 times gliner2.5-multi-v1, and the intervals do not overlap. Thresholds were chosen on the evaluated split, so scores are slightly optimistic.](./SauerkrautLM-GLiNEREL-german-relations.png) *German relations (RED^FM test split).* ![Zero-shot strict entity F1 on the German test splits of RED^FM (285 documents) and MobIE (623 documents), with English and German label names. A SauerkrautLM-GLiNEREL model has the highest F1 in 9 of 10 rows, but on RED^FM all four models are within 3 points. Thresholds were chosen on the evaluated split.](./SauerkrautLM-GLiNEREL-german-ner.png) *German entities (RED^FM and MobIE test splits).* ![Token usage, API latency and cost of the two LLM baselines on all 14,580 dev records, zero-shot with reasoning off. Costs are as reported by OpenRouter. Latency was measured with 16 calls in flight, so it is not comparable to local CPU time.](./SauerkrautLM-GLiNEREL-llm-cost-tokens.png) *Tokens, latency and cost of the LLM baselines.* ![LLM baselines with medium reasoning and four zero-shot instructions on a 600-record subset. Reasoning adds 2–3 overall points at 3 to 4.5 times the cost per record, and the best run (Qwen3.8-27B, p1) reaches 53.5. SauerkrautLM-GLiNEREL-128m's 65.2 was measured on the full dev splits, so the comparison is indicative.](./SauerkrautLM-GLiNEREL-128m-llm-reasoning.png) *LLM baselines with reasoning and four prompts (600-record subset).* Thresholds are chosen per benchmark; LLMs get the same label sets with one fixed instruction at temperature 0. ## Citation ```bibtex @misc{sauerkrautlm_glinerel_2026, title={SauerkrautLM-GLiNEREL: Joint Entity, PII and Relation Extraction in One Encoder Pass}, author={Michele Montebovi and Hakim Bedhief}, organization={VAGO Solutions}, url={https://huggingface.co/VAGOsolutions/SauerkrautLM-GLiNEREL-128m}, year={2026} } ``` Family: [SauerkrautLM-GLiNEREL-314m](https://huggingface.co/VAGOsolutions/SauerkrautLM-GLiNEREL-314m) · [SauerkrautLM-GLiNEREL-128m](https://huggingface.co/VAGOsolutions/SauerkrautLM-GLiNEREL-128m) · [SauerkrautLM-GLiNEREL-72m](https://huggingface.co/VAGOsolutions/SauerkrautLM-GLiNEREL-72m)