prvn-ramesh/gliner-small-v2.5-onnx

GLiNER Small v2.5 (ONNX Runtime)

Export of gliner-community/gliner_small-v2.5 for high-performance CPU/GPU inference with ONNX Runtime.

Exported and validated using pii-protect[onnx] for low-latency PII masking and named entity recognition.

Files

File	Precision	Size	Description
model.onnx	FP32	~583 MB	Full Precision: 100% detection recall parity with PyTorch, 1.78x faster CPU inference.
model_quantized.onnx	Dynamic INT8	~187 MB	Quantized: 2.85x faster CPU inference with smaller memory footprint.
gliner_config.json	-	-	Model configuration and label space metadata.
tokenizer.json	-	-	Fast HuggingFace tokenizer.
tokenizer_config.json	-	-	Tokenizer configuration.

Benchmark (53-Category Banking PII Benchmark)

Evaluated across 530 conversations (1,600 turns, 1,380 planted PII values):

• PyTorch Baseline: 70.87% Recall (978/1,380), 359 FP, 167 ms/turn
• ONNX FP32 (model.onnx): 70.87% Recall (978/1,380), 359 FP, 94 ms/turn (1.78x faster)
• ONNX INT8 (model_quantized.onnx): 53.12% Recall (733/1,380), 167 FP, 59 ms/turn (2.85x faster)

Usage in Python (with gliner)

from gliner import GLiNER

# Load full precision FP32 ONNX
model = GLiNER.from_pretrained(
    "prvn-ramesh/gliner-small-v2.5-onnx",
    load_onnx_model=True,
    onnx_model_file="model.onnx",
)

text = "Hi, my name is John Doe and my phone number is 9876543210"
labels = ["person", "phone number", "email", "address"]
entities = model.predict_entities(text, labels, threshold=0.5)

for entity in entities:
    print(entity["text"], "=>", entity["label"], f"({entity['score']:.2f})")

Usage in Python (with pii-protect)

from pii_protect import NEREngine

engine = NEREngine(
    enable_gliner=True,
    gliner_onnx_dir="path/to/downloaded_dir",
    gliner_onnx_file="model.onnx",  # use 'model.onnx' for FP32 or 'model_quantized.onnx' for INT8
)

entities = engine.detect("Hi my name is John Doe")