docs: polish model card — link dataset, add limitations, raw ONNX snippet, label-index callout

README.md

@@ -3,35 +3,43 @@ language: en
 license: apache-2.0
 library_name: transformers
 base_model: cross-encoder/nli-MiniLM2-L6-H768
+datasets:
+  - labrat-aiko/popia-compliance-nli
 tags:
   - nli
   - zero-shot-classification
   - compliance
   - popia
   - south-africa
+  - privacy
+  - legal
   - onnx
   - quantized
+  - cpu
 pipeline_tag: zero-shot-classification
 ---
 
 # nli-popia-v1
 
-Cross-encoder NLI model fine-tuned for semantic validation of LLM outputs against
-**POPIA** (South Africa's Protection of Personal Information Act, 2013) compliance
-clauses. Used by the `POPIAJudge` in [semantix-ai](https://github.com/labrat-akhona/semantix-ai).
+Cross-encoder NLI model fine-tuned for deterministic, local, auditable
+validation of LLM outputs against **POPIA** (South Africa's Protection of
+Personal Information Act, 2013) compliance clauses. Drop-in judge for the
+`POPIAJudge` in [semantix-ai](https://github.com/labrat-akhona/semantix-ai).
 
-- **Base model:** `cross-encoder/nli-MiniLM2-L6-H768` (~22M params)
-- **Fine-tuning:** 180 hand-authored premise/hypothesis/label triples (60 seeds + 120 paraphrases)
-- **Evaluation:** 150 pinned holdout pairs, 7 canonical POPIA clauses
-- **Deployment:** INT8 ONNX, ~15 ms CPU inference, 4 CPU-architecture variants
+- **Base:** `cross-encoder/nli-MiniLM2-L6-H768` (~22M params)
+- **Training data:** 180 hand-authored triples → [`labrat-aiko/popia-compliance-nli`](https://huggingface.co/datasets/labrat-aiko/popia-compliance-nli)
+- **Evaluation:** 150-pair pinned holdout, hash-verified by release gate
+- **Deployment:** 4 INT8 ONNX CPU variants (AVX2 / AVX512 / AVX512-VNNI / ARM64), auto-selected
+- **Inference:** ~70 ms mean on CPU, $0 API cost, no internet required
+- **Release gate:** macro F1 delta ≥ 0.10 AND no per-clause regression — enforced in CI on every push to master
 
-## Results
+## Results (150-pair pinned holdout)
 
-Evaluated on the pinned `eval.jsonl` (150 pairs) against the stock base model as baseline:
+Fine-tuned checkpoint versus the stock base model:
 
 | Metric                              | Stock NLI | POPIA fine-tune | Delta       |
 |-------------------------------------|-----------|-----------------|-------------|
-| Macro F1                            | 0.517     | **0.813**       | **+0.296**  |
+| **Macro F1**                        | 0.517     | **0.813**       | **+0.296**  |
 | Accuracy                            | 0.707     | **0.833**       | +0.127      |
 | POPIA consent                       | 0.612     | 0.902           | +0.290      |
 | POPIA minimality                    | 0.646     | 0.753           | +0.106      |
@@ -41,21 +49,9 @@ Evaluated on the pinned `eval.jsonl` (150 pairs) against the stock base model as
 | POPIA general processing            | 0.400     | 0.893           | +0.493      |
 | POPIA data subject rights           | 0.400     | 0.883           | +0.483      |
 
-**Release gate:** macro delta ≥ 0.10 AND no per-clause regression — **PASS**.
+Every clause improved. No regressions. Release gate: **PASS**.
 
-## Labels
-
-Label order matches the base model's `config.id2label`:
-
-| Index | Label          |
-|-------|----------------|
-| 0     | contradiction  |
-| 1     | entailment     |
-| 2     | neutral        |
-
-The `POPIAJudge` reads `probs[1]` (entailment) as the "compliance matches intent" score.
-
-## Usage
+## Quick start
 
 ```bash
 pip install 'semantix-ai[popia]'
@@ -72,8 +68,49 @@ verdict = judge.evaluate(
     threshold=POPIAJudge.recommended_threshold,  # 0.75
 )
 print(verdict.passed, verdict.score)
+# False 0.031
+```
+
+Using the raw ONNX directly (no PyTorch, no semantix dependency):
+
+```python
+import numpy as np, onnxruntime as ort
+from tokenizers import Tokenizer
+from huggingface_hub import hf_hub_download
+
+model_path = hf_hub_download("labrat-aiko/nli-popia-v1", "onnx/model_quint8_avx2.onnx")
+tok_path   = hf_hub_download("labrat-aiko/nli-popia-v1", "tokenizer.json")
+session = ort.InferenceSession(model_path)
+tok = Tokenizer.from_file(tok_path)
+
+encoded = tok.encode(
+    "Your data will be used to generate marketing offers without opt-in.",
+    "The responsible party is obtaining explicit opt-in consent.",
+)
+logits = session.run(None, {
+    "input_ids": np.array([encoded.ids], dtype=np.int64),
+    "attention_mask": np.array([encoded.attention_mask], dtype=np.int64),
+})[0][0]
+probs = np.exp(logits) / np.exp(logits).sum()
+# Label order (id2label): {0: contradiction, 1: entailment, 2: neutral}
+print("entailment:", probs[1])
 ```
 
+## Label order (read this before using the scores)
+
+Label indices match the base model's `config.id2label`:
+
+| Index | Label          |
+|-------|----------------|
+| 0     | contradiction  |
+| 1     | **entailment** |
+| 2     | neutral        |
+
+`POPIAJudge` (semantix-ai ≥ 0.2.0) reads `probs[1]` as the compliance score.
+A Python-side bug in v0.1.5–v0.1.13 of semantix-ai was reading `probs[2]`
+(neutral) — fixed in v0.2.0. The ONNX artefacts in this repo are unchanged;
+only downstream Python code needed the label index correction.
+
 ## Files
 
 | Path                                     | Purpose                                                   |
@@ -84,23 +121,67 @@ print(verdict.passed, verdict.score)
 | `onnx/model_qint8_arm64.onnx`            | INT8 quantized, ARM64 (Apple Silicon, Graviton)           |
 | `tokenizer.json`                         | Rust `tokenizers` fast-tokenizer config                   |
 | `config.json`                            | Model config (shared across variants)                     |
-| `eval.jsonl`                             | Pinned 150-pair holdout set (bundled for reproducibility) |
+| `eval.jsonl`                             | Pinned 150-pair holdout (bundled for reproducibility)     |
 
-The judge auto-detects the best variant for the host CPU; consumers never pick manually.
+`POPIAJudge` auto-detects the best variant at load time; you never pick manually.
 
 ## Training
 
 - 5 epochs, batch size 8, lr 2e-5, warmup 10%, weight decay 0.01
-- Cross-entropy loss, early stopping on `eval_loss` against a 10% dev split
+- Cross-entropy loss, early stopping on `eval_loss` against 10% dev split
 - CPU training on 180 rows: ~6 min
-- Reproducible via `scripts/train_popia.py` in the semantix-ai repo
+- Reproducible via [`scripts/train_popia.py`](https://github.com/labrat-akhona/semantix-ai/blob/master/scripts/train_popia.py)
 
 ## Intended use
 
-Compliance-sensitive audit of LLM outputs where a deterministic, auditable,
-non-API-dependent judge is required. Not a legal opinion — flags semantic
-mismatch between text and a clause intent.
+- Compliance-sensitive audit of LLM outputs where a deterministic,
+  auditable, non-API-dependent judge is required (e.g. CI validation,
+  audit trails, POPIA-aware `@validate_intent` decorators, MCP tools).
+- Per-clause threshold-tunable gating: each of the 7 POPIA clauses has
+  a pre-tuned F1-optimal threshold shipped in
+  `semantix.presets.popia` — you can override per-deployment.
+- Educational / research benchmark for local NLI approaches to
+  compliance judging versus LLM-as-judge pipelines.
+
+## Limitations
+
+- **Does not replace a POPIA specialist, a DPIA, or the Information
+  Regulator.** Flags semantic mismatch between text and a clause intent;
+  not a legal opinion.
+- **English-only.** No Afrikaans / isiZulu / Sesotho coverage yet.
+- **Narrow domain.** 7 canonical clauses out of POPIA's full surface; some
+  adjacent areas (direct marketing under §69, automated decision-making
+  under §71) are not separately modelled.
+- **Hand-authored eval set** (same author as train). External validation
+  is welcome and the eval set is Apache 2.0 so you can add your own cases.
+- **Threshold-sensitive.** Per-clause F1-optimal thresholds are tuned on
+  *this* holdout. If your domain's distribution differs (dialect,
+  adversarial inputs, industry jargon), re-tune on your own held-out
+  set — the full recipe is in the semantix-ai repo.
+
+## Ecosystem
+
+- **Training dataset**: [`labrat-aiko/popia-compliance-nli`](https://huggingface.co/datasets/labrat-aiko/popia-compliance-nli) (Apache 2.0)
+- **Library**: [`semantix-ai`](https://pypi.org/project/semantix-ai/) (MIT), ships `POPIAJudge` + 7 presets
+- **Source**: [labrat-akhona/semantix-ai](https://github.com/labrat-akhona/semantix-ai)
+- **Writeup**: [I Fine-Tuned a Compliance Judge and Beat the Stock Model by +29.6pp F1](https://github.com/labrat-akhona/semantix-ai/blob/master/articles/devto-popia-finetune.md)
+
+## Citation
+
+```bibtex
+@misc{nli-popia-v1-2026,
+  author       = {Eland, Akhona},
+  title        = {nli-popia-v1: A local NLI cross-encoder fine-tuned for POPIA compliance validation},
+  year         = 2026,
+  publisher    = {Hugging Face},
+  howpublished = {\url{https://huggingface.co/labrat-aiko/nli-popia-v1}},
+  note         = {Apache 2.0}
+}
+```
 
 ## License
 
-Apache 2.0.
+Apache 2.0. Commercial and derivative use explicitly permitted. Forking
+to build sibling compliance models (GDPR, HIPAA, EU AI Act, UK DPA) is
+actively encouraged — same recipe, different clause corpus. A link back is
+appreciated but not required.