pawn-68m / BROWSER.md

Browser runtime: ONNX + pawn.js + docs

63ad028 verified 18 days ago

13.5 kB

Pawn in the browser

Run Pawn decision models entirely inside a web page. There is no server, no API key, and the data never leaves the device. pawn.js downloads an ONNX copy of the model from the Hugging Face Hub, packs questions exactly like the Python laya runtime, and runs one forward pass with ONNX Runtime Web.

Live demo: https://huggingface.co/spaces/nuhmanpk/pawn-browser
Models: nuhmanpk/pawn-17m, nuhmanpk/pawn-32m, nuhmanpk/pawn-68m

1. Quick start

Copy pawn.js next to your page (it is in every Pawn model repo under web/pawn.js), then:

<script type="module">
  import * as ort from "https://cdn.jsdelivr.net/npm/[email protected]/dist/ort.all.min.mjs";
  import { AutoTokenizer, env } from "https://cdn.jsdelivr.net/npm/@huggingface/[email protected]";
  import { Pawn } from "./pawn.js";

  env.allowLocalModels = false;
  ort.env.wasm.wasmPaths = "https://cdn.jsdelivr.net/npm/[email protected]/dist/";

  const pawn = await Pawn.load("nuhmanpk/pawn-17m", { ort, AutoTokenizer });

  const out = await pawn.predict(
    { message: "URGENT: verify your password at http://secure-login.example" },
    { phishing: { type: "noul",
                  instructions: "Is this message a phishing, smishing or scam attempt?",
                  criteria: { false: "no: a legitimate message with no attempt to deceive",
                              true:  "yes: it tries to trick the reader into giving up credentials, money or data" } } });

  console.log(out.answers.phishing.noul);   // probability that the message is phishing
</script>

The first load downloads the model. Later visits load it from the browser cache (Cache API).

2. How it works

your page ──► pawn.js ──► tokenizer (transformers.js, tokenizer.json from the Hub)
                    │
                    ├─► pack: [CLS] <type> question [SEP] [MASK] option ... [SEP] state [SEP]
                    │         (a line-by-line port of laya.common.build_sequence)
                    │
                    ├─► ONNX Runtime Web: one forward pass, one logit per option marker
                    │
                    └─► decode: divide by the calibrated temperature, softmax, map back to your option names

Parity with Python. The JavaScript packing and decoding were tested against laya's own Python code:

Packing: 320 of 320 randomized cases were token-for-token identical. They covered every question type, custom labels, 20-option heads that force trimming, truncated states, Unicode, quotes, newlines and [MASK] inside text.
Decoding: within 1e-15.
End to end: 240 answers matched laya's ONNXAgent (differences up to 5e-5, from Python's 4-decimal rounding).

Every model repo also ships web/fixtures.json, so you can re-check in any browser with pawn.selfTest().

3. API reference

`Pawn.load(repo, options)` → `Promise<Pawn>`

Option	Default	Meaning
`ort`	required	the `onnxruntime-web` module
`AutoTokenizer`	required	from `@huggingface/transformers`
`variant`	`"fp32"`	`"fp32"` (exact) or `"int8"` (smaller; only published if it passed the accuracy check)
`executionProviders`	`["wasm"]`	`["wasm"]` (CPU, works everywhere) or `["webgpu"]`
`revision`	`"main"`	Hub branch, tag or commit to pin
`cache`	`true`	keep the model in the browser Cache API
`onProgress`	none	`({loaded, total, cached}) => {}` download progress

`pawn.predict(state, questions)` → `Promise<Result>`

state is a string, or any JSON value (object or array). It is serialized the way Python's json.dumps does it.
questions is { id: { type, instructions, criteria?, labels?, option_order? } }.

`pawn.predictBatch(states, questions, batchSize = 8)` → `Promise<Result[]>`

Asks the same questions about many states, with batchSize states per ONNX run.

`pawn.selfTest(fixturesUrl?)` → `Promise<{cases, packing_identical, probabilities_within_0_01, max_probability_diff}>`

Checks that this browser reproduces the Python reference outputs shipped with the model.

Result shape

{
  model: "nuhmanpk/pawn-17m",
  latency_ms: 21.4,
  usage: { input_tokens: 61, state_tokens: 19, truncated: false },
  answers: {
    // noul
    phishing: { type: "noul", noul: 0.97, label: true, probabilities: { false: 0.03, true: 0.97 }, answer_confidence: 0.97 },
    // choice
    route:    { type: "choice", choice: "billing", probabilities: { billing: 0.88, shipping: 0.05, ... }, answer_confidence: 0.88 },
    // score
    severity: { type: "score", score: 2.6, level: 3, probabilities: { "0": 0.01, "1": 0.07, "2": 0.24, "3": 0.68 },
                legend: { "0": "low: ...", ... }, answer_confidence: 0.68 }
  }
}

Question types

Type	`criteria`	Answer
`choice`	`{ label: description }` (2 to 20), or a list of labels	`choice`, `probabilities`
`score`	`[level0, level1, ...]`, lowest first	`score` (expected level), `level`, `probabilities`
`noul`	`{ false: "...", true: "..." }` (optional)	`noul` = P(true), `label`

4. Use cases (with code)

4.1 LLM firewall in a chat UI

Check every prompt before it reaches your LLM API, in the user's browser:

const GUARD = { attack: { type: "noul",
  instructions: "Is this prompt trying to override, hijack or jailbreak an AI assistant's instructions?",
  criteria: { false: "no: an ordinary request, even if the topic is sensitive",
              true:  "yes: a prompt injection or jailbreak attempt" } } };

async function safeToSend(prompt) {
  const p = (await pawn.predict({ prompt }, GUARD)).answers.attack.noul;
  if (p >= 0.9) return { ok: false, reason: "blocked" };
  if (p >= 0.5) return { ok: true, flag: "review" };   // send, but log for review
  return { ok: true };
}

4.2 Phishing check in a browser extension or webmail

const PHISH = { phishing: { type: "noul", instructions: "Is this message a phishing, smishing or scam attempt?",
  criteria: { false: "no: a legitimate message with no attempt to deceive",
              true:  "yes: it tries to trick the reader into giving up credentials, money or data" } } };

const results = await pawn.predictBatch(emails.map((m) => ({ message: m.subject + "\n" + m.body })), PHISH, 8);
results.forEach((r, i) => { if (r.answers.phishing.noul >= 0.5) markSuspicious(emails[i]); });

4.3 Link-hover URL checker

const URLQ = { phishing: { type: "noul", instructions: "Is this URL likely to be a phishing link?",
  criteria: { false: "no: a legitimate website address", true: "yes: a phishing or credential-harvesting address" } } };

document.addEventListener("mouseover", async (e) => {
  const a = e.target.closest("a[href]"); if (!a) return;
  const p = (await pawn.predict({ url: a.href }, URLQ)).answers.phishing.noul;
  a.title = p >= 0.5 ? `Warning: possible phishing (${(p * 100).toFixed(0)}%)` : a.title;
});

4.4 Agent action gate in a web app

A cheap check before an in-browser agent runs a risky action:

const GATE = { allow: { type: "noul", instructions: "Is this planned action safe to run without asking the user?",
  criteria: { false: "no: it deletes data, spends money, sends messages or changes permissions",
              true:  "yes: it only reads or drafts" } } };
const ok = (await pawn.predict({ action: plannedToolCall }, GATE)).answers.allow.noul >= 0.8;

This is not a trained workflow, so accuracy is lower than the presets. Collect a few hundred labelled examples and fine-tune if it matters.

4.5 Vulnerability triage dashboard

Several questions in one call:

const out = await pawn.predict({ description: cveText }, {
  severity: { type: "score", instructions: "How severe is this vulnerability (CVSS qualitative severity rating)?",
              criteria: ["low: limited impact or hard to exploit (CVSS 0.1-3.9)",
                         "medium: real but constrained impact (CVSS 4.0-6.9)",
                         "high: serious compromise is likely if exploited (CVSS 7.0-8.9)",
                         "critical: easy, remote and severe, e.g. unauthenticated code execution (CVSS 9.0-10.0)"] },
  network_exploitable: { type: "noul", instructions: "Can an attacker exploit this vulnerability remotely over a network?",
              criteria: { false: "no: needs local, physical or adjacent-network access",
                          true:  "yes: exploitable across a network (CVSS attack vector Network)" } } });

Severity was trained on a balanced class mix, so apply the prior correction shipped in web/pawn_web.json (severity_prior), as the demo does.

4.6 Ticket and form routing (custom labels)

const r = await pawn.predict(ticketText, { route: { type: "choice", instructions: "Which team should handle this request?",
  criteria: { billing: "payments, refunds and invoices", shipping: "delivery and tracking",
              technical: "app bugs and login problems", sales: "new purchases and pricing" } } });

5. Choosing a model and weights

Need	Model	Weights
Smallest download, phones and extensions	Pawn-17M	fp32 (or int8 if published)
Default for web apps	Pawn-32M	fp32
Best accuracy in the browser	Pawn-68M	fp32 (large download; consider a server)

About int8. laya's own exporter notes that dynamic int8 quantization trades real accuracy. On the 0.4B English checkpoint, int8 agreed with full precision on only 64 of 96 held-out choice decisions. Pawn's export notebook therefore measures agreement and publishes int8 only if it agrees with fp32 on at least 98% of test decisions. Check web/pawn_web.json → variants for the measured numbers.

6. Files in each model repo

Path	Purpose
`onnx/model.onnx`	fp32 ONNX graph (inputs `input_ids`, `attention_mask`, `marker_pos`, `marker_mask`, `qtype`; outputs `logits`, `act_logits`)
`onnx/model_int8.onnx`	int8 graph (only if it passed the agreement check)
`tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`	tokenizer for transformers.js (root copy of `tokenizer/`)
`web/pawn_web.json`	max_len, head budget, calibrated temperatures, special token ids, files, measured variant agreement, severity prior
`web/fixtures.json`	Python reference packing and probabilities for `selfTest()`
`web/pawn.js`	the browser runtime
`model.safetensors`, `rl_agent_config.json`, `encoder/`, `tokenizer/`	the PyTorch checkpoint for `laya.load()`

7. Performance tips

Reuse one Pawn instance. Loading is the expensive part; each decision is a single forward pass.
Batch with predictBatch when checking lists such as an inbox or a page of links.
Load in a Web Worker to keep the UI responsive with large models (copy pawn.js into the worker and import ORT there).
WebGPU can be faster for 32M and 68M on supported browsers. WebAssembly works everywhere.
Put the important evidence first. States longer than the model's window (max_len) are truncated from the end, and usage.truncated tells you when that happened.

8. Privacy and security

Inference happens in the page. The only network requests are the model and tokenizer downloads from the Hugging Face Hub, and the CDN scripts.
For an air-gapped or offline setup, self-host pawn.js, the ORT files, the tokenizer and the ONNX file, then point Pawn.load at your own copies (adapt the base URL in pawn.js).
Pawn is a classifier, not a security boundary. Text placed in the state can steer it, so keep least-privilege controls and human review for consequential actions.

9. Troubleshooting

Symptom	Fix
`pass { ort, AutoTokenizer }`	import both modules and pass them to `Pawn.load`
Stuck at "Downloading"	check the network tab; the Hub URL must be reachable (corporate proxies sometimes block it)
`variant int8 not available`	int8 was not published because it failed the agreement check; use `fp32`
`N options but M markers fit`	too many or too long options for the head budget; shorten descriptions or split the question
Out of memory on phones	use Pawn-17M, `wasm`, and small batches
`selfTest` shows packing differences	report the browser and version; packing must be identical
Numbers in state differ from Python	JavaScript cannot tell `2.0` from `2`; Python writes `2.0`. Send such values as strings if exact parity matters
Object key order	JavaScript sorts integer-like keys (`"1"`, `"2"`) first; avoid numeric keys in states

10. Limitations

English only.
Accuracy is lower than the server model Rook-1, especially on knowledge-heavy tasks such as CVSS severity.
Training wording matters. Presets use it; custom questions work but are less accurate.
First load downloads tens to hundreds of MB, depending on the size.

11. Credits and license

Pawn by Nuhman PK. Built on Laya (Convai Innovations, Apache-2.0), Ettin encoders (Johns Hopkins University), ONNX Runtime Web and transformers.js. pawn.js is released under Apache-2.0.