---
license: apache-2.0
base_model: convaiinnovations/laya-multilingual
tags:
  - ggmlc
  - gguf
  - laya
  - jev
  - mmbert
  - multilingual
  - decision
  - system-1
language:
  - multilingual
---

# Laya Multilingual GGUF (ggmlc)

**Multilingual System 1 decision model** compiled from [convaiinnovations/laya-multilingual](https://huggingface.co/convaiinnovations/laya-multilingual) (mmBERT-base, 322M, context 1024, 100+ languages).

These files are **not** llama.cpp / `llama-cli` GGUFs. They are produced by **[ggmlc](https://github.com/monatis/ggmlc)**, a neural network compiler that lowers PyTorch, JAX, Flax, and Keras models to high-performance GGML execution. Loading them in llama.cpp will fail.

Typed questions (`choice` / `score` / `noul`) are scored in **one encoder pass**. There is no autoregressive token generation.

Source, CLI, and binaries: **[examples/laya](https://github.com/monatis/ggmlc/tree/main/examples/laya)**

Other families: [laya-GGUF](https://huggingface.co/mys/laya-GGUF) (English) · [laya-typed-decisions-GGUF](https://huggingface.co/mys/laya-typed-decisions-GGUF)

## Files

| File | Quant | Size | Notes |
| :--- | :--- | ---: | :--- |
| `laya_multilingual_f16.gguf` | F16 | ~633 MB | Full precision. |
| `laya_multilingual_q8_0.gguf` | Q8_0 | ~345 MB | Usually the best size / quality trade. |
| `laya_multilingual_ud_q4_k_m.gguf` | UD_Q4_K_M | ~500 MB | Larger than Q8_0 here because embeddings stay F16. |

Tokenizer is **Gemma BPE + Metaspace** (`▁`), specials `<bos>/<eos>/<pad>/<mask>` (ids 2/1/0/4), not ModernBERT `[CLS]/[SEP]/[MASK]`.

```bash
huggingface-cli download mys/laya-multilingual-GGUF laya_multilingual_f16.gguf --local-dir .
```

## Run with `laya`

Download a binary from [ggmlc releases](https://github.com/monatis/ggmlc/releases/latest) (`laya.exe` / `laya`). `--device` defaults to `auto` (CUDA or Metal if present, else CPU).

```bash
laya help
laya list-presets
laya info laya_multilingual_f16.gguf

laya decide laya_multilingual_f16.gguf --preset email --text "二重に請求されました" --device auto --cuda-graph
laya decide laya_multilingual_f16.gguf --preset triage --text "Bitte erstatten Sie die doppelte Abbuchung." --json
laya serve laya_multilingual_f16.gguf --port 8080 --device auto --cuda-graph
laya bench laya_multilingual_f16.gguf --preset email --device auto --cuda-graph
```

Put English + multilingual GGUFs in one directory to route before the forward (non-Latin script → multilingual; otherwise English function-word count):

```bash
laya detect-lang --text "二重に請求されました"
laya decide --models-dir . --preset email --text "二重に請求されました"
```

`serve` starts Decision Studio (`GET /`) and `POST /api/decide`. `daemon` is newline JSON-RPC on stdin/stdout.

## What this is

[Laya](https://github.com/NandhaKishorM/laya) is the open reproduction of TypeSafe **Jev**: given a *state* and typed questions, it returns calibrated probabilities instead of generating tokens. The English checkpoint stays confident while collapsing off English; use this family for non-English (or mixed) text.

## License

Apache 2.0, same as the upstream Laya weights. Compiler: [ggmlc](https://github.com/monatis/ggmlc) (MIT).