Borcherding/voice-decisions-laya-v4

Laya voice decisions v4

One Laya decision model that makes the three fast calls voice mode needs in Unsloth Studio and Unsloth Desktop, in a single forward pass each:

question	type	asks	answers
turn_done	yes/no (noul)	has the user finished talking, so the assistant should answer now?	probability of yes
filler	choice, 12 classes	which short line to say while the assistant thinks, or none	none, minimal, task, code, explain, plan, creative, compare, transform, advice, opinion, correction
interject	choice, 4 classes	the user spoke while the assistant was still thinking: what to do with it	amend, replace, queue, ignore

Full fine-tune of convaiinnovations/laya (ModernBERT-large encoder plus a calibrated decision head; it classifies, it does not generate) on Borcherding/voice-decisions-v4 with Unsloth Studio's decision trainer. One resident model replaces separate end-of-turn, filler and barge-in classifiers.

Uploaded finetuned model

• Developed by: Borcherding
• License: apache-2.0 (same as the base model)
• Finetuned from model: convaiinnovations/laya

Usage

In Unsloth Studio / Unsloth Desktop voice mode

Voice mode with a pluggable decision model ships in unslothai/unsloth#12384 (open as of 2026-10-19). On a build with it:

1. Download the repo into Unsloth Studio's outputs folder. The folder name becomes the model name laya-ft:<folder>.

   hf download Borcherding/voice-decisions-laya-v4 --local-dir ~/.unsloth/studio/outputs/voice-decisions-laya-v4
   

2. Pick it: in a chat, open the Voice pill, then Decide, and choose voice-decisions-laya-v4 under Fine-tuned. The switch loads in the background and the next turn uses it. Only the installation owner can change it.

   Headless alternative: export UNSLOTH_VOICE_DECISION_CHECKPOINT=laya-ft:voice-decisions-laya-v4 before starting Unsloth Studio. The picker overrides the variable.

Voice mode then asks this model for the end-of-turn verdict, the filler class and the interject action. amend and replace both restart the reply (with the original request kept or dropped); queue and ignore do what they say.

Through the Decision API

With the Decision API on (Settings > API), any client can call POST /v1/systemone. The model answers three questions, each asked with the exact wording it was trained on (the same text is in every dataset row and in Unsloth Studio's studio/backend/core/inference/voice_decisions.py). Change the wording and accuracy drops.

question	type	state	answer
turn_done	noul	the transcript so far	noul: probability they have finished (0 to 1)
filler	choice, 12 classes	the finished request, with (the assistant just asked them a question) on the line before it when that is true	choice, confidence, probabilities
interject	choice, 4 classes	Request: <request> then Spoken over the thinking (N seconds into thinking): <utterance>; the seconds part is optional	choice, confidence, probabilities

import requests

URL = "http://localhost:8888/v1/systemone"
HEADERS = {"Authorization": "Bearer <your Unsloth Studio API key>"}
MODEL = "laya-ft:voice-decisions-laya-v4"

# 1. turn_done: has the person finished speaking? Answers one probability ("noul").
TURN_DONE = {"turn_done": {
    "type": "noul",
    "instructions": "The state is what a person has said so far to a voice assistant, as speech "
                    "recognition wrote it. Have they finished what they wanted to say, so the "
                    "assistant should answer now, rather than paused mid-thought?",
}}

# 2. filler: which short line, if any, to say aloud while the assistant thinks.
FILLER = {"filler": {
    "type": "choice",
    "instructions": "The state is a finished request to a voice assistant, as speech recognition "
                    "wrote it. The assistant may say one short line aloud while it thinks. Which "
                    "kind of line fits, or none?",
    "criteria": {
        "minimal": "a medium-length ask: a sentence is too much, silence is wrong. one or two words.",
        "task": "a concrete do-this request (write, make, draft, list) that is big enough to think about",
        "code": "programming: write, debug, or explain code, scripts, errors",
        "explain": "a how/why/what-is question that needs a real explanation",
        "plan": "planning, scheduling, step-by-step organizing, itineraries, roadmaps",
        "creative": "poems, stories, jokes, songs, scripts, names, slogans",
        "compare": "compare, pros and cons, which is better, recommend between options",
        "transform": "summarize, rewrite, translate, proofread, extract, classify given text",
        "advice": "personal situation plus what should I do, help me decide, is this normal",
        "opinion": "open-ended or philosophical, what do you think, is it ethical",
        "correction": "the user says the last answer was wrong, not what they asked, redo it",
        "none": "say nothing: greetings, one-word replies, yes/no, control commands, quick lookups, "
                "answers to the assistant's own question",
    },
}}

# 3. interject: the person spoke while the assistant was still thinking. What now?
INTERJECT = {"interject": {
    "type": "choice",
    "instructions": "A voice assistant is still thinking about the request and has not answered yet. "
                    "The person spoke again. What should the assistant do with what they said?",
    "criteria": {
        "amend": "it adds to or adjusts the request: keep the request, fold this in and start over",
        "replace": "it cancels the request or changes direction: drop the request and start over with only this",
        "queue": "it is a separate request: hold it and answer it after this reply",
        "ignore": "a backchannel, self-talk or someone else in the room: nothing to act on",
    },
}}


def ask(state, questions):
    r = requests.post(URL, headers=HEADERS, json={"model": MODEL, "state": state, "questions": questions})
    r.raise_for_status()
    return r.json()["answers"]


# turn_done: the state is the transcript so far.
print(ask("can you tell me how to", TURN_DONE)["turn_done"]["noul"])                  # low: mid-thought
print(ask("what's the weather like in Paris today", TURN_DONE)["turn_done"]["noul"])  # high: finished

# filler: the state is the finished request.
print(ask("write me a short poem about the ocean", FILLER)["filler"]["choice"])        # creative
# When the assistant's last turn asked them something, put this line first:
print(ask("(the assistant just asked them a question)\nyes please", FILLER)["filler"]["choice"])  # none

# interject: the original request, then what was said over the thinking.
# "(N seconds into thinking)" is optional; leave it out when you don't know.
request = "write me a snake game in python"
for spoken in ["and add a score counter",              # amend
               "never mind, tell me a joke instead",   # replace
               "also what's the weather tomorrow",     # queue
               "mm-hm"]:                               # ignore
    state = f"Request: {request}\nSpoken over the thinking (2 seconds into thinking): {spoken}"
    a = ask(state, INTERJECT)["interject"]
    print(spoken, "->", a["choice"], a["confidence"], a["probabilities"])

What this model answered for those states (Unsloth Studio on an Apple M5, 2026-10-20). turn_done shows the finished probability; the others show the choice and its confidence:

question	state	answer
turn_done	can you tell me how to	finished 0.002
turn_done	what's the weather like in Paris today	finished 0.80
filler	write me a short poem about the ocean	creative 1.00
filler	why is the sky blue	explain 0.73
filler	fix this python error: list index out of range	code 0.99
filler	thanks	none 0.97
filler	(the assistant just asked them a question) yes please	none 0.99
interject	and add a score counter	amend 0.99
interject	never mind, tell me a joke instead	replace 1.00
interject	also what's the weather tomorrow	queue 0.99
interject	mm-hm	ignore 0.98

Results

Two held-out sets, scored with decide() one decision per row, never trained on:

• Synthetic eval: a 1,499-row stratified sample of the dataset's eval split (493 turn_done, 470 filler, 536 interject), in the style of speech-recognition output.
• Real speech: 1,318 real utterances to a voice assistant and a coding agent, redacted (716 turn_done, 580 filler, only 22 interject).

set	all	turn_done	filler	interject
synthetic eval	94.2% (1,499)	93.9%	91.5%	96.8%
real speech	82.2% (1,318)	89.9%	73.1%	72.7% (22 rows)

The trainer's own end-of-run check on 2,000 held-out decisions: base Laya 49.0% accuracy, ECE 0.089; this model 93.7%, ECE 0.015.

interject holds 96 to 98% recall on every class of the synthetic set. On real speech the weak spot is filler: minimal (23%), advice (25%), opinion (33%) and correction (41%) are often answered none or a neighbouring class. For voice mode that mostly means a missing or generic thinking line, not a wrong answer.

Latency

p50	p95	max
102 ms	198 ms	238 ms

Measured on an Apple M5 (macOS 26.6), PyTorch CPU, 3 threads, model.safetensors, one question per call, 99 warm calls on real-speech rows. The first call loads the model and took 5.5 s.

Where it fails

Probe phrases: 14 of 16 right. Known wrong calls, all from probes or live use:

utterance	expected	got
"wait no, make it about Greece instead"	replace	amend
"What's the capital of France?"	filler none	minimal
"Can I interrupt you right now?", "What's going on right now?"	replace	ignore
"Why did you stop?"	replace	queue
speech-recognition-garbled interruptions aimed at the assistant	replace	ignore

• Questions or complaints aimed at the assistant mid-thought ("why did you stop", "are you listening") are under-represented and get ignored or queued.
• "wait no, X instead" direction changes read as amend.
• On real speech, queue and replace have 2 rows each and both were missed, so interject on real speech is a spot check, not a measurement.

Training

data	37,514 train rows (13,358 turn_done, 12,444 filler, 11,712 interject); interject classes balanced
method	full fine-tune, soft cross-entropy, Unsloth Studio decision trainer
steps	587 (1 epoch), batch 64
hardware	1 x NVIDIA A100 40GB (Colab), 601 s training, 8.3 GB peak memory
loss	1.445 first step; eval loss 0.323, 0.205, 0.193, 0.179 at each quarter epoch
calibration	per-question-type temperatures fitted after training (in rl_agent_config.json)

Eval loss was still going down at the end of the epoch, so a slightly longer run may help; the bigger lever is the data gaps listed above.

Files

file	size	what
model.safetensors	843 MB	fine-tuned weights (PyTorch)
rl_agent_config.json	1 KB	Laya config: encoder, head, calibration temperatures, training record
encoder/config.json	2 KB	ModernBERT-large config
tokenizer/	3.6 MB	tokenizer.json, tokenizer_config.json
gguf/model-Q8_0.gguf	449 MB	Q8_0 GGUF for the llama.cpp path
gguf/export.json	297 B	Unsloth decision GGUF manifest (layout: laya; no mmproj, text only)
charts/		the charts above

Limitations

• English only.
• About 85% of training rows are synthetic text in the style of speech-recognition output, not transcribed audio. The real rows come from a small number of speakers and lean toward software work, so expect lower filler accuracy on other domains.
• Labels follow a written rubric with scripted and spot-checked audits, not row-by-row human review.
• Filler lines never imply yes or no; the model only picks the kind of line, the phrase itself comes from voice mode's phrase bank.

This Laya model was trained with Unsloth.