Papers
arxiv:2611.16933

Kezek: Splitting Mono Two-Person Conversations into Per-Speaker Tracks for Full-Duplex Dialogue Data

Published on Nov 12
· Submitted by
Bagzhan Karl
on Nov 19
Authors:

Abstract

Kezek separates a mono two-speaker conversation into per-speaker tracks while preserving timing and short replies, using a phase-insensitive speaker-fingerprint-guided two-voice activity model followed by a conditioned time-domain separator, and it improves short-reply retention, leakage, channel-swap, and runtime metrics over an open baseline.

Full-duplex spoken dialogue models learn turn-taking, overlaps and backchannels from conversations recorded with one channel per speaker, but most available conversational audio is a single mono mix. We present Kezek, a model that turns a mono recording of exactly two people into one track per person with the original length and timing; overlaps and short replies ("yeah", "mm-hm") stay on the track of the person who said them. Kezek decides who speaks when and who is who once for the whole recording (stage 1: a two-voice activity model guided by a speaker fingerprint trained to be insensitive to phase), and a conditioned time-domain separator only carries out that decision (stage 2), with outputs that always sum to the input. Standard separation metrics miss the failures that matter for dialogue data, so we report short-reply retention, leakage onto the other person's solo, channel swaps and coverage. Against the open pipeline pyannote -> DialogueSidon on the same recordings, Kezek keeps more short replies (0.83 vs. 0.69 on Open Yap test; 0.80 vs. 0.71 over the other's speech on unseen telephone speech from CallHome English), mostly because it returns the whole recording while the external pipeline drops part of it (on the audio both return, reply retention is about the same: 0.859 vs. 0.865 on Open Yap); it also leaks less, swaps channels less often, and runs 4-5 times faster (about 230 hours of audio per GPU-hour). We also report negative results and an ideal-mask ceiling that locate the remaining gap: quiet replies under a louder voice. Code and weights are released under permissive licences.

Community

Paper author Paper submitter

Kezek splits a mono recording of two people into one track per speaker, same length and timing. Overlaps and short replies ("yeah", "mm-hm") stay on the right person's track, and the two tracks sum exactly to the input.

We built it to get two-channel conversation data for full-duplex voice models, since almost all real conversations online exist only as a mono mix.

On Open Yap test (121 real conversations), compared to pyannote → DialogueSidon (DuplexChat pipeline, run unchanged): 0.83 vs 0.69 of short replies kept, 0.93 vs 0.79 recordings without swapped channels, 231 vs 51 hours of audio per GPU-hour.

41M params, weights Apache-2.0, code MIT.
Demo: https://huggingface.co/spaces/manifestro/kezek
Weights: https://huggingface.co/manifestro/kezek-v1

•
This comment has been hidden (marked as Resolved)

Sign up or log in to comment

Models citing this paper 1

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.16933 in a dataset README.md to link it from this page.

Spaces citing this paper 2

Collections including this paper 1