yigagilbert/stepaudio2-mini-luganda-english-bidirectional-s2st-lora

Step-Audio 2 Mini Luganda-English Bidirectional S2ST LoRA

This repository contains a LoRA adapter for stepfun-ai/Step-Audio-2-mini trained with direction prompts for both Luganda-to-English and English-to-Luganda speech-to-speech translation.

Adapter repository: yigagilbert/stepaudio2-mini-luganda-english-bidirectional-s2st-lora

Intended Use

Research and development for bidirectional Luganda-English speech translation. Validate with native speakers before production use.

Training Data

The adapter was trained from yigagilbert/luganda-english-cleaned-v1-split by expanding each aligned pair into two supervised examples:

• Luganda speech input -> English text and English speech tokens
• English speech input -> Luganda text and Luganda speech tokens

Each example uses a direction-specific system prompt so that one adapter can learn both translation directions.

Evaluation

The checkpoint uploaded here was evaluated on 200 held-out validation examples per direction. Text metrics are computed on the generated text channel; audio samples are synthesized from generated Step-Audio tokens with the upstream Token2wav decoder.

Text Metrics

Direction	System	Count	BLEU higher	chrF higher	WER lower	COMET higher
Luganda -> English	Focused one-direction LoRA	200	32.530	54.535	0.574	0.717
Luganda -> English	This bidirectional LoRA	200	19.032	42.447	0.737	0.640
English -> Luganda	Focused one-direction LoRA control	200	0.120	14.050	1.300	-
English -> Luganda	This bidirectional LoRA	200	11.155	45.331	0.811	0.678
English -> Luganda	ASR + MT cascade baseline	200	4.026	34.734	2.039	0.624

Speech Metrics

Speech metrics were computed on 20 audio-aligned samples per direction using WavLM-large SpeechBERTScore-style similarity and MFCC+DTW MCD. These should be treated as supporting audio-channel diagnostics, not as standalone translation-quality measures.

Direction	Count	SpeechBERT P higher	SpeechBERT R higher	SpeechBERT F1 higher	MCD lower
Luganda -> English	20	0.560	0.582	0.570	639.360
English -> Luganda	20	0.502	0.572	0.523	614.130

Retention and Composition Probes

The bidirectional adapter does not recover the base model's unrelated multilingual speech translation ability on small FLEURS probes: Spanish -> English BLEU is 0.293 and Japanese -> English BLEU is 0.163 on 20 samples each. However, it does recover target-script compliance on Luganda -> Chinese and Luganda -> Japanese prompt probes (100% target-script hit rate in both cases). The semantic quality of those composition outputs is poor, with roundtrip BLEU below 1, so these probes should not be read as successful Luganda -> Chinese/Japanese translation.

Compared with the earlier Luganda-to-English-only LoRA adapter, this bidirectional adapter adds English-to-Luganda capability, but Luganda-to-English quality is lower than the specialized one-direction adapter. The bidirectional run was also shorter (9,000 steps) than the focused adapter run, so the gap combines multi-task interference and training duration.

Loading

Load this as a PEFT adapter on top of stepfun-ai/Step-Audio-2-mini.

Limitations

• Automatic metrics for Luganda can understate useful translations because of spelling and morphology variation.
• Speech quality should be checked by listening to synthesized samples.
• This adapter is a research artifact and should not be used for high-stakes translation without human validation.

License

Adapter code and metadata are Apache-2.0. Check the dataset and base model licenses separately before redistribution.