---
library_name: vllm
language:
- en
- fr
- es
- pt
- it
- nl
- de
- ar
- hi
license: cc-by-nc-4.0
inference: false
base_model:
- mistralai/Ministral-3-3B-Base-2512
extra_gated_description: >-
If you want to learn more about how we process your personal data, please read
our Privacy Policy.
tags:
- mistral-common
pipeline_tag: text-to-speech
---
# Voxtral 4B TTS 2603
As a **4B-parameter model**, is optimized for **on-device deployment**, requiring minimal hardware resources.
This model is released in **BF16** under the **CC BY NC 4**.
Voxtral TTS: A frontier, open-weights text-to-speech model that’s fast, instantly adaptable, and produces lifelike speech for voice agents. Today we’re releasing Voxtral TTS, our first text-to-speech model with state-of-the-art performance in multilingual voice generation.
For more details, see our:
TODO:
- [Blog post]()
- [Demo]()
## Key Features
The Voxtral Mini 4B Realtime model offers the following capabilities:
- **Text-to-Speech** generation with 20 preset voices across different languages
- **Multilingual support**: English, French, Spanish, German, Italian, Portuguese, Dutch, Arabic, and Hindi
- **24 kHz audio output** in WAV, PCM, FLAC, MP3, AAC, and Opus formats
- **Low-latency, high-throughput inference** with batch inference and streaming output support
### Use Cases
- **Sales and Marketing**: build a consistent brand voice for internal and external communication like newsletters, podcasts, dubbing, and voice overs.
- **Conversational Assistant**: create a voice agent that speaks in natural language and triggers actions in conversation.
- **Fictional Characters**: design interactive storytelling and games via emotion-steering controls for tone, personality, and emotive expressiveness.
- **Customer Support**: customize the model to your brand style guide and implement into your contact support and customer support infrastructure.
- **Real Time Translation**: with Voxtral STT, create a voice-to-voice translation across 9 languages.
## Benchmark Results
- Measured using [vllm_omni/examples/offline_inference/voxtral_tts/end2end.py](https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/voxtral_tts).
- Input: 500-character text with a 10-second audio reference.
- Hardware: single NVIDIA H200.
- vllm version: v0.18.0.
*Note*: The RTF in `end2end.py` uses an inverted formula (higher = better). The table below converts it back to the standard RTF convention (lower = better)
| Concurrency | Latency | RTF | Throughput (char/s/GPU) |
|:-----------:|:-------:|:-----:|:-----------------------:|
| 1 | 86 ms | 0.147 | 95.0 |
| 16 | 377 ms | 0.329 | 631.1 |
| 32 | 600 ms | 0.395 | 1065.3 |
## Demo
[](https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtime)
## Usage
The model can also be deployed with the following libraries:
- [`vllm-omni (recommended)`](https://github.com/vllm-project/vllm-omni): See [here](#vllm-omni-recommended)
### vLLM Omni (recommended)
> [!Tip]
> We've worked hand-in-hand with the vLLM-Omni team to have production-grade support for Voxtral 4B TTS 2603 with vLLM-Omni.
> Special thanks goes out to Han Gao, Hongsheng Liu, Roger Wang, and Yueqian Lin from the vLLM-Omni team.
**Installation**
Make sure to install [vllm](https://github.com/vllm-project/vllm) from the latest (>= 0.18.0) pypi package.
See [here](https://docs.vllm.ai/en/latest/getting_started/installation/) for a full installation guide.
```
uv pip install -U vllm
```
You can also make use of a ready-to-go [docker image](https://github.com/vllm-project/vllm/blob/main/docker/Dockerfile) or on the [docker hub](https://hub.docker.com/layers/vllm/vllm-openai/v0.18.0/images/sha256-96c7e88811a07030f27bc44cd71b9007258a15f130cfec2bb4ab057512238b05).
Next, you should install [`vllm-omni`](https://github.com/vllm-project/vllm-omni) from "main".
```
uv pip install git+https://github.com/vllm-project/vllm-omni.git --upgrade
```
> [!Warning]
> If you do are seeing an error due to `git` not being installed, make sure to
> run `apt update;apt install -y git` and try again.
Installing `vllm >= 0.18.0` should automatically install `mistral_common >= 1.10.0` which you can verify by running:
```sh
python3 -c "import mistral_common; print(mistral_common.__version__)" # should print >= 1.10.0
```
#### Serve
Due to size and the BF16 format of the weights - `Voxtral-4B-TTS-2603` can run on a single GPU with >= 16GB memory.
```bash
vllm serve mistralai/Voxtral-4B-TTS-2603 --omni
```
#### Client
```py
import io
import httpx
import soundfile as sf
BASE_URL = "http://:8000/v1"
payload = {
"input": "Hello, this is a test of the Voxtral text to speech system.",
"model": "mistralai/Voxtral-4B-TTS-2603",
"response_format": "wav",
"voice": "Neutral_Male",
}
response = httpx.post(f"{BASE_URL}/audio/speech", json=payload, timeout=120.0)
response.raise_for_status()
audio_array, sr = sf.read(io.BytesIO(response.content), dtype="float32")
print(f"Got audio: {len(audio_array)} samples at {sr} Hz")
# you can play the audio with a library like `sounddevice.play` for example
```
#### Demo
For a fully local demo, please refer to [`vllm-omni/examples/online_serving/voxtral_tts/gradio_demo.py`](https://github.com/vllm-project/vllm-omni/blob/main/examples/online_serving/voxtral_tts/gradio_demo.py)
or you can refer to our [**official demo**](https://console.mistral.ai/build/audio/text-to-speech).
## License
This model is licensed under the [CC BY NC 4](https://creativecommons.org/licenses/by-nc/4.0/).
*You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.*