---
library_name: speechbrain
pipeline_tag: feature-extraction
tags:
  - speechbrain
  - pytorch
  - speech
  - multilingual
  - semantic-embeddings
  - speech-embeddings
  - w2v-bert
base_model: facebook/w2v-bert-2.0
---

# SENSE multilingual semantic speech embeddings

This repository provides a pretrained **SENSE** model for extracting multilingual semantic speech embeddings with [SpeechBrain](https://speechbrain.github.io/).

The model was trained on **90 Common Voice languages** following the SENSE framework described in:

**SENSE models: an open source solution for multilingual and multimodal semantic-based tasks**  
https://arxiv.org/abs/2509.12093

## Pipeline description

SENSE produces utterance-level semantic speech embeddings from 16 kHz audio.

## Install SpeechBrain

```bash
pip install speechbrain
```

## Compute SENSE embeddings

```python
from speechbrain.inference.interfaces import foreign_class

sense = foreign_class(
    source="LIA-AvignonUniversity/SENSE",
    pymodule_file="custom.py",
    classname="SENSEEncoder",
)

embedding = sense.encode_file("audio.wav")

print(embedding.shape)
# torch.Size([1, 1024])
```


## Inference on GPU

```python
from speechbrain.inference.interfaces import foreign_class

sense = foreign_class(
    source="LIA-AvignonUniversity/SENSE",
    pymodule_file="custom.py",
    classname="SENSEEncoder",
    run_opts={"device": "cuda:0"},
)

embedding = sense.encode_file("audio.wav")
```

## Training

The model was trained with SpeechBrain on multilingual Common Voice data.

The complete training recipe is available here:

https://github.com/speechbrain/speechbrain/tree/develop/recipes/CommonVoice/SENSE


## Referencing SENSE

If you use this model, please cite:

```bibtex
@article{mdhaffar2025sense,
  title={SENSE models: an open source solution for multilingual and multimodal semantic-based tasks},
  author={Mdhaffar, Salima and Elleuch, Haroun and Chellaf, Chaimae and Nguyen, Ha and Est{\`e}ve, Yannick},
  journal={arXiv preprint arXiv:2509.12093},
  year={2025}
}
```

# Citing SpeechBrain

Please cite SpeechBrain if you use it in your research:

```bibtex
@misc{speechbrainV1,
  title={Open-Source Conversational AI with SpeechBrain 1.0},
  author={Mirco Ravanelli and Titouan Parcollet and Adel Moumen and Sylvain de Langen and Cem Subakan and Peter Plantinga and Yingzhi Wang and Pooneh Mousavi and Luca Della Libera and Artem Ploujnikov and Francesco Paissan and Davide Borra and Salah Zaiem and Zeyu Zhao and Shucong Zhang and Georgios Karakasidis and Sung-Lin Yeh and Pierre Champion and Aku Rouhe and Rudolf Braun and Florian Mai and Juan Zuluaga-Gomez and Seyed Mahed Mousavi and Andreas Nautsch and Xuechen Liu and Sangeet Sagar and Jarod Duret and Salima Mdhaffar and Gaelle Laperriere and Mickael Rouvier and Renato De Mori and Yannick Esteve},
  year={2024},
  eprint={2407.00463},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2407.00463}
}
```

# About SpeechBrain

- Website: https://speechbrain.github.io/
- Code: https://github.com/speechbrain/speechbrain/
- Hugging Face: https://huggingface.co/speechbrain/