Detail publikace

Probing Self-Supervised Learning Models With Target Speech Extraction

PENG, J. DELCROIX, M. OCHIAI, T. ASHIHARA, T. PLCHOT, O. ARAKI, S. ČERNOCKÝ, J.

Originální název

Probing Self-Supervised Learning Models With Target Speech Extraction

Typ

článek ve sborníku ve WoS nebo Scopus

Jazyk

angličtina

Originální abstrakt

Large-scale pre-trained self-supervised learning (SSL) models have shown remarkable advancements in speech-related tasks. However, the utilization of these models in complex multi-talker scenarios, such as extracting a target speaker in a mixture, is yet to be fully evaluated. In this paper, we introduce target speech extraction (TSE) as a novel downstream task to evaluate the feature extraction capabilities of pre-trained SSL models. TSE uniquely requires both speaker identification and speech separation, distinguishing it from other tasks in the Speech processing Universal PERformance Benchmark (SUPERB) evaluation. Specifically, we propose a TSE downstream model composed of two lightweight task-oriented modules based on the same frozen SSL model. One module functions as a speaker encoder to obtain target speaker information from an enrollment speech, while the other estimates the target speaker's mask to extract its speech from the mixture. Experimental results on the Libri2mix datasets reveal the relevance of the TSE downstream task to probe SSL models, as its performance cannot be simply deduced from other related tasks such as speaker verification and separation.

Klíčová slova

Target speech extraction, self-supervised learning, SUPERB

Autoři

PENG, J.; DELCROIX, M.; OCHIAI, T.; ASHIHARA, T.; PLCHOT, O.; ARAKI, S.; ČERNOCKÝ, J.

Vydáno

14. 4. 2024

Nakladatel

IEEE Signal Processing Society

Místo

Seoul

ISBN

979-8-3503-7451-3

Kniha

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Strany od

535

Strany do

539

Strany počet

5

URL

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10627502

BibTex

@inproceedings{BUT189780,
  author="PENG, J. and DELCROIX, M. and OCHIAI, T. and ASHIHARA, T. and PLCHOT, O. and ARAKI, S. and ČERNOCKÝ, J.",
  title="Probing Self-Supervised Learning Models With Target Speech Extraction",
  booktitle="ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",
  year="2024",
  pages="535--539",
  publisher="IEEE Signal Processing Society",
  address="Seoul",
  doi="10.1109/ICASSPW62465.2024.10627502",
  isbn="979-8-3503-7451-3",
  url="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10627502"
}

Dokumenty

peng_icassp2024_Probing_Self-Supervised_Learning.pdf