Publication detail

Investigation of Speaker Representation for Target-Speaker Speech Processing

ASHIHARA, T. MORIYA, T. HORIGUCHI, S. PENG, J. OCHIAI, T. DELCROIX, M. MATSUURA, K. SATO, H.

Original Title

Investigation of Speaker Representation for Target-Speaker Speech Processing

Type

article in a collection out of WoS and Scopus

Language

English

Original Abstract

Target-speaker speech processing (TS) tasks, such as target-speaker automatic speech recognition (TS-ASR), target speech extraction (TSE), and personal voice activity detection (p-VAD), are important for extracting information about a desired speaker's speech even when it is corrupted by interfering speakers. While most studies have focused on training schemes or system architectures for each specific task, the auxiliary network for embedding target-speaker cues has not been investigated comprehensively in a unified cross- task evaluation. Therefore, this paper aims to address a fundamental question: what is the preferred speaker embedding for TS tasks? To this end, for the TS-ASR, TSE, and p-VAD tasks, we compare pre-trained speaker encoders (i.e., self-supervised or speaker recog- nition models) that compute speaker embeddings from pre-recorded enrollment speech of the target speaker with ideal speaker embed- dings derived directly from the target speaker's identity in the form of a one-hot vector. To further understand the properties of ideal speaker embedding, we optimize it using a gradient-based approach to improve performance on the TS task. Our analysis reveals that speaker verification performance is somewhat unrelated to TS task performances, the one-hot vector outperforms enrollment-based ones, and the optimal embedding depends on the input mixture.

Keywords

peaker representation, target-speaker automatic speech recognition, target speech extraction, personal voice activity detection, self-supervised learning

Authors

ASHIHARA, T.; MORIYA, T.; HORIGUCHI, S.; PENG, J.; OCHIAI, T.; DELCROIX, M.; MATSUURA, K.; SATO, H.

Publisher

IEEE Signal Processing Society

Location

Macao

ISBN

979-8-3503-9225-8

Book

Proc. 2024 IEEE Spoken Language Technology Workshop (SLT)

Pages from

423

Pages to

430

Pages count

8

URL

https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10832160

BibTex

@inproceedings{BUT196770,
  author="ASHIHARA, T. and MORIYA, T. and HORIGUCHI, S. and PENG, J. and OCHIAI, T. and DELCROIX, M. and MATSUURA, K. and SATO, H.",
  title="Investigation of Speaker Representation for Target-Speaker Speech Processing",
  booktitle="Proc.  2024 IEEE Spoken Language Technology Workshop (SLT)",
  pages="423--430",
  publisher="IEEE Signal Processing Society",
  address="Macao",
  doi="10.1109/SLT61566.2024.10832160",
  isbn="979-8-3503-9225-8",
  url="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10832160"
}

Documents

ashihara_slt2024_Investigation_of_Speaker_Representation_for_Target-Speaker_Speech_Processing.pdf