Přístupnostní navigace
E-application
Search Search Close
Publication detail
STAFYLAKIS, T. MOŠNER, L. KAKOUROS, S. PLCHOT, O. BURGET, L. ČERNOCKÝ, J.
Original Title
Extracting speaker and emotion information from self-supervised speech models via channel-wise correlations
Type
conference paper
Language
English
Original Abstract
Self-supervised learning of speech representations from large amounts of unlabeled data has enabled state-of-the-art results in several speech processing tasks. Aggregating these speech representations across time is typically approached by using descriptive statistics, and in particular, using the first- and second-order statistics of representation coefficients. In this paper, we examine an alternative way of extracting speaker and emotion information from self-supervised trained models, based on the correlations between the coefficients of the representations - correlation pooling. We show improvements over mean pooling and further gains when the pooling methods are combined via fusion. The code is available at github.com/Lamomal/s3prl_correlation.
Keywords
Speaker identification, speaker verification, emotion recognition, self-supervised models
Authors
STAFYLAKIS, T.; MOŠNER, L.; KAKOUROS, S.; PLCHOT, O.; BURGET, L.; ČERNOCKÝ, J.
Released
9. 1. 2023
Publisher
IEEE Signal Processing Society
Location
Doha
ISBN
978-1-6654-7189-3
Book
2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings
Pages from
1136
Pages to
1143
Pages count
8
URL
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10023345
BibTex
@inproceedings{BUT185160, author="STAFYLAKIS, T. and MOŠNER, L. and KAKOUROS, S. and PLCHOT, O. and BURGET, L. and ČERNOCKÝ, J.", title="Extracting speaker and emotion information from self-supervised speech models via channel-wise correlations", booktitle="2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings", year="2023", pages="1136--1143", publisher="IEEE Signal Processing Society", address="Doha", doi="10.1109/SLT54892.2023.10023345", isbn="978-1-6654-7189-3", url="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10023345" }