Project detail

From perceptron to perception: psychoacoustically motivated audio reconstruction using learned components

Duration: 01.01.2023 — 31.12.2025

Funding resources

Czech Science Foundation - Standardní projekty

- whole funder

On the project

Description in English
State-of-the-art methods for the reconstruction of degraded audio signals are successful at their performance. However, they still suffer from perceptually unpleasant or annoying artifacts coming from the reconstruction process. Only a few recent approaches involved psychoacoustics to alleviate this disturbing phenomena. Unfortunately, it turns out that the incorporation of auditory models into current methods it strongly limited. Their use therein is prevented by their complexity, non-differentiability and non-convexity. Recent results from the field of deep learning show that functionals can be trained to distinguish between faithful and implausible audio. Such discriminators come in the form of a neural network, thus being non-linear and non-convex, but, most importantly, differentiable. The project aims at using these discriminators as universal regularizers in algorithms inspired in convex optimization. This will not only lead to a general reconstruction framework, but also to significant improvements of perceptual quality in a wide range of audio inverse problems.

Keywords
rekonstrukce signálu; regularizace; hluboké učení; neuronová síť; iterativní algoritmy; diskriminátor; kodek

Key words in English
signal processing, audio, signal reconstruction, regularization, deep learning, neural network, discriminator, iterative algorithms, auditory modeling, psychoacoustics

Mark

23-07294S

Default language

Czech

People responsible

Rajmic Pavel, prof. Mgr., Ph.D. - principal person responsible

Units

Department of Telecommunications
- beneficiary (2022-03-31 - not assigned)

Results

ZÁVIŠKA, P.; RAJMIC, P.; MOKRÝ, O. Multiple Hankel matrix rank minimization for audio inpainting. In Proceedings of the 2023 46th International Conference on Telecommunications and Signal Processing (TSP). Prague, Czech republic: IEEE, 2023. p. 47-51. ISBN: 979-8-3503-0396-4.
Detail

ŠVENTO, M.; RAJMIC, P.; MOKRÝ, O. Plug-and-play audio restoration with diffusion denoiser. In 2024 18th International Workshop on Acoustic Signal Enhancement (IWAENC). Aalborg, Denmark: IEEE, 2024. p. 115-119. ISBN: 979-8-3503-6185-8.
Detail

ŠVENTO, M.; BALUŠÍK, P. Deep prior audio compression. Proceedings I of the 30th Student EEICT 2024 (General Papers). Proceedings II of the Conference STUDENT EEICT. 1. Brno: Brno University of Technology, Faculty of Electrical Engineering and Communication, 2024. ISSN: 2788-1334.
Detail