Project detail

Restoration of lost information in digital signals

Duration: 01.01.2017 — 31.12.2019

Funding resources

Czech Science Foundation - LA granty

- part funder (2017-01-01 - not assigned)

On the project

Locally degraded or lost information is a problem in signal processing, e.g. old audio recordings or lost blocks of time‐frequency (TF) coefficients in VoIP. Automatic recovering lost segments in recent years have been collectively referred to as inpainting. However, state‐of‐the‐art methods assume simplistic signal models that fail to capture the characteristic structures of speech and music data. MERLIN provides the development of novel inpainting methods through the combination of adapted TF representations, appropriate signal models, state‐of‐the‐art methods and the consideration of perceptual indicators. Signal models will be learned on the reliable TF information. Input signals will be decomposed into several layers, e.g. transients and harmonics. The inpainting process will be driven by modern algorithms implementing structured sparsity, low‐rank approximation or state‐of‐the‐art image processing techniques alongside preexisting procedures in an open‐source software toolbox facilitating evaluation via objective and subjective criteria.

Keywords
Audio restoration, inpainting, perceptual evaluation

Mark

GA17-33798L

Default language

English

People responsible

Mokrý Ondřej, Ing., Ph.D. - fellow researcher
Záviška Pavel, Ing., Ph.D. - fellow researcher
Rajmic Pavel, prof. Mgr., Ph.D. - principal person responsible

Units

Department of Telecommunications
- responsible department (2017-07-20 - not assigned)
Faculty of Electrical Engineering and Communication
- beneficiary (2017-01-01 - not assigned)

Results

PRŮŠA, Z.; RAJMIC, P. Toward High-Quality Real-Time Signal Reconstruction from STFT Magnitude. IEEE SIGNAL PROCESSING LETTERS, 2017, vol. 24, no. 6, p. 892-896. ISSN: 1070-9908.
Detail

MOKRÝ, O.; RAJMIC, P. Audio Inpainting: Revisited and Reweighted. IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH AND LANGUAGE PROCESSING, 2020, no. 28, p. 2906-2918. ISSN: 2329-9290.
Detail

ZÁVIŠKA, P.; RAJMIC, P.; PRŮŠA, Z.; VESELÝ, V. Revisiting synthesis model in Sparse Audio Declipper. In Latent Variable Analysis and Signal Separation, 14th International Conference, LVA/ICA 2018 Proceedings. Lecture Notes in Computer Science. Cham: Springer, 2018. p. 429-445. ISBN: 978-3-319-93764-9.
Detail

ZÁVIŠKA, P.; MOKRÝ, O.; RAJMIC, P. S-SPADE Done Right: Detailed Study of the Sparse Audio Declipper Algorithms. 2018. p. 1-8.
Detail

ZÁVIŠKA, P.; RAJMIC, P.; MOKRÝ, O.; PRŮŠA, Z. A Proper version of Synthesis-based Sparse Audio Declipper. In 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Proc. International Conference on Acoustics, Speech, and Signal Processing. Brighton, United Kingdom: IEEE, 2019. p. 591-595. ISBN: 978-1-4799-8131-1. ISSN: 1520-6149.
Detail

MOKRÝ, O.; ZÁVIŠKA, P.; RAJMIC, P.; VESELÝ, V. Introducing SPAIN (SParse Audio INpainter). In 2019 27th European Signal Processing Conference (EUSIPCO). European Signal Processing Conference, 2019. p. 666-670. ISBN: 978-9-0827-9702-2.
Detail

RAJMIC, P.; KOLDOVSKÝ, Z.; DAŇKOVÁ, M. Fast reconstruction of sparse relative impulse responses via second-order cone programming. In Proceedings of the 2017 Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA). Institute of Electrical and Electronics Engineers Inc., 2017. p. 364-368. ISBN: 978-1-5386-1631-4.
Detail

RAJMIC, P.; ZÁVIŠKA, P.; VESELÝ, V.; MOKRÝ, O. A new generalized projection and its application to acceleration of audio declipping. Axioms, 2019, vol. 8, no. 3, p. 1-20. ISSN: 2075-1680.
Detail

MOKRÝ, O.; RAJMIC, P. Reweighted l1 minimization for audio inpainting. Toulouse, France: 2019.
Detail

ZÁVIŠKA, P.; RAJMIC, P.; SCHIMMEL, J. Psychoacoustically motivated audio declipping based on weighted l1 minimization. In Proceedings of the 2019 42nd International Conference on Telecommunications and Signal Processing (TSP). Budapest, Hungary: IEEE, 2019. p. 338-342. ISBN: 978-1-7281-1864-2.
Detail

ZÁVIŠKA, P.: Generalized projector; Projector onto a box-constrained set with a Parseval frame operator. pracovna SD5.66, Technická 12, 61600 Brno. URL: http://www.utko.feec.vutbr.cz/~rajmic/software/project_accel_declip.zip. (software)
Detail

ZÁVIŠKA, P.: Declip_psychoacoust; Psychoacoustically motivated audio declipper based on sparsity. pracovna SD5.66, Technická 12, 61600 Brno. URL: http://www.utko.feec.vutbr.cz/~rajmic/software/declip_psychoacoust.zip. (software)
Detail

MOKRÝ, O.: SPAIN; Sparse Audio Inpainter. pracovna SD5.66, Technická 12, 61600 Brno. URL: http://www.utko.feec.vutbr.cz/~rajmic/software/SPAIN.zip. (software)
Detail

ZÁVIŠKA, P.: S-SPADE DR; Synthesis-based Sparse Audio Declipper Done Right. pracovna SD5.66, Technická 12, 61600 Brno. URL: http://www.utko.feec.vutbr.cz/~rajmic/software/SPADE-DR.zip. (software)
Detail

ZÁVIŠKA, P.: ASPADE vs SSPADE; Software for Sparse Restoration of Clipped Audio Signals. pracovna SD5.66, Technická 12, 61600 Brno. URL: http://www.utko.feec.vutbr.cz/~rajmic/software/aspade_vs_sspade.zip. (software)
Detail

Responsibility: Rajmic Pavel, prof. Mgr., Ph.D.