Joint Energy-Based Model for Robust Speech Classification System against Dirty-Label Backdoor Poisoning Attacks

article in a collection out of WoS and Scopus

English

Our novel technique utilizes a Joint Energy-based Model (JEM) that integrates both discriminative and generative approaches to increase resistance against dirty-label backdoor attacks. Our approach is especially effective when the trigger is short or hardly perceivable. We simulate the attack on the Speech Commands Dataset consisting of 1 s audio clips. During training, we use JEM to model a view of the input implemented by a randomly selected 610 ms window. During inference, we combine all (40) possible views utilizing a generative part of JEM. The resulting system has slightly decreased accuracy but significantly increased resistance shown in multiple scenarios. Interestingly, replacing JEM with a standard discriminative model (Disc) provides increased resistance with a lesser effect compared to JEM but maintains accuracy. We introduce an extension motivated by semi-supervised training that further improves JEM but not Disc. JEM can also benefit from Gaussian noise during evaluation.

joint energy-based model, poisoning attacks, speech commands classification, dirty-label backdoor

ŠŮSTEK, M.; JOSHI, S.; LI, H.; THEBAUD, T.; VILLALBA LOPEZ, J.; KHUDANPUR, S.; DEHAK, N.

13. 10. 2023

IEEE Signal Processing Society

Taipei

979-8-3503-0689-7

Proceedings of IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

1

8

8

https://ieeexplore.ieee.org/document/10389697

Joint_Energy-Based_Model_for_Robust_Speech_Classification_System_Against_Dirty-Label_Backdoor_Poisoning_Attacks.pdf