The VoicePrivacy 2024 Challenge Evaluation Plan

The VoicePrivacy 2024 Challenge Evaluation Plan

12 Jun 2024 | Natalia Tomashenko, Xiaoxiao Miao, Pierre Champion, Sarina Meyer, Xin Wang, Emmanuel Vincent, Michele Panariello, Nicholas Evans, Junichi Yamagishi, and Massimiliano Todisco
The VoicePrivacy 2024 Challenge aims to develop voice anonymization systems that conceal the speaker's identity while preserving linguistic content and emotional states. The challenge involves creating systems that anonymize speech data at the utterance level, ensuring that each utterance is assigned a pseudo-speaker independently. Participants are provided with development and evaluation datasets, evaluation scripts, and baseline anonymization systems. The evaluation metrics include the equal error rate (EER) for privacy, word error rate (WER) for automatic speech recognition (ASR), and unweighted average recall (UAR) for speech emotion recognition (SER). The challenge will be held in conjunction with Interspeech 2024, where participants can present their systems and submit additional papers. The organizers have introduced several changes from the 2022 edition, including removing the requirement to preserve voice distinctiveness and intonation, providing an extended list of datasets and pretrained models, and simplifying the evaluation protocol. New baseline systems (B3, B4, B5, B6) have been added to better protect privacy and improve utility. The challenge emphasizes the importance of preserving emotional states, which are crucial in many real-world applications.The VoicePrivacy 2024 Challenge aims to develop voice anonymization systems that conceal the speaker's identity while preserving linguistic content and emotional states. The challenge involves creating systems that anonymize speech data at the utterance level, ensuring that each utterance is assigned a pseudo-speaker independently. Participants are provided with development and evaluation datasets, evaluation scripts, and baseline anonymization systems. The evaluation metrics include the equal error rate (EER) for privacy, word error rate (WER) for automatic speech recognition (ASR), and unweighted average recall (UAR) for speech emotion recognition (SER). The challenge will be held in conjunction with Interspeech 2024, where participants can present their systems and submit additional papers. The organizers have introduced several changes from the 2022 edition, including removing the requirement to preserve voice distinctiveness and intonation, providing an extended list of datasets and pretrained models, and simplifying the evaluation protocol. New baseline systems (B3, B4, B5, B6) have been added to better protect privacy and improve utility. The challenge emphasizes the importance of preserving emotional states, which are crucial in many real-world applications.
Reach us at info@study.space
[slides and audio] The VoicePrivacy 2024 Challenge Evaluation Plan