12 Jun 2024 | Natalia Tomashenko, Xiaoxiao Miao, Pierre Champion, Sarina Meyer, Xin Wang, Emmanuel Vincent, Michele Panariello, Nicholas Evans, Junichi Yamagishi, and Massimiliano Todisco
The VoicePrivacy 2024 Challenge Evaluation Plan outlines a task for participants to develop voice anonymization systems that conceal speaker identity while preserving linguistic and emotional content. The challenge provides development and evaluation datasets, baseline systems, and training resources. Participants submit their systems, run evaluation scripts, and submit results and anonymized speech data. Results will be presented at a workshop alongside Interspeech 2024.
Compared to 2022, the 2024 challenge removes metrics related to voice distinctiveness and intonation, and all data are anonymized at the utterance level. An extended list of datasets and pretrained models is provided based on participant requests. The evaluation protocol and scripts are simplified, primarily in Python, to ease participation. Only objective metrics are used: EER for privacy and WER and UAR for utility. Models for utility evaluation are trained on original data to ensure content integrity.
The challenge includes four conditions with minimum target EERs of 10%, 20%, 30%, and 40%. Participants are encouraged to submit systems for multiple conditions. The evaluation involves an ASV system trained on LibriSpeech and SER systems trained on IEMOCAP data. Baseline systems B1-B6 are provided, with B3-B6 offering better privacy and varying utility performance.
Participants must anonymize data at the utterance level and train ASV models on anonymized data. Evaluation metrics include EER, WER, and UAR. Results are ranked based on these metrics. Participants must submit anonymized speech data and system descriptions. The challenge culminates in a workshop with papers on anonymization systems. The evaluation deadline is June 15, 2024. The challenge is supported by French and Japanese research agencies.The VoicePrivacy 2024 Challenge Evaluation Plan outlines a task for participants to develop voice anonymization systems that conceal speaker identity while preserving linguistic and emotional content. The challenge provides development and evaluation datasets, baseline systems, and training resources. Participants submit their systems, run evaluation scripts, and submit results and anonymized speech data. Results will be presented at a workshop alongside Interspeech 2024.
Compared to 2022, the 2024 challenge removes metrics related to voice distinctiveness and intonation, and all data are anonymized at the utterance level. An extended list of datasets and pretrained models is provided based on participant requests. The evaluation protocol and scripts are simplified, primarily in Python, to ease participation. Only objective metrics are used: EER for privacy and WER and UAR for utility. Models for utility evaluation are trained on original data to ensure content integrity.
The challenge includes four conditions with minimum target EERs of 10%, 20%, 30%, and 40%. Participants are encouraged to submit systems for multiple conditions. The evaluation involves an ASV system trained on LibriSpeech and SER systems trained on IEMOCAP data. Baseline systems B1-B6 are provided, with B3-B6 offering better privacy and varying utility performance.
Participants must anonymize data at the utterance level and train ASV models on anonymized data. Evaluation metrics include EER, WER, and UAR. Results are ranked based on these metrics. Participants must submit anonymized speech data and system descriptions. The challenge culminates in a workshop with papers on anonymization systems. The evaluation deadline is June 15, 2024. The challenge is supported by French and Japanese research agencies.