a-DCF: an architecture agnostic metric with application to spoofing-robust speaker verification

a-DCF: an architecture agnostic metric with application to spoofing-robust speaker verification

3 Mar 2024 | Hye-jin Shim, Jee-weon Jung, Tomi Kinnunen, Nicholas Evans, Jean-Francois Bonastre and Itshak Lapidot
The paper introduces a-DCF, an architecture-agnostic detection cost function for evaluating spoofing-robust automatic speaker verification (ASV) systems. Traditional metrics like DCF and t-DCF have limitations in supporting various architectures, leading to inconsistent benchmarking. a-DCF generalizes the original DCF to accommodate different system designs, including tandem, jointly optimized, and single-model systems. It explicitly defines class priors and detection costs, reflecting decisions in a Bayes risk sense. Unlike t-DCF, which requires two detection thresholds, a-DCF uses a single threshold and is applicable to any architecture that produces a single score. The a-DCF is normalized to a range between 0 and 1, making it easier to compare across systems. The paper evaluates a-DCF on the ASVspoof 2019 dataset, demonstrating its effectiveness in benchmarking different spoofing-robust ASV solutions. Results show that a-DCF provides a more flexible and accurate evaluation than traditional EER-based metrics, which suffer from issues like dependency on class priors. The a-DCF is also more general and can be applied to other biometric verification tasks. The study highlights the need for new metrics that are not limited to specific architectures and can be used for a wide range of spoofing-robust ASV systems.The paper introduces a-DCF, an architecture-agnostic detection cost function for evaluating spoofing-robust automatic speaker verification (ASV) systems. Traditional metrics like DCF and t-DCF have limitations in supporting various architectures, leading to inconsistent benchmarking. a-DCF generalizes the original DCF to accommodate different system designs, including tandem, jointly optimized, and single-model systems. It explicitly defines class priors and detection costs, reflecting decisions in a Bayes risk sense. Unlike t-DCF, which requires two detection thresholds, a-DCF uses a single threshold and is applicable to any architecture that produces a single score. The a-DCF is normalized to a range between 0 and 1, making it easier to compare across systems. The paper evaluates a-DCF on the ASVspoof 2019 dataset, demonstrating its effectiveness in benchmarking different spoofing-robust ASV solutions. Results show that a-DCF provides a more flexible and accurate evaluation than traditional EER-based metrics, which suffer from issues like dependency on class priors. The a-DCF is also more general and can be applied to other biometric verification tasks. The study highlights the need for new metrics that are not limited to specific architectures and can be used for a wide range of spoofing-robust ASV systems.
Reach us at info@study.space
[slides and audio] a-DCF%3A an architecture agnostic metric with application to spoofing-robust speaker verification