This paper presents the first benchmark of uncertainty disentanglement, evaluating a comprehensive range of uncertainty estimators across various tasks on ImageNet. The authors find that no existing approach provides disentangled uncertainty estimators in practice, and specialized uncertainty tasks are harder than predictive uncertainty tasks. The results provide practical advice for selecting appropriate uncertainty estimators for specific tasks and highlight the need for task-centric and disentangled uncertainties. The benchmark includes nineteen uncertainty quantification methods, categorized into distributional and deterministic methods, and evaluates them on thirteen tasks. The findings emphasize the importance of tailoring uncertainty estimators to specific tasks and suggest combining individual estimators that strongly reflect one type of uncertainty while being unrelated to the other. The paper also discusses the limitations and future directions of the field, emphasizing the need for more specialized and disentangled uncertainty estimators.This paper presents the first benchmark of uncertainty disentanglement, evaluating a comprehensive range of uncertainty estimators across various tasks on ImageNet. The authors find that no existing approach provides disentangled uncertainty estimators in practice, and specialized uncertainty tasks are harder than predictive uncertainty tasks. The results provide practical advice for selecting appropriate uncertainty estimators for specific tasks and highlight the need for task-centric and disentangled uncertainties. The benchmark includes nineteen uncertainty quantification methods, categorized into distributional and deterministic methods, and evaluates them on thirteen tasks. The findings emphasize the importance of tailoring uncertainty estimators to specific tasks and suggest combining individual estimators that strongly reflect one type of uncertainty while being unrelated to the other. The paper also discusses the limitations and future directions of the field, emphasizing the need for more specialized and disentangled uncertainty estimators.