Conformal prediction sets improve human decision making by providing calibrated uncertainty information, similar to how humans express uncertainty and offer alternatives. This study presents a pre-registered randomized controlled trial showing that humans using conformal prediction sets achieve higher accuracy on tasks compared to fixed-size prediction sets with the same coverage guarantee. Conformal prediction sets, which vary in size based on model uncertainty, help humans make better decisions by quantifying uncertainty, although they do not always speed up decision making. The results suggest that conformal prediction is useful for human-in-the-loop decision making and human-AI teams. The study evaluated three tasks: image classification, sentiment analysis, and named entity recognition. Humans using conformal prediction sets showed statistically significant improvements in accuracy compared to top-k sets, with medium effect sizes. While conformal sets may take longer to process, they provide more accurate information. The study also found that conformal prediction sets can help identify difficult examples for humans, allowing them to focus on challenging cases. However, when models perform poorly on certain groups, prediction sets can drag down human performance. The study highlights the importance of considering model fairness and the potential benefits of augmenting expert decision-making with conformal prediction sets. The findings support the integration of humans into decision-making pipelines to improve trust and reliability of machine learning models.Conformal prediction sets improve human decision making by providing calibrated uncertainty information, similar to how humans express uncertainty and offer alternatives. This study presents a pre-registered randomized controlled trial showing that humans using conformal prediction sets achieve higher accuracy on tasks compared to fixed-size prediction sets with the same coverage guarantee. Conformal prediction sets, which vary in size based on model uncertainty, help humans make better decisions by quantifying uncertainty, although they do not always speed up decision making. The results suggest that conformal prediction is useful for human-in-the-loop decision making and human-AI teams. The study evaluated three tasks: image classification, sentiment analysis, and named entity recognition. Humans using conformal prediction sets showed statistically significant improvements in accuracy compared to top-k sets, with medium effect sizes. While conformal sets may take longer to process, they provide more accurate information. The study also found that conformal prediction sets can help identify difficult examples for humans, allowing them to focus on challenging cases. However, when models perform poorly on certain groups, prediction sets can drag down human performance. The study highlights the importance of considering model fairness and the potential benefits of augmenting expert decision-making with conformal prediction sets. The findings support the integration of humans into decision-making pipelines to improve trust and reliability of machine learning models.