This paper investigates the impact of language models (LMs) on human-AI interactions, focusing on how LMs communicate uncertainties and the subsequent behavior of downstream users. The authors find that LMs are reluctant to express uncertainties even when they produce incorrect responses, and when prompted to express confidence, they tend to be overconfident, leading to high error rates (47% among confident responses). Human experiments reveal that users heavily rely on LM-generated responses, regardless of their certainty markers. The study also identifies biases in human-annotated datasets, showing that humans are biased against texts with uncertainty. The findings highlight new safety risks in human-LM interactions and propose design recommendations and mitigating strategies, such as generating weakeners without explicit elicitation and using plain statements only when the model is confident. The research underscores the need for LMs to autonomously emit expressions of uncertainty, generate a comprehensive range of epistemic markers, and calibrate their certainty levels contextually.This paper investigates the impact of language models (LMs) on human-AI interactions, focusing on how LMs communicate uncertainties and the subsequent behavior of downstream users. The authors find that LMs are reluctant to express uncertainties even when they produce incorrect responses, and when prompted to express confidence, they tend to be overconfident, leading to high error rates (47% among confident responses). Human experiments reveal that users heavily rely on LM-generated responses, regardless of their certainty markers. The study also identifies biases in human-annotated datasets, showing that humans are biased against texts with uncertainty. The findings highlight new safety risks in human-LM interactions and propose design recommendations and mitigating strategies, such as generating weakeners without explicit elicitation and using plain statements only when the model is confident. The research underscores the need for LMs to autonomously emit expressions of uncertainty, generate a comprehensive range of epistemic markers, and calibrate their certainty levels contextually.