This paper explores the impact of introducing LLM-powered devil's advocates in AI-assisted group decision-making processes. The authors design four different styles of LLM-powered devil's advocates, varying in their target of objection (challenging the AI recommendation or the majority opinion) and interactivity (static vs. dynamic). Through a randomized human-subject experiment, the study finds that interactive LLM-powered devil's advocates that challenge the AI model's decision recommendation can significantly enhance groups' appropriate reliance on AI, particularly in in-distribution decision-making instances. Non-interactive devil's advocates also slightly reduce under-reliance on AI in out-of-distribution cases. Interactive devil's advocates are perceived as more collaborative and of higher quality, but they do not significantly affect perceived workload or teamwork quality. The study concludes by discussing practical implications and future research directions.This paper explores the impact of introducing LLM-powered devil's advocates in AI-assisted group decision-making processes. The authors design four different styles of LLM-powered devil's advocates, varying in their target of objection (challenging the AI recommendation or the majority opinion) and interactivity (static vs. dynamic). Through a randomized human-subject experiment, the study finds that interactive LLM-powered devil's advocates that challenge the AI model's decision recommendation can significantly enhance groups' appropriate reliance on AI, particularly in in-distribution decision-making instances. Non-interactive devil's advocates also slightly reduce under-reliance on AI in out-of-distribution cases. Interactive devil's advocates are perceived as more collaborative and of higher quality, but they do not significantly affect perceived workload or teamwork quality. The study concludes by discussing practical implications and future research directions.