This paper introduces a novel framework for quantifying predictive uncertainty in Large Language Models (LLMs) during in-context learning. The authors propose a method to decompose uncertainty into aleatoric and epistemic components, which arise from the provided demonstrations and model configurations, respectively. The proposed method uses entropy-based estimation to quantify both types of uncertainty, providing an unsupervised way to understand the prediction of in-context learning. The framework is evaluated on various natural language understanding tasks, demonstrating its effectiveness in assessing the reliability of LLM responses. The results show that the proposed method outperforms existing approaches in terms of uncertainty quantification and decomposition, particularly in tasks involving sentiment analysis, linguistic acceptability, and topic classification. The method is also shown to be effective in detecting out-of-domain demonstrations and semantic out-of-distribution samples. The authors conclude that their approach provides a significant advancement in understanding and quantifying the uncertainty of LLMs, enabling more reliable and robust applications of these models.This paper introduces a novel framework for quantifying predictive uncertainty in Large Language Models (LLMs) during in-context learning. The authors propose a method to decompose uncertainty into aleatoric and epistemic components, which arise from the provided demonstrations and model configurations, respectively. The proposed method uses entropy-based estimation to quantify both types of uncertainty, providing an unsupervised way to understand the prediction of in-context learning. The framework is evaluated on various natural language understanding tasks, demonstrating its effectiveness in assessing the reliability of LLM responses. The results show that the proposed method outperforms existing approaches in terms of uncertainty quantification and decomposition, particularly in tasks involving sentiment analysis, linguistic acceptability, and topic classification. The method is also shown to be effective in detecting out-of-domain demonstrations and semantic out-of-distribution samples. The authors conclude that their approach provides a significant advancement in understanding and quantifying the uncertainty of LLMs, enabling more reliable and robust applications of these models.