CFaiRLLM is a comprehensive evaluation framework designed to assess consumer fairness in Large Language Model (LLM) Recommender Systems (RecLLMs). As LLMs like ChatGPT become increasingly integrated into recommendation systems, they offer unprecedented personalization but also raise concerns about biases related to sensitive user attributes such as gender, age, and their intersections. CFaiRLLM addresses these concerns by evaluating how recommendations vary based on the inclusion of sensitive attributes, using both similarity alignment and true preference alignment. The framework analyzes recommendations generated under different conditions, including the use of sensitive attributes in user prompts, to identify potential biases. A key aspect of the study involves exploring how different user profile construction strategies (random, top-rated, recent) impact the alignment between recommendations made without considering sensitive attributes and those that are sensitive-attribute-aware, highlighting bias mechanisms within RecLLMs. The findings reveal significant disparities in recommendation fairness, particularly when sensitive attributes are integrated into the recommendation process. The choice of user profile sampling strategy plays a crucial role in affecting fairness outcomes, emphasizing the complexity of achieving fair recommendations in the era of LLMs. CFaiRLLM introduces an enhanced framework for assessing "Consumer Fairness in RecLLMs," focusing on two aspects: (i) the definition of unfairness (whether through similarity alignment or true preference alignment), and (ii) the granularity of groups (considering both individual and intersectional prompts). The framework also studies user-profile construction strategies and the scope of recommendations. CFaiRLLM builds on and refines the foundational framework proposed by Zhang et al., enhancing its evaluation foundation and application. The framework evaluates fairness by examining whether recommendations are truly personalized or biased by stereotypes associated with sensitive attributes. It ensures that recommendations align with users' actual preferences, moving beyond stereotypes to provide equitable service to all users. The framework introduces a nuanced consideration of "intersectional fairness," recognizing that individuals may have multiple overlapping identities that influence recommendation outcomes. By comparing neutral and sensitive attribute-influenced ranking lists, the framework quantifies the impact of sensitive attributes on recommendation fairness, identifying potential biases introduced by their consideration. The evaluation procedure involves collecting neutral and sensitive recommendations, analyzing their alignment with user preferences, and assessing the fairness of recommendations across independent and intersectional groups. The framework aims to ensure that every recommendation is fair and personalized, addressing the ethical dimensions of RecLLMs and promoting equitable service to all users.CFaiRLLM is a comprehensive evaluation framework designed to assess consumer fairness in Large Language Model (LLM) Recommender Systems (RecLLMs). As LLMs like ChatGPT become increasingly integrated into recommendation systems, they offer unprecedented personalization but also raise concerns about biases related to sensitive user attributes such as gender, age, and their intersections. CFaiRLLM addresses these concerns by evaluating how recommendations vary based on the inclusion of sensitive attributes, using both similarity alignment and true preference alignment. The framework analyzes recommendations generated under different conditions, including the use of sensitive attributes in user prompts, to identify potential biases. A key aspect of the study involves exploring how different user profile construction strategies (random, top-rated, recent) impact the alignment between recommendations made without considering sensitive attributes and those that are sensitive-attribute-aware, highlighting bias mechanisms within RecLLMs. The findings reveal significant disparities in recommendation fairness, particularly when sensitive attributes are integrated into the recommendation process. The choice of user profile sampling strategy plays a crucial role in affecting fairness outcomes, emphasizing the complexity of achieving fair recommendations in the era of LLMs. CFaiRLLM introduces an enhanced framework for assessing "Consumer Fairness in RecLLMs," focusing on two aspects: (i) the definition of unfairness (whether through similarity alignment or true preference alignment), and (ii) the granularity of groups (considering both individual and intersectional prompts). The framework also studies user-profile construction strategies and the scope of recommendations. CFaiRLLM builds on and refines the foundational framework proposed by Zhang et al., enhancing its evaluation foundation and application. The framework evaluates fairness by examining whether recommendations are truly personalized or biased by stereotypes associated with sensitive attributes. It ensures that recommendations align with users' actual preferences, moving beyond stereotypes to provide equitable service to all users. The framework introduces a nuanced consideration of "intersectional fairness," recognizing that individuals may have multiple overlapping identities that influence recommendation outcomes. By comparing neutral and sensitive attribute-influenced ranking lists, the framework quantifies the impact of sensitive attributes on recommendation fairness, identifying potential biases introduced by their consideration. The evaluation procedure involves collecting neutral and sensitive recommendations, analyzing their alignment with user preferences, and assessing the fairness of recommendations across independent and intersectional groups. The framework aims to ensure that every recommendation is fair and personalized, addressing the ethical dimensions of RecLLMs and promoting equitable service to all users.