This survey provides a comprehensive overview of the current state of fairness in Large Language Models (LLMs). It begins by introducing LLMs and their training process, highlighting the importance of addressing social biases and discrimination. The survey then delves into the factors contributing to bias in LLMs, including training data bias, embedding bias, and label bias. It discusses various metrics for quantifying bias, such as embedding-based, probability-based, and generation-based metrics, and reviews existing algorithms for mitigating bias at different stages of the LLMs workflow, including pre-processing, in-training, intra-processing, and post-processing. The survey also summarizes resources for evaluating bias, including toolkits and datasets. Finally, it addresses existing research challenges and open questions, emphasizing the need for more tailored datasets and methods to address multiple types of fairness. The conclusion underscores the ethical and societal implications of ensuring fairness in LLMs and the ongoing efforts to develop robust solutions.This survey provides a comprehensive overview of the current state of fairness in Large Language Models (LLMs). It begins by introducing LLMs and their training process, highlighting the importance of addressing social biases and discrimination. The survey then delves into the factors contributing to bias in LLMs, including training data bias, embedding bias, and label bias. It discusses various metrics for quantifying bias, such as embedding-based, probability-based, and generation-based metrics, and reviews existing algorithms for mitigating bias at different stages of the LLMs workflow, including pre-processing, in-training, intra-processing, and post-processing. The survey also summarizes resources for evaluating bias, including toolkits and datasets. Finally, it addresses existing research challenges and open questions, emphasizing the need for more tailored datasets and methods to address multiple types of fairness. The conclusion underscores the ethical and societal implications of ensuring fairness in LLMs and the ongoing efforts to develop robust solutions.