**BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMs** **Abstract:** The rapid development of Large Language Models (LLMs) has raised concerns about their potential social biases. Existing evaluation methods, which rely on fixed-form outputs, are inadequate for the flexible open-text generation scenarios of LLMs. To address this, the authors introduce BiasAlert, a plug-and-play tool designed to detect social bias in open-text generations of LLMs. BiasAlert integrates external human knowledge with inherent reasoning capabilities to reliably detect bias. Extensive experiments show that BiasAlert outperforms existing state-of-the-art methods like GPT4-as-A-Judge in detecting bias. Application studies demonstrate the utility of BiasAlert in reliable LLM bias evaluation and mitigation across various scenarios. **Introduction:** LLMs, characterized by their extensive parameter sets and training datasets, have shown significant efficiency improvements but also exhibit social biases stemming from their training data. Evaluating these biases is crucial for enhancing fairness and reliability. Current methods, such as embedding or probability-based approaches and generated-text-based methods, often rely on fixed-form inputs and outputs, which are inadequate for open-text generation tasks. BiasAlert addresses this gap by integrating external human knowledge and enhancing internal reasoning capabilities. **Method:** BiasAlert takes generated content as input and outputs judgments and explanations. It constructs a social bias retrieval database and employs an instruction-following paradigm to enhance reasoning. The retrieval database includes real-world social biases from the SBIC dataset, standardized into refined corpora. Step-by-step instructions guide the model to identify specific groups and potential biased descriptions, define judgment criteria, and make judgments based on retrieved references. **Experiment and Analysis:** Experiments on RedditBias and Crows-pairs datasets show that BiasAlert outperforms existing tools and LLMs in detecting bias. Ablation studies validate the effectiveness of different components, including retrieval, step-by-step instructions, and in-context demonstrations. **Applications:** BiasAlert is validated for bias evaluation in open-text generation tasks and bias mitigation during LLM deployment. Results show that BiasAlert can significantly reduce the proportion of biased generations, demonstrating its utility in fairer and more reliable LLM evaluation and deployment. **Conclusion:** BiasAlert is a plug-and-play tool that effectively detects social bias in open-text generations of LLMs. Its superior performance and reliability make it an indispensable tool for fairer and more reliable LLM evaluation and deployment.**BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMs** **Abstract:** The rapid development of Large Language Models (LLMs) has raised concerns about their potential social biases. Existing evaluation methods, which rely on fixed-form outputs, are inadequate for the flexible open-text generation scenarios of LLMs. To address this, the authors introduce BiasAlert, a plug-and-play tool designed to detect social bias in open-text generations of LLMs. BiasAlert integrates external human knowledge with inherent reasoning capabilities to reliably detect bias. Extensive experiments show that BiasAlert outperforms existing state-of-the-art methods like GPT4-as-A-Judge in detecting bias. Application studies demonstrate the utility of BiasAlert in reliable LLM bias evaluation and mitigation across various scenarios. **Introduction:** LLMs, characterized by their extensive parameter sets and training datasets, have shown significant efficiency improvements but also exhibit social biases stemming from their training data. Evaluating these biases is crucial for enhancing fairness and reliability. Current methods, such as embedding or probability-based approaches and generated-text-based methods, often rely on fixed-form inputs and outputs, which are inadequate for open-text generation tasks. BiasAlert addresses this gap by integrating external human knowledge and enhancing internal reasoning capabilities. **Method:** BiasAlert takes generated content as input and outputs judgments and explanations. It constructs a social bias retrieval database and employs an instruction-following paradigm to enhance reasoning. The retrieval database includes real-world social biases from the SBIC dataset, standardized into refined corpora. Step-by-step instructions guide the model to identify specific groups and potential biased descriptions, define judgment criteria, and make judgments based on retrieved references. **Experiment and Analysis:** Experiments on RedditBias and Crows-pairs datasets show that BiasAlert outperforms existing tools and LLMs in detecting bias. Ablation studies validate the effectiveness of different components, including retrieval, step-by-step instructions, and in-context demonstrations. **Applications:** BiasAlert is validated for bias evaluation in open-text generation tasks and bias mitigation during LLM deployment. Results show that BiasAlert can significantly reduce the proportion of biased generations, demonstrating its utility in fairer and more reliable LLM evaluation and deployment. **Conclusion:** BiasAlert is a plug-and-play tool that effectively detects social bias in open-text generations of LLMs. Its superior performance and reliability make it an indispensable tool for fairer and more reliable LLM evaluation and deployment.