OffsetBias: Leveraging Debiased Data for Tuning Evaluators This paper identifies six types of biases in judge models and proposes EVALBIASBENCH, a benchmark for testing these biases. It also introduces OFFSETBIAS, a dataset designed to reduce these biases by providing counterexamples. The dataset is created using GPT-4 and Claude-3, and includes good and bad responses where the bad responses contain errors but are stylistically appealing. The dataset is used to train judge models, which significantly improves their performance on bias-related tasks. The paper also shows that incorporating OFFSETBIAS into training improves the robustness of judge models against biases and enhances their overall performance. The results demonstrate that fine-tuning on OFFSETBIAS enhances the robustness of judge models against biases and improves performance across various evaluation scenarios. The datasets and fine-tuned judge model are made publicly available.OffsetBias: Leveraging Debiased Data for Tuning Evaluators This paper identifies six types of biases in judge models and proposes EVALBIASBENCH, a benchmark for testing these biases. It also introduces OFFSETBIAS, a dataset designed to reduce these biases by providing counterexamples. The dataset is created using GPT-4 and Claude-3, and includes good and bad responses where the bad responses contain errors but are stylistically appealing. The dataset is used to train judge models, which significantly improves their performance on bias-related tasks. The paper also shows that incorporating OFFSETBIAS into training improves the robustness of judge models against biases and enhances their overall performance. The results demonstrate that fine-tuning on OFFSETBIAS enhances the robustness of judge models against biases and improves performance across various evaluation scenarios. The datasets and fine-tuned judge model are made publicly available.