The article evaluates the performance of six small-molecule force fields (OpenFF, Parsley, Sage, GAFF, CGenFF, and OPLS3e) in predicting protein–ligand binding affinities using alchemical relative binding free energy (RBFE) calculations. The study uses a benchmark dataset of 598 ligands and 22 protein targets. The results show that while OpenFF, Parsley, and Sage perform comparably based on aggregated statistics, OPLS3e is significantly more accurate. A consensus approach using Sage, GAFF, and CGenFF achieves accuracy comparable to OPLS3e. The analysis reveals that improved parameters can significantly enhance prediction accuracy, particularly for subsets involving those parameters. However, lower accuracy can also be attributed to input preparation, sampling convergence, and the complexity of the perturbations. The study highlights the importance of careful system preparation and the need for enhanced sampling protocols to improve the accuracy of RBFE calculations. The input structures, Gromacs force field files, and analysis notebooks are available on GitHub for reproducibility.The article evaluates the performance of six small-molecule force fields (OpenFF, Parsley, Sage, GAFF, CGenFF, and OPLS3e) in predicting protein–ligand binding affinities using alchemical relative binding free energy (RBFE) calculations. The study uses a benchmark dataset of 598 ligands and 22 protein targets. The results show that while OpenFF, Parsley, and Sage perform comparably based on aggregated statistics, OPLS3e is significantly more accurate. A consensus approach using Sage, GAFF, and CGenFF achieves accuracy comparable to OPLS3e. The analysis reveals that improved parameters can significantly enhance prediction accuracy, particularly for subsets involving those parameters. However, lower accuracy can also be attributed to input preparation, sampling convergence, and the complexity of the perturbations. The study highlights the importance of careful system preparation and the need for enhanced sampling protocols to improve the accuracy of RBFE calculations. The input structures, Gromacs force field files, and analysis notebooks are available on GitHub for reproducibility.