Large-scale collaborative assessment of binding free energy calculations for drug discovery using OpenFE

Accurately measuring compound binding affinities is key to driving the pharmaceutical development process. Rigorous physics-based in silico approaches, particularly alchemical free energy methods, have become a gold standard tool for estimating compound affinity changes. Here we present the results of a large-scale pre-competitive collaborative assessment of relative binding free energy (RBFE) calculations generated by 15 pharmaceutical companies. We evaluate an open-source and MIT licensed RBFE Protocol from the Open Free Energy (OpenFE) ecosystem across both public and blinded private datasets, encompassing over 1,700 ligands in total. For the public dataset, the weighted RMSE across the 58 systems was 1.73 [1.53, 1.96] kcal/mol, with 10 of the systems reaching sub-kcal/mol accuracy. For the private dataset, the weighted RMSE across the 37 systems was 2.44 [1.94, 3.06] kcal/mol, with only 2 of the systems reaching sub-kcal/mol accuracy, reflecting the increased complexity of real-world drug discovery. The protocols performance was system-dependent, with no single dominant error source, indicating that accuracy is primarily influenced by input quality and transformation type. Overall, these benchmark results are encouraging and indicate that OpenFE is ready for large-scale industrial applications, with an "out-of-the-box" accuracy that approaches that of commercial solutions. While comparison against published FEP+ results, which were obtained after manual parameter optimization, shows comparable ranking statistics, a gap remains in error statistics, for which we outline possible paths toward improvement. The protocol meets key criteria required for production use in an industrial setting: it shows robust performance, generates reproducible results, and achieves both sufficient throughput and rapid convergence.