> ## Documentation Index
> Fetch the complete documentation index at: https://docs.revilico.bio/llms.txt
> Use this file to discover all available pages before exploring further.
# RevFEP Benchmarks
> How well RevFEP's default protocol reproduces measured binding free energies on the public JACS and Merck benchmark sets
## What This Page Shows
This page reports how closely [RevFEP](/docs/revfep) reproduces experimentally measured binding free energies on two public protein–ligand benchmark sets: the JACS set (Wang et al., 2015) and the Merck set (Schindler et al., 2020), using the curated structures from Ross et al. (2023).
Every number comes from a production RevFEP run with the default settings described in [How the Benchmarks Were Run](#how-the-benchmarks-were-run), so it reflects what you get when you submit a series with the defaults. For reference, each target also shows Schrödinger's published FEP+ results for the same ligands.
## Summary
Averages of the per-target statistics below. Each target counts once, whatever its size.
| Set | Targets | RMSE | MAE | R² | ρ | FEP+ RMSE | FEP+ MAE | FEP+ R² | FEP+ ρ |
| - | - | - | - | - | - | - | - | - | - |
| JACS | 6 | 1.00 | 0.77 | 0.52 | 0.68 | 0.68 | 0.54 | 0.63 | 0.77 |
| Merck | 11 | 1.43 | 1.18 | 0.29 | 0.38 | 0.85 | 0.69 | 0.49 | 0.64 |
| **Overall** | **17** | **1.28** | **1.04** | **0.37** | **0.48** | **0.79** | **0.64** | **0.54** | **0.68** |
RMSE and MAE are in kcal/mol. JNK1 and the redocked c-Met (+1) series are left out of the averages; see the notes under the per-target table.
## Results by Target
| Set | Target | Ligands | Edges | RMSE | MAE | R² | ρ | FEP+ RMSE | FEP+ MAE | FEP+ R² | FEP+ ρ |
| - | - | - | - | - | - | - | - | - | - | - | - |
| JACS | TYK2 | 16 | 15 | 0.63 | 0.51 | 0.86 | 0.89 | 0.47 | 0.37 | 0.86 | 0.93 |
| JACS | CDK2 | 16 | 15 | 0.69 | 0.62 | 0.73 | 0.86 | 0.90 | 0.72 | 0.42 | 0.62 |
| JACS | JNK1 \* | 38 | 37 | 1.15 | 0.88 | 0.07 | 0.16 | 0.57 | 0.45 | 0.72 | 0.89 |
| JACS | MCL1 | 41 | 40 | 1.46 | 1.11 | 0.35 | 0.60 | 0.86 | 0.68 | 0.50 | 0.67 |
| JACS | p38 | 34 | 33 | 1.65 | 1.13 | 0.06 | 0.27 | 0.81 | 0.63 | 0.49 | 0.72 |
| JACS | PTP1B | 23 | 22 | 1.04 | 0.79 | 0.50 | 0.71 | 0.51 | 0.41 | 0.85 | 0.88 |
| JACS | Thrombin | 11 | 10 | 0.50 | 0.46 | 0.64 | 0.73 | 0.53 | 0.42 | 0.68 | 0.78 |
| Merck | CDK8 ‡ | 33 | 32 | 1.53 | 1.31 | 0.47 | 0.74 | 1.13 | 0.93 | 0.56 | 0.76 |
| Merck | c-Met (neutral) | 11 | 10 | 1.33 | 1.01 | 0.46 | 0.63 | 0.86 | 0.76 | 0.57 | 0.79 |
| Merck | c-Met (+1) | 6 | 5 | 1.87 | 1.69 | 0.18 | −0.38 | 0.63 | 0.56 | 0.24 | 0.55 |
| Merck | c-Met (+1, redocked) † | 6 | 5 | 0.70 | 0.57 | 0.52 | 0.58 | 0.46 | 0.38 | 0.56 | 0.81 |
| Merck | Eg5 (neutral) | 9 | 8 | 1.32 | 1.09 | 0.59 | 0.54 | 0.64 | 0.55 | 0.54 | 0.74 |
| Merck | Eg5 (+1) | 19 | 18 | 2.05 | 1.64 | 0.07 | −0.21 | 0.94 | 0.68 | 0.23 | 0.46 |
| Merck | HIF-2α | 41 | 40 | 1.66 | 1.33 | 0.38 | 0.64 | 0.77 | 0.66 | 0.61 | 0.65 |
| Merck | PFKFB3 | 40 | 39 | 1.10 | 0.91 | 0.35 | 0.56 | 0.99 | 0.77 | 0.58 | 0.76 |
| Merck | SHP2 ‡ | 26 | 25 | 2.12 | 1.82 | 0.02 | 0.16 | 1.13 | 0.95 | 0.48 | 0.72 |
| Merck | SYK (neutral) ‡ | 38 | 37 | 1.04 | 0.75 | 0.31 | 0.52 | 0.71 | 0.55 | 0.48 | 0.63 |
| Merck | SYK (−1) ‡ | 4 | 3 | 0.73 | 0.67 | 0.11 | 0.50 | 0.39 | 0.32 | 0.79 | 0.50 |
| Merck | TNKS2 | 27 | 26 | 0.99 | 0.79 | 0.30 | 0.45 | 1.11 | 0.82 | 0.29 | 0.45 |
\* **JNK1:** the RevFEP run treats the two flipped binding poses of 17 molecules as separate ligands (38 entries for 21 molecules). FEP+ combines each pair into one value per molecule, so this row is not a like-for-like comparison and is left out of the averages.
† **c-Met (+1, redocked):** the same six ligands as c-Met (+1), started from redocked poses instead of the poses supplied with the benchmark. It is shown to illustrate how much the starting pose matters, and is left out of the averages.
‡ A few ligands have no FEP+ prediction in the public data: CDK8 (1 ligand), SHP2 (1), SYK neutral (3) and SYK −1 (1). For these targets both columns are computed on the ligands that have both predictions, so the RevFEP statistics can differ slightly from the figure below, which shows every ligand.
## How the Benchmarks Were Run
Every target was run through the standard RevFEP pipeline with its default settings (Protocol A). Nothing was tuned per target.
| Setting | Value |
| - | - |
| Engine | OpenFE 1.8.1, OpenMM 8.2, relative hybrid-topology protocol |
| Force fields | Amber ff14SB (protein), OpenFF 2.2.0 (ligands), TIP3P water, 0.15 M NaCl |
| Ligand charges | AM1-BCC |
| Network | LOMAP atom mapping, minimal spanning tree (one path between each pair of ligands) |
| Sampling | 11 λ windows, Hamiltonian replica exchange, 1 ns equilibration and 5 ns production per window |
| Integrator | 2 fs time step, hydrogen bonds constrained, no hydrogen mass repartitioning |
| Repeats | 1 |
| Solvent padding | 1.5 nm |
| Hardware | NVIDIA L40S GPUs on AWS (about 4.3 GPU-hours per edge, median) |
Structures, ligand poses and protonation states are the ones published with the benchmark. RevFEP does not change the net charge of a ligand along an edge, so series that mix charge states (c-Met, Eg5, SYK) were run as separate same-charge subsets. The published FEP+ results include charge-changing edges and run each of these series as one network.
## How the Numbers Are Calculated
* **ΔG per ligand.** RevFEP's relative free energies (ΔΔG) are combined into one ΔG per ligand. Because relative calculations fix ΔG only up to a constant, the predictions for each target are shifted so that their mean equals the mean experimental value. The FEP+ values are shifted the same way, over the same ligands.
* **RMSE and MAE** are the root-mean-square and mean absolute errors of ΔG, in kcal/mol.
* **R²** is the square of the Pearson correlation coefficient; **ρ** is Spearman's rank correlation.
* **FEP+ reference.** Ligand-level FEP+ (release 21-4) predictions are taken from Schrödinger's public benchmark repository, matched to RevFEP ligands by name. The experimental values in both data sets agree to within 0.01 kcal/mol for every matched ligand.
## Reading These Results
* **These are single runs.** Each edge was simulated once, and a minimal spanning tree has no closed cycles, so these results come without a statistical uncertainty or a cycle-closure check. Repeat runs and networks with cycles would add an error bar and a consistency check.
* **Small sets are noisy.** R² and ρ computed on fewer than about 15 ligands (c-Met, Eg5 neutral, SYK −1, Thrombin) can change a lot when one ligand moves.
* **The FEP+ results are not a like-for-like workflow.** They were produced with a different force field (OPLS4), a different sampling protocol, and per-target corrections for protonation states, tautomers and alternative ring flips that were applied by experts. RevFEP runs every target the same way with no manual corrections.
* **Charged series are harder.** The largest errors are on the +1 series (c-Met, Eg5) and on SHP2. The redocked c-Met run shows that the starting pose alone can move the RMSE from 1.87 to 0.70 kcal/mol for the same ligands.
* **Not yet covered.** BACE from the JACS set, and the other groups in Ross et al. (2023), have not been run with RevFEP.
## Sources
* Wang, L. et al. *Accurate and Reliable Prediction of Relative Ligand Binding Potency in Prospective Drug Discovery by Way of a Modern Free-Energy Calculation Protocol and Force Field.* J. Am. Chem. Soc. 137, 2695–2703 (2015). [doi:10.1021/ja512751q](https://doi.org/10.1021/ja512751q)
* Schindler, C. E. M. et al. *Large-Scale Assessment of Binding Free Energy Calculations in Active Drug Discovery Projects.* J. Chem. Inf. Model. 60, 5457–5474 (2020). [doi:10.1021/acs.jcim.0c00900](https://doi.org/10.1021/acs.jcim.0c00900)
* Ross, G. A. et al. *The maximal and current accuracy of rigorous protein-ligand binding free energy calculations.* Commun. Chem. 6, 222 (2023). [doi:10.1038/s42004-023-01019-9](https://doi.org/10.1038/s42004-023-01019-9)
* FEP+ 21-4 ligand predictions: [schrodinger/public\_binding\_free\_energy\_benchmark](https://github.com/schrodinger/public_binding_free_energy_benchmark), `21_4_results/ligand_predictions/`
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.