Skip to main content
CASF-2016 is a standard test set of protein–ligand complexes with crystal structures and measured binding affinities, built to compare docking and scoring methods on equal terms. This page scores two of our docking engines on it two ways: how closely the docked pose matches the crystal pose, and how well the score tracks measured affinity.
All results are ours. Static docking keeps the receptor rigid and sets the search box from the reference structure. The flexible-docking engine, built on GNINA, ranks poses with learned CNN scores. Pose accuracy is measured as the heavy-atom RMSD between each docked ligand and its crystal ligand, recomputed from the docked poses themselves.

​
What the comparison shows

Both engines generate a pose within 2 Å of the crystal pose for about two in three complexes (73% for static docking, 68% for flexible docking, best of up to nine poses). Picking that pose is the harder part. Static docking ranks a near-native pose first for 48% of complexes, and the flexible engine, with its learned pose score, does so for 53%. Both do less well on large ligands: for static docking, success falls from about 77% for mid-sized ligands to about 60% above 35 heavy atoms. On affinity, the flexible engine’s CNN prediction reaches a Pearson r of 0.77 against 0.57 for its physics score, and static docking’s raw score correlates at about 0.56 in magnitude.

​
Pose accuracy

The distance between the docked and crystal ligand, for every complex, measured as the best of up to nine poses. Static docking is in blue and flexible docking in grey. The two engines were not scored on identical sets: 1 static and 16 flexible complexes are left out because the ligand’s bond graph could not be matched to the crystal ligand, so the comparison is indicative rather than strictly paired. How success depends on the complex, for static docking (best of nine within 2 Å):

​
Affinity scoring

Each static-docking score against the measured affinity of the complex. The score is in kcal/mol, so tighter binders have more negative scores. The flexible-docking engine’s scoring, as reported for the same benchmark:

​
Reading these results

For every complex, each docked pose was compared with the crystal ligand using heavy atoms only. The poses are in the same frame as the crystal receptor, so no realignment is applied. The ligand’s bonds are perceived from the pose geometry, its atoms are matched to the crystal ligand by bond topology including symmetric equivalents, and the smallest RMSD over the matches is used. A complex is excluded when the pose’s bond graph does not match the crystal ligand’s, which affected 1 static and 16 flexible complexes.
The best-of-nine figure shows whether the engine generates a good pose at all. The top-ranked figure, using pose 1 as the engine ordered it, shows whether it also puts that pose first, which is what matters in use.
Both engines dock into the crystal receptor structure, and the poses share its coordinate frame, so the binding site is effectively known, the easier of the two docking tasks. Performance when the site is not known, or on a different receptor conformation, will be lower.
Learned scoring functions are trained on public protein–ligand data, and CASF-2016 complexes come from the same public archives. We have not checked for overlap between this test set and the data behind the CNN models, so treat the learned-scoring results as agreement with a public benchmark, not a strict test on unseen complexes.
The CNN and physics scoring figures are taken as reported for the flexible-docking engine. The per-complex scores behind them were not part of the files this page was built from, so they cannot be re-plotted here.
This page reports our engines on their own. It makes no comparison with other docking programs.

​
Sources

  • Benchmark: CASF-2016, the Comparative Assessment of Scoring Functions core set, with crystal poses and measured affinities from PDBbind.
  • Results: poses from our static and flexible docking runs, with RMSD recomputed from the poses against the CASF-2016 crystal ligands, and the flexible-docking engine’s scoring summary for the same set.