Skip to main content
How well do predicted ADMET properties agree with measured ones? This page scores our ADMET engine, which runs the ADMET-AI models, on eight public benchmark sets: four with yes-or-no outcomes and four with measured values. The models were built on public datasets that overlap with some of these benchmarks, so every headline number here uses only the benchmark molecules that are not in the matching training dataset. The effect of removing the overlap is shown for each endpoint below.
All predictions are ours. The measured labels come from public benchmark sets, listed under Sources. A molecule is counted only if the engine produced a prediction for it, the benchmark has a label for it, and it is not in the matching public training dataset.

​
What the comparison shows

On molecules the models have not seen, the engine is strongest for blood–brain barrier penetration (AUROC 0.89) and solid for aqueous solubility and logD (correlations of 0.76 and 0.70). Ames mutagenicity looks very strong on the full benchmark (0.93) only because 80% of its molecules are in the public training data. On the 1,836 that are not, the AUROC is 0.75. Liver-injury (0.68) and hERG (0.70) predictions are moderate, and at the usual 0.5 cutoff the hERG model calls almost every molecule a blocker (it finds 94% of true blockers but only 17% of non-blockers). Human microsomal clearance correlates at only 0.16, and Caco-2 permeability at 0.30 on a small set of 271 molecules.

​
How much the training overlap matters

For each endpoint, the share of benchmark molecules that are also in the matching public training dataset, and the result with and without them.

​
Yes-or-no endpoints

AUROC measures how well the predicted probability separates positives from negatives (0.5 is chance, 1 is perfect). Sensitivity and specificity use a 0.5 probability cutoff. All values are for held-out molecules.

​
Measured-value endpoints

Pearson r and Spearman ρ between predicted and measured values, on held-out molecules. For solubility and clearance, the predictions and the benchmark’s values are on different scales, so only correlations are reported.

​
Predicted against measured

Pick an endpoint. Each point is one held-out molecule, a random sample of the full set; the statistics use every held-out molecule. Where prediction and measurement share units (logD and Caco-2) the dashed line is perfect agreement.

​
How the comparison was made

The engine produced predictions for 51,590 molecules. Each benchmark’s labels were matched to those predictions by canonical SMILES, and molecules with no prediction or no label were dropped. Repeated molecules were counted once.
The ADMET-AI models are built on public datasets from the Therapeutics Data Commons. For each endpoint we took the matching dataset (for example the Ames, blood–brain barrier, liver-injury, hERG, lipophilicity, Caco-2, solubility and microsomal-clearance sets) and removed every benchmark molecule that appears in it, in either its training or its test portion. Molecules are compared by canonical SMILES.
Ames and blood–brain barrier labels are the benchmarks’ own yes-or-no classes. A hERG measurement of pIC50 5 or higher (10 µM or stronger) counts as a blocker. For liver injury, drugs with most or less DILI concern are positive and drugs with no concern are negative. A predicted probability of 0.5 or higher is a positive call.
AUROC uses the predicted probability directly. Sensitivity, specificity and balanced accuracy use the 0.5 cutoff. For measured-value endpoints, r and ρ are over all held-out molecules, and RMSE and MAE are given only where prediction and label are in the same units.

​
Reading these results

Removing molecules found in the matching public dataset is the check we can make. The models were trained on many public datasets, and a benchmark molecule could appear in one of the others, so the held-out numbers may still be slightly optimistic.
Sensitivity and specificity depend on the probability cutoff. The hERG result shows it clearly: at 0.5 the engine flags almost everything, but the AUROC of 0.70 shows the ranking still carries information, and a cutoff chosen for a project can trade sensitivity against specificity.
This page reports the engine’s own accuracy. It does not compare it with other ADMET predictors, so it makes no claim about being better or worse than them.
The engine predicts many more properties, including cytochrome P450 inhibition, than appear on this page. Only endpoints with a clear label and unit mapping are scored here.

​
Sources

  • Labels: PharmaBench (Ames, blood–brain barrier, logD, aqueous solubility and microsomal clearance), a DILIrank-based liver injury set, a curated Caco-2 permeability set, and a hERG inhibition set from Zenodo.
  • Training datasets used to identify overlap: Therapeutics Data Commons ADMET benchmark group.
  • Predictions: ADMET-AI models, run through Revilico’s ADMET engine on 51,590 molecules.