> ## Documentation Index > Fetch the complete documentation index at: https://docs.revilico.bio/llms.txt > Use this file to discover all available pages before exploring further. # ADMET Property Prediction Benchmarks > ADMET predictions scored on molecules held out from training, across eight public benchmark sets from mutagenicity to clearance. How well do predicted ADMET properties agree with measured ones? This page scores our ADMET engine, which runs the ADMET-AI models, on eight public benchmark sets: four with yes-or-no outcomes and four with measured values. The models were built on public datasets that overlap with some of these benchmarks, so every headline number here uses only the benchmark molecules that are **not** in the matching training dataset. The effect of removing the overlap is shown for each endpoint below. > [!NOTE] > > All predictions are ours. The measured labels come from public benchmark sets, listed under Sources. A molecule is counted only if the engine produced a prediction for it, the benchmark has a label for it, and it is not in the matching public training dataset. ## What the comparison shows On molecules the models have not seen, the engine is strongest for blood–brain barrier penetration (AUROC 0.89) and solid for aqueous solubility and logD (correlations of 0.76 and 0.70). Ames mutagenicity looks very strong on the full benchmark (0.93) only because 80% of its molecules are in the public training data. On the 1,836 that are not, the AUROC is 0.75. Liver-injury (0.68) and hERG (0.70) predictions are moderate, and at the usual 0.5 cutoff the hERG model calls almost every molecule a blocker (it finds 94% of true blockers but only 17% of non-blockers). Human microsomal clearance correlates at only 0.16, and Caco-2 permeability at 0.30 on a small set of 271 molecules. ## How much the training overlap matters For each endpoint, the share of benchmark molecules that are also in the matching public training dataset, and the result with and without them. | Endpoint | Benchmark molecules | In training data | Held-out molecules | Metric | All molecules | Held-out only | | - | - | - | - | - | - | - | | Ames | 9,028 | 80% | 1,836 | AUROC | 0.93 | **0.75** | | Blood–brain barrier | 8,137 | 18% | 6,671 | AUROC | 0.90 | **0.89** | | Liver injury | 886 | 13% | 769 | AUROC | 0.72 | **0.68** | | hERG | 8,737 | 2% | 8,603 | AUROC | 0.70 | **0.70** | | logD | 12,953 | 27% | 9,467 | Pearson r | 0.75 | **0.70** | | Caco-2 | 271 | 0% | 271 | Pearson r | 0.30 | **0.30** | | Solubility | 11,478 | 21% | 9,028 | Pearson r | 0.80 | **0.76** | | Clearance | 2,284 | 0.5% | 2,273 | Pearson r | 0.16 | **0.16** | ## Yes-or-no endpoints AUROC measures how well the predicted probability separates positives from negatives (0.5 is chance, 1 is perfect). Sensitivity and specificity use a 0.5 probability cutoff. All values are for held-out molecules. ROC curves on held-out molecules ROC curves on held-out molecules Each curve plots the share of true positives found against the share of negatives wrongly flagged as the probability cutoff moves. The dashed diagonal is chance. Click a name to hide or show its curve. ## Measured-value endpoints Pearson r and Spearman ρ between predicted and measured values, on held-out molecules. | Endpoint | Held-out molecules | Pearson r | Spearman ρ | RMSE | MAE | | - | - | - | - | - | - | | logD | 9,467 | 0.70 | 0.71 | 1.10 | 0.78 | | Caco-2 (log Papp) | 271 | 0.30 | 0.27 | 0.77 | 0.61 | | Solubility | 9,028 | 0.76 | 0.78 | - | - | | Human microsomal clearance | 2,273 | 0.16 | 0.15 | - | - | For solubility and clearance, the predictions and the benchmark's values are on different scales, so only correlations are reported. ## Predicted against measured Pick an endpoint. Each point is one held-out molecule, a random sample of the full set; the statistics use every held-out molecule. Where prediction and measurement share units (logD and Caco-2) the dashed line is perfect agreement. ## How the comparison was made > **Predictions and labels** > > The engine produced predictions for 51,590 molecules. Each benchmark's labels were matched to those predictions by canonical SMILES, and molecules with no prediction or no label were dropped. Repeated molecules were counted once. > **Held-out molecules** > > The ADMET-AI models are built on public datasets from the Therapeutics Data Commons. For each endpoint we took the matching dataset (for example the Ames, blood–brain barrier, liver-injury, hERG, lipophilicity, Caco-2, solubility and microsomal-clearance sets) and removed every benchmark molecule that appears in it, in either its training or its test portion. Molecules are compared by canonical SMILES. > **How yes-or-no outcomes are defined** > > Ames and blood–brain barrier labels are the benchmarks' own yes-or-no classes. A hERG measurement of pIC50 5 or higher (10 µM or stronger) counts as a blocker. For liver injury, drugs with most or less DILI concern are positive and drugs with no concern are negative. A predicted probability of 0.5 or higher is a positive call. > **How the metrics are calculated** > > AUROC uses the predicted probability directly. Sensitivity, specificity and balanced accuracy use the 0.5 cutoff. For measured-value endpoints, r and ρ are over all held-out molecules, and RMSE and MAE are given only where prediction and label are in the same units. ## Reading these results > **Held-out is not the same as unseen** > > Removing molecules found in the matching public dataset is the check we can make. The models were trained on many public datasets, and a benchmark molecule could appear in one of the others, so the held-out numbers may still be slightly optimistic. > **The 0.5 cutoff matters** > > Sensitivity and specificity depend on the probability cutoff. The hERG result shows it clearly: at 0.5 the engine flags almost everything, but the AUROC of 0.70 shows the ranking still carries information, and a cutoff chosen for a project can trade sensitivity against specificity. > **No competing method is shown** > > This page reports the engine's own accuracy. It does not compare it with other ADMET predictors, so it makes no claim about being better or worse than them. > **Not every endpoint is here** > > The engine predicts many more properties, including cytochrome P450 inhibition, than appear on this page. Only endpoints with a clear label and unit mapping are scored here. ## Sources * **Labels:** PharmaBench (Ames, blood–brain barrier, logD, aqueous solubility and microsomal clearance), a DILIrank-based liver injury set, a curated Caco-2 permeability set, and a hERG inhibition set from Zenodo. * **Training datasets used to identify overlap:** Therapeutics Data Commons ADMET benchmark group. * **Predictions:** ADMET-AI models, run through Revilico's ADMET engine on 51,590 molecules. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.