> ## Documentation Index
> Fetch the complete documentation index at: https://docs.revilico.bio/llms.txt
> Use this file to discover all available pages before exploring further.

# RevFEP Benchmarks

> How well RevFEP's default protocol reproduces measured binding free energies on the public JACS and Merck benchmark sets

export const FepBenchmarkDetails = ({targets}) => {
  const COLORS = {
    r: "#1371e4",
    f: "#f97316"
  };
  const NAMES = {
    r: "RevFEP",
    f: "FEP+"
  };
  const border = "1px solid rgba(128, 128, 128, 0.25)";
  const fmt = v => v == null ? "—" : (v < 0 ? "−" : "") + Math.abs(v).toFixed(2);
  const signed = v => v == null ? "—" : (v > 0 ? "+" : v < 0 ? "−" : "") + Math.abs(v).toFixed(2);
  const safe = n => n.replace(/[^A-Za-z0-9._-]/g, "_");
  const [selected, setSelected] = useState(targets[0].id);
  const [show, setShow] = useState("both");
  const [bandOn, setBandOn] = useState(true);
  const [ligand, setLigand] = useState(null);
  const [hover, setHover] = useState(null);
  const [sortKey, setSortKey] = useState("err_r");
  useEffect(() => {
    const onSelect = e => {
      if (targets.some(t => t.id === e.detail)) {
        setSelected(e.detail);
        setLigand(null);
      }
    };
    window.addEventListener("fep-benchmark-select", onSelect);
    return () => window.removeEventListener("fep-benchmark-select", onSelect);
  }, []);
  const target = targets.find(t => t.id === selected);
  const keys = show === "both" ? ["r", "f"] : [show];
  const values = target.points.flatMap(p => [p.e, ...keys.map(k => p[k])]).filter(v => v != null);
  const lo = Math.floor(Math.min(...values) - 0.5);
  const hi = Math.ceil(Math.max(...values) + 0.5);
  const step = hi - lo > 8 ? 2 : 1;
  const ticks = [];
  for (let v = lo; v <= hi; v += step) ticks.push(v);
  const W = 560;
  const PAD = 58;
  const TOP = 14;
  const span = W - PAD - TOP;
  const sx = v => PAD + (v - lo) / (hi - lo) * span;
  const sy = v => W - PAD - (v - lo) / (hi - lo) * span;
  const band = [[lo, lo + 1], [hi, hi + 1], [hi, hi - 1], [lo, lo - 1]].map(([x, y]) => `${sx(x)},${sy(y)}`).join(" ");
  const pill = active => ({
    padding: "5px 14px",
    borderRadius: "999px",
    border,
    cursor: "pointer",
    fontSize: "13px",
    background: active ? "#1371e4" : "transparent",
    color: active ? "#ffffff" : "inherit"
  });
  const cell = {
    padding: "7px 10px",
    borderBottom: border,
    textAlign: "right",
    whiteSpace: "nowrap",
    fontVariantNumeric: "tabular-nums"
  };
  const METRICS = [{
    key: "r2",
    short: "R²",
    lower: false
  }, {
    key: "rho",
    short: "ρ",
    lower: false
  }, {
    key: "mae",
    short: "MAE",
    lower: true
  }, {
    key: "rmse",
    short: "RMSE",
    lower: true
  }];
  const winner = m => {
    const a = target.revfep[m.key];
    const b = target.fepplus[m.key];
    if (a == null || b == null || a === b) return null;
    return (m.lower ? a < b : a > b) ? "revfep" : "fepplus";
  };
  const rows = target.points.map(p => ({
    ...p,
    err_r: p.r == null ? null : p.r - p.e,
    err_f: p.f == null ? null : p.f - p.e
  }));
  const sortVal = row => {
    if (sortKey === "l") return row.l;
    const v = row[sortKey];
    return v == null ? -Infinity : sortKey.startsWith("err") ? Math.abs(v) : v;
  };
  rows.sort((a, b) => {
    const va = sortVal(a);
    const vb = sortVal(b);
    return typeof va === "string" ? va.localeCompare(vb, undefined, {
      numeric: true
    }) : vb - va;
  });
  const shown = ligand || hover;
  const SETS = ["JACS", "Merck"];
  return <div className="fep-wide" style={{
    border,
    borderRadius: "12px",
    padding: "18px",
    margin: "16px 0"
  }}>
      <div style={{
    display: "flex",
    flexDirection: "column",
    gap: "8px",
    marginBottom: "16px"
  }}>
        {SETS.map(s => <div key={s} style={{
    display: "flex",
    flexWrap: "wrap",
    gap: "6px",
    alignItems: "center"
  }}>
            <span style={{
    fontSize: "12px",
    fontWeight: 700,
    letterSpacing: "0.04em",
    textTransform: "uppercase",
    opacity: 0.7,
    width: "86px"
  }}>
              {s} set
            </span>
            {targets.filter(t => t.set === s).map(t => <button key={t.id} type="button" onClick={() => {
    setSelected(t.id);
    setLigand(null);
    setHover(null);
  }} style={{
    ...pill(t.id === selected),
    padding: "4px 11px"
  }}>
                {t.name}{t.excluded ? " †" : ""} <span style={{
    opacity: 0.65,
    fontSize: "11.5px"
  }}>{t.points.length}</span>
              </button>)}
          </div>)}
      </div>
      <div>
        <div>
          <div style={{
    display: "flex",
    flexWrap: "wrap",
    alignItems: "baseline",
    gap: "10px",
    marginBottom: "4px"
  }}>
            <span style={{
    fontWeight: 700,
    fontSize: "22px"
  }}>{target.name}</span>
            <span style={{
    opacity: 0.7,
    fontSize: "14px"
  }}>
              {target.set} set · {target.points.length} ligands · statistics on the {target.fepplus.n} with both predictions
            </span>
          </div>
          {target.note && <div style={{
    margin: "8px 0",
    padding: "8px 12px",
    borderRadius: "8px",
    border,
    fontSize: "13px"
  }}>
              {target.excluded ? "Left out of the averages. " : ""}{target.note}
            </div>}
          <div style={{
    display: "flex",
    flexWrap: "wrap",
    gap: "8px",
    alignItems: "center",
    margin: "10px 0"
  }}>
            {[["both", "Both"], ["r", "RevFEP"], ["f", "FEP+"]].map(([k, label]) => <button key={k} type="button" onClick={() => setShow(k)} style={pill(show === k)}>{label}</button>)}
            <label style={{
    fontSize: "13px",
    marginLeft: "8px",
    cursor: "pointer"
  }}>
              <input type="checkbox" checked={bandOn} onChange={e => setBandOn(e.target.checked)} style={{
    marginRight: "6px"
  }} />
              Show ±1 kcal/mol band
            </label>
          </div>

          <div style={{
    display: "flex",
    flexWrap: "wrap",
    gap: "20px",
    alignItems: "flex-start"
  }}>
            <svg viewBox={`0 0 ${W} ${W}`} style={{
    width: "100%",
    maxWidth: "560px",
    flex: "1 1 420px",
    fontSize: "13px"
  }} role="img" aria-label={`Predicted against experimental binding free energy for ${target.name}`}>
              <defs>
                <clipPath id="fep-details-plot-area">
                  <rect x={PAD} y={TOP} width={span} height={span} />
                </clipPath>
              </defs>
              {ticks.map(v => <g key={v}>
                  <line x1={sx(v)} x2={sx(v)} y1={TOP} y2={W - PAD} stroke="currentColor" strokeOpacity="0.08" />
                  <line x1={PAD} x2={W - TOP} y1={sy(v)} y2={sy(v)} stroke="currentColor" strokeOpacity="0.08" />
                  <text x={sx(v)} y={W - PAD + 20} textAnchor="middle" fill="currentColor" fillOpacity="0.7">{fmt(v).replace(".00", "")}</text>
                  <text x={PAD - 8} y={sy(v) + 4} textAnchor="end" fill="currentColor" fillOpacity="0.7">{fmt(v).replace(".00", "")}</text>
                </g>)}
              {bandOn && <polygon points={band} fill="currentColor" fillOpacity="0.07" clipPath="url(#fep-details-plot-area)" />}
              <line x1={sx(lo)} y1={sy(lo)} x2={sx(hi)} y2={sy(hi)} stroke="currentColor" strokeOpacity="0.45" strokeDasharray="5 5" />
              <rect x={PAD} y={TOP} width={span} height={span} fill="none" stroke="currentColor" strokeOpacity="0.3" />
              {shown && shown.r != null && shown.f != null && show === "both" && <line x1={sx(shown.e)} y1={sy(shown.r)} x2={sx(shown.e)} y2={sy(shown.f)} stroke="currentColor" strokeOpacity="0.4" />}
              {keys.map(k => target.points.filter(p => p[k] != null).map(p => {
    const active = shown && shown.l === p.l;
    return <circle key={`${k}-${p.l}`} cx={sx(p.e)} cy={sy(p[k])} r={active ? 9 : 6} fill={COLORS[k]} fillOpacity={active ? 1 : 0.72} stroke={active ? "currentColor" : "#ffffff"} strokeWidth={active ? 2 : 0.8} style={{
      cursor: "pointer"
    }} onMouseEnter={() => setHover(p)} onMouseLeave={() => setHover(null)} onClick={() => setLigand(p)}>
                      <title>{`${p.l}: experimental ${fmt(p.e)}, ${NAMES[k]} ${fmt(p[k])} kcal/mol`}</title>
                    </circle>;
  }))}
              <text x={PAD + span / 2} y={W - 12} textAnchor="middle" fill="currentColor" fontSize="14">Experimental ΔG (kcal/mol)</text>
              <text transform={`translate(16 ${TOP + span / 2}) rotate(-90)`} textAnchor="middle" fill="currentColor" fontSize="14">Predicted ΔG (kcal/mol)</text>
              <g transform={`translate(${PAD + 12} ${TOP + 18})`}>
                {keys.map((k, i) => <g key={k} transform={`translate(0 ${i * 20})`}>
                    <circle cx="6" cy="-4" r="6" fill={COLORS[k]} />
                    <text x="18" y="0" fill="currentColor">{k === "r" ? "RevFEP (our runs)" : "FEP+ 21-4 (published)"}</text>
                  </g>)}
              </g>
            </svg>

            <div style={{
    flex: "1 1 260px",
    minWidth: "240px"
  }}>
              <div style={{
    border,
    borderRadius: "10px",
    padding: "12px",
    marginBottom: "14px"
  }}>
                {shown ? <div>
                    <div style={{
    background: "#ffffff",
    borderRadius: "8px",
    display: "flex",
    justifyContent: "center",
    marginBottom: "10px"
  }}>
                      <img src={`/images/benchmarks/ligands/${target.id}/${safe(shown.l)}.svg`} alt={`2D structure of ligand ${shown.l}`} style={{
    width: "100%",
    maxWidth: "320px",
    height: "auto",
    margin: 0
  }} />
                    </div>
                    <div style={{
    fontWeight: 700,
    fontSize: "16px",
    marginBottom: "6px",
    wordBreak: "break-all"
  }}>{shown.l}</div>
                    <table style={{
    width: "100%",
    borderCollapse: "collapse",
    fontSize: "13.5px",
    margin: 0
  }}>
                      <tbody>
                        <tr>
                          <td style={{
    ...cell,
    textAlign: "left"
  }}>Experimental ΔG</td>
                          <td style={cell}>{fmt(shown.e)}</td>
                          <td style={cell}></td>
                        </tr>
                        {["r", "f"].map(k => <tr key={k}>
                            <td style={{
    ...cell,
    textAlign: "left",
    color: COLORS[k],
    fontWeight: 600
  }}>{NAMES[k]}</td>
                            <td style={cell}>{fmt(shown[k])}</td>
                            <td style={{
    ...cell,
    opacity: 0.8
  }}>{shown[k] == null ? "no prediction" : `error ${signed(shown[k] - shown.e)}`}</td>
                          </tr>)}
                      </tbody>
                    </table>
                    <div style={{
    fontSize: "12px",
    opacity: 0.65,
    marginTop: "6px"
  }}>kcal/mol. Structure as run by RevFEP.</div>
                  </div> : <div style={{
    fontSize: "14px",
    opacity: 0.7,
    padding: "24px 6px",
    textAlign: "center"
  }}>
                    Click a point, or a row in the ligand table, to inspect the molecule and its values.
                  </div>}
              </div>

              <table style={{
    width: "100%",
    borderCollapse: "collapse",
    fontSize: "13.5px",
    margin: 0
  }}>
                <thead>
                  <tr>
                    <th style={{
    ...cell,
    textAlign: "left"
  }}>Method</th>
                    {METRICS.map(m => <th key={m.key} style={cell}>{m.short}</th>)}
                  </tr>
                </thead>
                <tbody>
                  {[["revfep", "RevFEP", "r"], ["fepplus", "FEP+", "f"]].map(([key, label, c]) => <tr key={key}>
                      <td style={{
    ...cell,
    textAlign: "left",
    color: COLORS[c],
    fontWeight: 600
  }}>{label}</td>
                      {METRICS.map(m => <td key={m.key} style={{
    ...cell,
    fontWeight: winner(m) === key ? 700 : 400
  }}>{fmt(target[key][m.key])}</td>)}
                    </tr>)}
                </tbody>
              </table>
              <div style={{
    fontSize: "12px",
    opacity: 0.65,
    marginTop: "6px"
  }}>
                MAE and RMSE in kcal/mol; ρ is the Spearman rank correlation. Bold: the better value. Dashed line: perfect agreement.
              </div>
            </div>
          </div>

          <div style={{
    marginTop: "18px"
  }}>
            <div style={{
    fontWeight: 600,
    fontSize: "15px",
    marginBottom: "6px"
  }}>
              Ligands <span style={{
    fontWeight: 400,
    fontSize: "12px",
    opacity: 0.7
  }}>· click a column to sort, or a row to inspect</span>
            </div>
            <div style={{
    maxHeight: "340px",
    overflowY: "auto",
    overflowX: "auto",
    border,
    borderRadius: "8px"
  }}>
              <table style={{
    width: "100%",
    borderCollapse: "collapse",
    fontSize: "13px",
    margin: 0
  }}>
                <thead style={{
    position: "sticky",
    top: 0,
    background: "var(--background, inherit)"
  }}>
                  <tr>
                    {[["l", "Ligand"], ["e", "Experimental"], ["r", "RevFEP"], ["err_r", "RevFEP error"], ["f", "FEP+"], ["err_f", "FEP+ error"]].map(([k, label]) => <th key={k} onClick={() => setSortKey(k)} style={{
    ...cell,
    textAlign: k === "l" ? "left" : "right",
    cursor: "pointer",
    userSelect: "none",
    backdropFilter: "blur(8px)"
  }}>
                        {label}{sortKey === k ? " ▼" : ""}
                      </th>)}
                  </tr>
                </thead>
                <tbody>
                  {rows.map(p => <tr key={p.l} onClick={() => setLigand(p)} style={{
    cursor: "pointer",
    background: ligand && ligand.l === p.l ? "rgba(19, 113, 228, 0.12)" : "transparent"
  }}>
                      <td style={{
    ...cell,
    textAlign: "left"
  }}>{p.l}</td>
                      <td style={cell}>{fmt(p.e)}</td>
                      <td style={{
    ...cell,
    color: COLORS.r
  }}>{fmt(p.r)}</td>
                      <td style={{
    ...cell,
    fontWeight: p.err_r != null && Math.abs(p.err_r) > 2 ? 700 : 400
  }}>{signed(p.err_r)}</td>
                      <td style={{
    ...cell,
    color: COLORS.f
  }}>{fmt(p.f)}</td>
                      <td style={{
    ...cell,
    fontWeight: p.err_f != null && Math.abs(p.err_f) > 2 ? 700 : 400
  }}>{signed(p.err_f)}</td>
                    </tr>)}
                </tbody>
              </table>
            </div>
            <div style={{
    fontSize: "12px",
    opacity: 0.65,
    marginTop: "6px"
  }}>
              ΔG in kcal/mol. Error = predicted − experimental; errors larger than 2 kcal/mol are bold. Error columns sort by size.
            </div>
          </div>
        </div>
      </div>
    </div>;
};

export const FepBenchmarkOverview = ({targets, averages}) => {
  const METRICS = [{
    key: "rmse",
    label: "RMSE",
    lower: true
  }, {
    key: "mae",
    label: "MAE",
    lower: true
  }, {
    key: "r2",
    label: "R²",
    lower: false
  }, {
    key: "rho",
    label: "ρ",
    lower: false
  }];
  const METHODS = [{
    key: "revfep",
    label: "RevFEP (our runs)",
    color: "#1371e4"
  }, {
    key: "fepplus",
    label: "FEP+ 21-4 (published)",
    color: "#f97316"
  }];
  const SETS = [{
    key: "JACS",
    label: "JACS set (Wang et al. 2015)"
  }, {
    key: "Merck",
    label: "Merck set (Schindler et al. 2020)"
  }];
  const border = "1px solid rgba(128, 128, 128, 0.25)";
  const cell = {
    padding: "7px 10px",
    borderBottom: border,
    textAlign: "right",
    whiteSpace: "nowrap",
    fontVariantNumeric: "tabular-nums"
  };
  const fmt = v => v == null ? "—" : (v < 0 ? "−" : "") + Math.abs(v).toFixed(2);
  const best = (stats, m) => {
    const a = stats.revfep[m.key];
    const b = stats.fepplus[m.key];
    if (a == null || b == null || a === b) return null;
    return (m.lower ? a < b : a > b) ? "revfep" : "fepplus";
  };
  const select = id => {
    window.dispatchEvent(new CustomEvent("fep-benchmark-select", {
      detail: id
    }));
    const el = document.getElementById("benchmark-details");
    if (el) el.scrollIntoView({
      behavior: "smooth",
      block: "start"
    });
  };
  const metricCells = (stats, strong) => METHODS.flatMap(meth => METRICS.map(m => {
    const win = best(stats, m) === meth.key;
    return <td key={`${meth.key}-${m.key}`} style={{
      ...cell,
      fontWeight: win || strong ? 700 : 400,
      borderLeft: m.key === "rmse" ? border : undefined,
      background: win ? `${meth.color}14` : undefined
    }}>
            {fmt(stats[meth.key][m.key])}
          </td>;
  }));
  const averageRow = (label, stats, key) => <tr key={key} style={{
    background: "rgba(128, 128, 128, 0.08)"
  }}>
      <td style={{
    ...cell,
    textAlign: "left",
    fontWeight: 700
  }}>{label}</td>
      <td style={cell}>{stats.n}</td>
      {metricCells(stats, false)}
    </tr>;
  return <div className="fep-wide" style={{
    border,
    borderRadius: "12px",
    overflow: "hidden",
    margin: "16px 0"
  }}>
      <div style={{
    overflowX: "auto"
  }}>
        <table style={{
    width: "100%",
    borderCollapse: "collapse",
    fontSize: "14px",
    margin: 0,
    minWidth: "860px"
  }}>
          <thead>
            <tr>
              <th rowSpan={2} style={{
    ...cell,
    textAlign: "left",
    verticalAlign: "bottom"
  }}>Target</th>
              <th rowSpan={2} style={{
    ...cell,
    verticalAlign: "bottom"
  }}>N</th>
              {METHODS.map(meth => <th key={meth.key} colSpan={METRICS.length} style={{
    ...cell,
    textAlign: "center",
    borderLeft: border,
    color: meth.color,
    fontSize: "15px"
  }}>
                  {meth.label}
                </th>)}
            </tr>
            <tr>
              {METHODS.flatMap(meth => METRICS.map(m => <th key={`${meth.key}-${m.key}`} style={{
    ...cell,
    borderLeft: m.key === "rmse" ? border : undefined
  }}>
                    {m.label}
                  </th>))}
            </tr>
          </thead>
          <tbody>
            {SETS.map(s => [<tr key={`${s.key}-head`}>
                <td colSpan={2 + METHODS.length * METRICS.length} style={{
    ...cell,
    textAlign: "left",
    fontWeight: 700,
    fontSize: "13px",
    letterSpacing: "0.04em",
    textTransform: "uppercase",
    opacity: 0.75,
    paddingTop: "16px"
  }}>
                  {s.label}
                </td>
              </tr>, ...targets.filter(t => t.set === s.key).map(t => <tr key={t.id} onClick={() => select(t.id)} title="Show this target in Benchmark details" style={{
    cursor: "pointer",
    opacity: t.excluded ? 0.6 : 1
  }}>
                  <td style={{
    ...cell,
    textAlign: "left",
    fontWeight: 600
  }}>
                    {t.name}{t.excluded ? " †" : ""}
                  </td>
                  <td style={cell}>{t.fepplus.n}</td>
                  {metricCells(t, false)}
                </tr>), averageRow(`${s.key} average`, averages[s.key], `${s.key}-avg`)])}
            {averageRow("Overall average", averages.Overall, "overall-avg")}
          </tbody>
        </table>
      </div>
      <div style={{
    fontSize: "12.5px",
    opacity: 0.75,
    padding: "10px 14px",
    borderTop: border,
    lineHeight: 1.5
  }}>
        N: ligands with both predictions, which the statistics use. RMSE and MAE in kcal/mol; R² is the squared Pearson
        correlation; ρ is the Spearman rank correlation. Bold and shaded: the better value in the row. Averages are means
        of the per-target statistics; N in an average row is the number of targets. † Shown for reference and left out of
        the averages (see the notes below). Click a target to open it in Benchmark details.
      </div>
    </div>;
};

export const benchmarkAverages = {
  "Overall": {
    "n": 17,
    "revfep": {
      "rmse": 1.28,
      "mae": 1.04,
      "r2": 0.37,
      "rho": 0.48
    },
    "fepplus": {
      "rmse": 0.79,
      "mae": 0.64,
      "r2": 0.54,
      "rho": 0.68
    }
  },
  "JACS": {
    "n": 6,
    "revfep": {
      "rmse": 1.00,
      "mae": 0.77,
      "r2": 0.52,
      "rho": 0.68
    },
    "fepplus": {
      "rmse": 0.68,
      "mae": 0.54,
      "r2": 0.63,
      "rho": 0.77
    }
  },
  "Merck": {
    "n": 11,
    "revfep": {
      "rmse": 1.43,
      "mae": 1.18,
      "r2": 0.29,
      "rho": 0.38
    },
    "fepplus": {
      "rmse": 0.85,
      "mae": 0.69,
      "r2": 0.49,
      "rho": 0.64
    }
  }
};

export const benchmarkTargets = [{
  "id": "tyk2",
  "name": "TYK2",
  "set": "JACS",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 16,
    "rmse": 0.63,
    "mae": 0.51,
    "r2": 0.86,
    "rho": 0.89
  },
  "fepplus": {
    "n": 16,
    "rmse": 0.47,
    "mae": 0.37,
    "r2": 0.86,
    "rho": 0.93
  },
  "points": [{
    "l": "ejm_31",
    "e": -9.54,
    "r": -9.69,
    "f": -9.91
  }, {
    "l": "ejm_42",
    "e": -9.78,
    "r": -9.21,
    "f": -9.95
  }, {
    "l": "ejm_43",
    "e": -8.26,
    "r": -7.31,
    "f": -8.3
  }, {
    "l": "ejm_44",
    "e": -7.42,
    "r": -5.97,
    "f": -6.48
  }, {
    "l": "ejm_45",
    "e": -9.56,
    "r": -8.8,
    "f": -9.66
  }, {
    "l": "ejm_46",
    "e": -11.31,
    "r": -11.06,
    "f": -10.82
  }, {
    "l": "ejm_47",
    "e": -9.7,
    "r": -9.61,
    "f": -9.94
  }, {
    "l": "ejm_48",
    "e": -9.0,
    "r": -9.18,
    "f": -8.86
  }, {
    "l": "ejm_49",
    "e": -7.75,
    "r": -8.62,
    "f": -8.05
  }, {
    "l": "ejm_50",
    "e": -8.98,
    "r": -9.26,
    "f": -9.4
  }, {
    "l": "ejm_54",
    "e": -10.53,
    "r": -10.55,
    "f": -10.38
  }, {
    "l": "ejm_55",
    "e": -9.21,
    "r": -9.97,
    "f": -10.34
  }, {
    "l": "jmc_23",
    "e": -11.7,
    "r": -12.07,
    "f": -11.27
  }, {
    "l": "jmc_27",
    "e": -11.28,
    "r": -11.9,
    "f": -11.1
  }, {
    "l": "jmc_28",
    "e": -10.98,
    "r": -11.14,
    "f": -10.34
  }, {
    "l": "jmc_30",
    "e": -10.94,
    "r": -11.61,
    "f": -11.13
  }]
}, {
  "id": "cdk2",
  "name": "CDK2",
  "set": "JACS",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 16,
    "rmse": 0.69,
    "mae": 0.62,
    "r2": 0.73,
    "rho": 0.86
  },
  "fepplus": {
    "n": 16,
    "rmse": 0.9,
    "mae": 0.72,
    "r2": 0.42,
    "rho": 0.62
  },
  "points": [{
    "l": "17",
    "e": -7.04,
    "r": -8.16,
    "f": -8.83
  }, {
    "l": "1h1q",
    "e": -8.18,
    "r": -7.35,
    "f": -7.82
  }, {
    "l": "1h1r",
    "e": -7.67,
    "r": -8.15,
    "f": -8.59
  }, {
    "l": "1h1s",
    "e": -11.25,
    "r": -10.96,
    "f": -9.97
  }, {
    "l": "1oi9",
    "e": -9.74,
    "r": -10.46,
    "f": -9.44
  }, {
    "l": "1oiu",
    "e": -9.08,
    "r": -9.9,
    "f": -10.7
  }, {
    "l": "1oiy",
    "e": -9.79,
    "r": -9.65,
    "f": -9.94
  }, {
    "l": "20",
    "e": -8.72,
    "r": -8.19,
    "f": -9.06
  }, {
    "l": "21",
    "e": -7.83,
    "r": -7.49,
    "f": -8.17
  }, {
    "l": "22",
    "e": -7.86,
    "r": -6.86,
    "f": -7.97
  }, {
    "l": "26",
    "e": -8.43,
    "r": -9.14,
    "f": -8.84
  }, {
    "l": "28",
    "e": -11.11,
    "r": -11.48,
    "f": -10.37
  }, {
    "l": "29",
    "e": -9.88,
    "r": -10.28,
    "f": -9.16
  }, {
    "l": "30",
    "e": -9.81,
    "r": -9.15,
    "f": -8.41
  }, {
    "l": "31",
    "e": -9.54,
    "r": -8.37,
    "f": -8.59
  }, {
    "l": "32",
    "e": -9.75,
    "r": -10.09,
    "f": -9.84
  }]
}, {
  "id": "jnk1",
  "name": "JNK1",
  "set": "JACS",
  "excluded": true,
  "note": "Flipped binding poses are separate ligands in the RevFEP run, so this is not like-for-like; excluded from the averages.",
  "revfep": {
    "n": 38,
    "rmse": 1.15,
    "mae": 0.88,
    "r2": 0.07,
    "rho": 0.16
  },
  "fepplus": {
    "n": 38,
    "rmse": 0.57,
    "mae": 0.45,
    "r2": 0.72,
    "rho": 0.89
  },
  "points": [{
    "l": "17124-1",
    "e": -9.68,
    "r": -11.23,
    "f": -10.62
  }, {
    "l": "17124-1_flip",
    "e": -9.68,
    "r": -7.89,
    "f": -10.62
  }, {
    "l": "18624-1",
    "e": -8.49,
    "r": -8.43,
    "f": -7.62
  }, {
    "l": "18625-1",
    "e": -8.11,
    "r": -9.05,
    "f": -7.46
  }, {
    "l": "18625-1_flip",
    "e": -8.11,
    "r": -8.24,
    "f": -7.46
  }, {
    "l": "18626-1",
    "e": -8.87,
    "r": -9.42,
    "f": -8.79
  }, {
    "l": "18626-1_flip",
    "e": -8.87,
    "r": -9.0,
    "f": -8.79
  }, {
    "l": "18627-1",
    "e": -8.48,
    "r": -8.99,
    "f": -8.63
  }, {
    "l": "18628-1",
    "e": -8.7,
    "r": -8.08,
    "f": -7.27
  }, {
    "l": "18628-1_flip",
    "e": -8.7,
    "r": -8.59,
    "f": -7.27
  }, {
    "l": "18629-1",
    "e": -8.67,
    "r": -8.94,
    "f": -8.47
  }, {
    "l": "18629-1_flip",
    "e": -8.67,
    "r": -9.2,
    "f": -8.47
  }, {
    "l": "18630-1",
    "e": -9.14,
    "r": -8.92,
    "f": -8.37
  }, {
    "l": "18631-1",
    "e": -9.41,
    "r": -9.76,
    "f": -8.73
  }, {
    "l": "18631-1_flip",
    "e": -9.41,
    "r": -8.19,
    "f": -8.73
  }, {
    "l": "18632-1",
    "e": -9.08,
    "r": -9.26,
    "f": -8.87
  }, {
    "l": "18632-1_flip",
    "e": -9.08,
    "r": -9.37,
    "f": -8.87
  }, {
    "l": "18633-1",
    "e": -9.17,
    "r": -9.0,
    "f": -8.83
  }, {
    "l": "18634-1",
    "e": -9.99,
    "r": -10.55,
    "f": -10.07
  }, {
    "l": "18634-1_flip",
    "e": -9.99,
    "r": -8.57,
    "f": -10.07
  }, {
    "l": "18635-1",
    "e": -7.29,
    "r": -8.67,
    "f": -7.85
  }, {
    "l": "18635-1_flip",
    "e": -7.29,
    "r": -8.5,
    "f": -7.85
  }, {
    "l": "18636-1",
    "e": -7.51,
    "r": -9.47,
    "f": -8.31
  }, {
    "l": "18636-1_flip",
    "e": -7.51,
    "r": -7.82,
    "f": -8.31
  }, {
    "l": "18637-1",
    "e": -10.14,
    "r": -11.01,
    "f": -10.53
  }, {
    "l": "18637-1_flip",
    "e": -10.14,
    "r": -7.42,
    "f": -10.53
  }, {
    "l": "18638-1",
    "e": -10.09,
    "r": -10.52,
    "f": -10.23
  }, {
    "l": "18638-1_flip",
    "e": -10.09,
    "r": -8.39,
    "f": -10.23
  }, {
    "l": "18639-1",
    "e": -9.74,
    "r": -11.25,
    "f": -10.01
  }, {
    "l": "18639-1_flip",
    "e": -9.74,
    "r": -8.09,
    "f": -10.01
  }, {
    "l": "18652-1",
    "e": -10.68,
    "r": -10.56,
    "f": -10.98
  }, {
    "l": "18652-1_flip",
    "e": -10.68,
    "r": -8.36,
    "f": -10.98
  }, {
    "l": "18658-1",
    "e": -9.7,
    "r": -10.6,
    "f": -9.68
  }, {
    "l": "18658-1_flip",
    "e": -9.7,
    "r": -8.53,
    "f": -9.68
  }, {
    "l": "18659-1",
    "e": -9.47,
    "r": -9.53,
    "f": -9.65
  }, {
    "l": "18659-1_flip",
    "e": -9.47,
    "r": -7.99,
    "f": -9.65
  }, {
    "l": "18660-1",
    "e": -8.7,
    "r": -10.86,
    "f": -9.21
  }, {
    "l": "18660-1_flip",
    "e": -8.7,
    "r": -8.72,
    "f": -9.21
  }]
}, {
  "id": "mcl1",
  "name": "MCL1",
  "set": "JACS",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 41,
    "rmse": 1.46,
    "mae": 1.11,
    "r2": 0.35,
    "rho": 0.6
  },
  "fepplus": {
    "n": 41,
    "rmse": 0.86,
    "mae": 0.68,
    "r2": 0.5,
    "rho": 0.67
  },
  "points": [{
    "l": "23",
    "e": -8.83,
    "r": -7.76,
    "f": -7.55
  }, {
    "l": "26",
    "e": -8.24,
    "r": -8.5,
    "f": -8.27
  }, {
    "l": "27",
    "e": -6.12,
    "r": -6.18,
    "f": -6.47
  }, {
    "l": "28",
    "e": -6.62,
    "r": -5.63,
    "f": -6.08
  }, {
    "l": "29",
    "e": -6.94,
    "r": -5.39,
    "f": -6.5
  }, {
    "l": "30_flip",
    "e": -7.85,
    "r": -6.34,
    "f": -7.0
  }, {
    "l": "31",
    "e": -7.92,
    "r": -8.62,
    "f": -7.96
  }, {
    "l": "32",
    "e": -6.58,
    "r": -6.64,
    "f": -6.61
  }, {
    "l": "33",
    "e": -6.88,
    "r": -7.69,
    "f": -7.75
  }, {
    "l": "34",
    "e": -6.87,
    "r": -8.26,
    "f": -7.35
  }, {
    "l": "35",
    "e": -8.81,
    "r": -7.84,
    "f": -8.03
  }, {
    "l": "36",
    "e": -8.18,
    "r": -8.97,
    "f": -8.19
  }, {
    "l": "37",
    "e": -8.95,
    "r": -8.92,
    "f": -8.75
  }, {
    "l": "38_flip",
    "e": -7.02,
    "r": -11.12,
    "f": -9.08
  }, {
    "l": "40_flip",
    "e": -7.25,
    "r": -8.39,
    "f": -8.23
  }, {
    "l": "41",
    "e": -7.13,
    "r": -8.39,
    "f": -8.2
  }, {
    "l": "42",
    "e": -8.9,
    "r": -7.19,
    "f": -7.65
  }, {
    "l": "43_flip",
    "e": -7.03,
    "r": -6.25,
    "f": -7.76
  }, {
    "l": "44",
    "e": -8.67,
    "r": -8.12,
    "f": -8.44
  }, {
    "l": "45",
    "e": -8.95,
    "r": -6.76,
    "f": -7.58
  }, {
    "l": "46_flip",
    "e": -7.6,
    "r": -6.81,
    "f": -8.01
  }, {
    "l": "47_flip",
    "e": -5.78,
    "r": -5.27,
    "f": -6.31
  }, {
    "l": "48_flip",
    "e": -6.66,
    "r": -2.66,
    "f": -4.65
  }, {
    "l": "49",
    "e": -8.36,
    "r": -8.05,
    "f": -8.75
  }, {
    "l": "50",
    "e": -9.33,
    "r": -8.54,
    "f": -8.77
  }, {
    "l": "51",
    "e": -8.45,
    "r": -6.44,
    "f": -7.17
  }, {
    "l": "52",
    "e": -9.23,
    "r": -9.53,
    "f": -9.74
  }, {
    "l": "53",
    "e": -9.96,
    "r": -11.07,
    "f": -10.11
  }, {
    "l": "54",
    "e": -9.78,
    "r": -9.72,
    "f": -9.07
  }, {
    "l": "56",
    "e": -9.26,
    "r": -9.37,
    "f": -9.07
  }, {
    "l": "57",
    "e": -9.04,
    "r": -7.4,
    "f": -7.86
  }, {
    "l": "58",
    "e": -9.41,
    "r": -11.24,
    "f": -10.39
  }, {
    "l": "60",
    "e": -8.92,
    "r": -9.48,
    "f": -8.8
  }, {
    "l": "61",
    "e": -8.08,
    "r": -8.7,
    "f": -8.34
  }, {
    "l": "62",
    "e": -7.96,
    "r": -6.66,
    "f": -8.09
  }, {
    "l": "63",
    "e": -9.06,
    "r": -11.23,
    "f": -9.85
  }, {
    "l": "64",
    "e": -9.5,
    "r": -9.59,
    "f": -8.99
  }, {
    "l": "65",
    "e": -8.41,
    "r": -10.67,
    "f": -9.65
  }, {
    "l": "66",
    "e": -8.43,
    "r": -9.03,
    "f": -8.69
  }, {
    "l": "67",
    "e": -7.58,
    "r": -10.1,
    "f": -9.24
  }, {
    "l": "68",
    "e": -7.69,
    "r": -7.7,
    "f": -7.18
  }]
}, {
  "id": "p38",
  "name": "p38",
  "set": "JACS",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 34,
    "rmse": 1.65,
    "mae": 1.13,
    "r2": 0.06,
    "rho": 0.27
  },
  "fepplus": {
    "n": 34,
    "rmse": 0.81,
    "mae": 0.63,
    "r2": 0.49,
    "rho": 0.72
  },
  "points": [{
    "l": "2aa",
    "e": -9.27,
    "r": -9.82,
    "f": -11.2
  }, {
    "l": "2bb",
    "e": -9.06,
    "r": -11.85,
    "f": -9.34
  }, {
    "l": "2c",
    "e": -10.46,
    "r": -11.06,
    "f": -11.54
  }, {
    "l": "2e",
    "e": -10.7,
    "r": -9.12,
    "f": -9.93
  }, {
    "l": "2ee",
    "e": -12.26,
    "r": -8.57,
    "f": -10.66
  }, {
    "l": "2f",
    "e": -8.48,
    "r": -9.91,
    "f": -9.38
  }, {
    "l": "2ff",
    "e": -11.44,
    "r": -9.97,
    "f": -10.35
  }, {
    "l": "2g",
    "e": -10.66,
    "r": -10.33,
    "f": -11.03
  }, {
    "l": "2gg",
    "e": -10.66,
    "r": -8.95,
    "f": -9.34
  }, {
    "l": "2h",
    "e": -9.48,
    "r": -9.34,
    "f": -8.96
  }, {
    "l": "2i",
    "e": -10.05,
    "r": -9.86,
    "f": -9.67
  }, {
    "l": "2j",
    "e": -10.08,
    "r": -9.69,
    "f": -9.29
  }, {
    "l": "2k",
    "e": -10.55,
    "r": -8.99,
    "f": -10.22
  }, {
    "l": "2l",
    "e": -12.26,
    "r": -11.04,
    "f": -12.09
  }, {
    "l": "2m",
    "e": -10.96,
    "r": -11.44,
    "f": -12.13
  }, {
    "l": "2n",
    "e": -11.85,
    "r": -11.31,
    "f": -11.89
  }, {
    "l": "2o",
    "e": -10.49,
    "r": -10.33,
    "f": -10.33
  }, {
    "l": "2p",
    "e": -11.2,
    "r": -11.36,
    "f": -10.69
  }, {
    "l": "2q",
    "e": -12.26,
    "r": -13.14,
    "f": -11.78
  }, {
    "l": "2r",
    "e": -10.79,
    "r": -8.42,
    "f": -10.76
  }, {
    "l": "2s",
    "e": -11.11,
    "r": -9.67,
    "f": -11.69
  }, {
    "l": "2t",
    "e": -11.85,
    "r": -11.84,
    "f": -10.91
  }, {
    "l": "2u",
    "e": -12.26,
    "r": -13.79,
    "f": -12.05
  }, {
    "l": "2v",
    "e": -8.97,
    "r": -8.76,
    "f": -9.02
  }, {
    "l": "2x",
    "e": -10.23,
    "r": -10.2,
    "f": -9.98
  }, {
    "l": "2y",
    "e": -9.78,
    "r": -11.05,
    "f": -9.56
  }, {
    "l": "2z",
    "e": -10.36,
    "r": -10.53,
    "f": -10.21
  }, {
    "l": "3fln",
    "e": -10.88,
    "r": -10.02,
    "f": -11.12
  }, {
    "l": "3flq",
    "e": -9.93,
    "r": -16.16,
    "f": -9.64
  }, {
    "l": "3flw",
    "e": -10.68,
    "r": -11.96,
    "f": -11.91
  }, {
    "l": "3fly",
    "e": -11.42,
    "r": -10.13,
    "f": -10.96
  }, {
    "l": "3flz",
    "e": -9.49,
    "r": -10.18,
    "f": -10.1
  }, {
    "l": "3fmh",
    "e": -10.83,
    "r": -11.45,
    "f": -11.21
  }, {
    "l": "3fmk",
    "e": -11.83,
    "r": -12.34,
    "f": -13.62
  }]
}, {
  "id": "ptp1b",
  "name": "PTP1B",
  "set": "JACS",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 23,
    "rmse": 1.04,
    "mae": 0.79,
    "r2": 0.5,
    "rho": 0.71
  },
  "fepplus": {
    "n": 23,
    "rmse": 0.51,
    "mae": 0.41,
    "r2": 0.85,
    "rho": 0.88
  },
  "points": [{
    "l": "20667",
    "e": -11.42,
    "r": -10.98,
    "f": -10.54
  }, {
    "l": "20669",
    "e": -8.61,
    "r": -7.62,
    "f": -8.64
  }, {
    "l": "20670",
    "e": -9.08,
    "r": -9.7,
    "f": -9.19
  }, {
    "l": "23330",
    "e": -10.13,
    "r": -11.37,
    "f": -10.66
  }, {
    "l": "23466",
    "e": -7.85,
    "r": -6.29,
    "f": -7.19
  }, {
    "l": "23467",
    "e": -7.34,
    "r": -7.06,
    "f": -7.24
  }, {
    "l": "23468",
    "e": -7.75,
    "r": -7.45,
    "f": -7.45
  }, {
    "l": "23469",
    "e": -7.72,
    "r": -6.43,
    "f": -8.03
  }, {
    "l": "23470",
    "e": -7.72,
    "r": -8.74,
    "f": -8.13
  }, {
    "l": "23471",
    "e": -7.75,
    "r": -8.07,
    "f": -8.02
  }, {
    "l": "23472",
    "e": -8.65,
    "r": -8.29,
    "f": -9.3
  }, {
    "l": "23473",
    "e": -8.39,
    "r": -8.47,
    "f": -9.3
  }, {
    "l": "23474",
    "e": -9.11,
    "r": -8.88,
    "f": -8.87
  }, {
    "l": "23475",
    "e": -8.72,
    "r": -9.93,
    "f": -8.92
  }, {
    "l": "23476",
    "e": -9.41,
    "r": -9.43,
    "f": -10.02
  }, {
    "l": "23477",
    "e": -8.85,
    "r": -9.38,
    "f": -8.77
  }, {
    "l": "23479",
    "e": -9.14,
    "r": -8.47,
    "f": -8.68
  }, {
    "l": "23480",
    "e": -8.72,
    "r": -9.75,
    "f": -9.16
  }, {
    "l": "23482",
    "e": -10.01,
    "r": -9.82,
    "f": -9.57
  }, {
    "l": "23483",
    "e": -9.87,
    "r": -9.98,
    "f": -9.86
  }, {
    "l": "23484",
    "e": -7.75,
    "r": -10.69,
    "f": -8.02
  }, {
    "l": "23485",
    "e": -11.01,
    "r": -9.18,
    "f": -10.67
  }, {
    "l": "23486",
    "e": -12.47,
    "r": -11.47,
    "f": -11.25
  }]
}, {
  "id": "thrombin",
  "name": "Thrombin",
  "set": "JACS",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 11,
    "rmse": 0.5,
    "mae": 0.46,
    "r2": 0.64,
    "rho": 0.73
  },
  "fepplus": {
    "n": 11,
    "rmse": 0.53,
    "mae": 0.42,
    "r2": 0.68,
    "rho": 0.78
  },
  "points": [{
    "l": "1a",
    "e": -7.48,
    "r": -7.43,
    "f": -7.32
  }, {
    "l": "1b",
    "e": -8.46,
    "r": -8.87,
    "f": -8.19
  }, {
    "l": "1c",
    "e": -8.56,
    "r": -9.17,
    "f": -9.04
  }, {
    "l": "1d",
    "e": -8.25,
    "r": -8.93,
    "f": -9.42
  }, {
    "l": "3a",
    "e": -8.32,
    "r": -8.03,
    "f": -7.79
  }, {
    "l": "3b",
    "e": -7.86,
    "r": -7.24,
    "f": -7.86
  }, {
    "l": "5",
    "e": -7.58,
    "r": -6.82,
    "f": -6.7
  }, {
    "l": "6a",
    "e": -9.18,
    "r": -8.98,
    "f": -9.66
  }, {
    "l": "6b",
    "e": -8.89,
    "r": -9.36,
    "f": -9.07
  }, {
    "l": "6e",
    "e": -8.91,
    "r": -8.32,
    "f": -8.61
  }, {
    "l": "7a",
    "e": -8.22,
    "r": -8.56,
    "f": -8.05
  }]
}, {
  "id": "cdk8",
  "name": "CDK8",
  "set": "Merck",
  "excluded": false,
  "note": "1 ligand has no FEP+ prediction.",
  "revfep": {
    "n": 32,
    "rmse": 1.53,
    "mae": 1.31,
    "r2": 0.47,
    "rho": 0.74
  },
  "fepplus": {
    "n": 32,
    "rmse": 1.13,
    "mae": 0.93,
    "r2": 0.56,
    "rho": 0.76
  },
  "points": [{
    "l": "13",
    "e": -11.28,
    "r": -12.49,
    "f": -9.95
  }, {
    "l": "14",
    "e": -8.34,
    "r": -8.09,
    "f": -7.66
  }, {
    "l": "15",
    "e": -7.69,
    "r": -7.98,
    "f": -6.43
  }, {
    "l": "16_flipped",
    "e": -9.29,
    "r": -7.58,
    "f": -8.48
  }, {
    "l": "17_flipped",
    "e": -8.99,
    "r": -6.27,
    "f": -8.95
  }, {
    "l": "18_flipped",
    "e": -10.98,
    "r": -8.27,
    "f": -9.08
  }, {
    "l": "19",
    "e": -10.74,
    "r": -10.66,
    "f": -9.63
  }, {
    "l": "20",
    "e": -10.7,
    "r": -12.3,
    "f": -9.69
  }, {
    "l": "21",
    "e": -9.75,
    "r": -6.87,
    "f": -9.63
  }, {
    "l": "22",
    "e": -10.29,
    "r": -9.85,
    "f": -10.26
  }, {
    "l": "23",
    "e": -9.01,
    "r": -9.46,
    "f": -7.42
  }, {
    "l": "24",
    "e": -10.34,
    "r": -8.73,
    "f": -8.64
  }, {
    "l": "25",
    "e": -9.79,
    "r": -7.85,
    "f": -10.66
  }, {
    "l": "26",
    "e": -10.22,
    "r": -8.8,
    "f": -8.72
  }, {
    "l": "27",
    "e": -8.46,
    "r": -11.19,
    "f": -11.03
  }, {
    "l": "28",
    "e": -6.21,
    "r": -8.38,
    "f": null
  }, {
    "l": "29",
    "e": -9.26,
    "r": -9.9,
    "f": -11.3
  }, {
    "l": "30",
    "e": -11.7,
    "r": -13.56,
    "f": -11.33
  }, {
    "l": "31",
    "e": -11.94,
    "r": -12.54,
    "f": -12.32
  }, {
    "l": "32",
    "e": -9.55,
    "r": -10.26,
    "f": -9.95
  }, {
    "l": "33",
    "e": -11.11,
    "r": -10.37,
    "f": -10.01
  }, {
    "l": "34",
    "e": -11.19,
    "r": -10.9,
    "f": -12.03
  }, {
    "l": "35",
    "e": -11.52,
    "r": -9.86,
    "f": -11.95
  }, {
    "l": "36",
    "e": -11.28,
    "r": -11.08,
    "f": -12.1
  }, {
    "l": "37",
    "e": -11.52,
    "r": -12.45,
    "f": -12.88
  }, {
    "l": "38",
    "e": -11.39,
    "r": -12.33,
    "f": -12.74
  }, {
    "l": "39",
    "e": -11.52,
    "r": -12.92,
    "f": -12.24
  }, {
    "l": "40",
    "e": -11.19,
    "r": -12.84,
    "f": -12.43
  }, {
    "l": "41",
    "e": -11.28,
    "r": -12.53,
    "f": -12.77
  }, {
    "l": "42_flipped",
    "e": -11.52,
    "r": -13.52,
    "f": -11.55
  }, {
    "l": "43",
    "e": -11.94,
    "r": -13.6,
    "f": -11.72
  }, {
    "l": "44_flipped",
    "e": -11.52,
    "r": -9.94,
    "f": -11.39
  }, {
    "l": "45_flipped",
    "e": -11.19,
    "r": -9.36,
    "f": -11.67
  }]
}, {
  "id": "cmet-q0",
  "name": "c-Met (neutral)",
  "set": "Merck",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 11,
    "rmse": 1.33,
    "mae": 1.01,
    "r2": 0.46,
    "rho": 0.63
  },
  "fepplus": {
    "n": 11,
    "rmse": 0.86,
    "mae": 0.76,
    "r2": 0.57,
    "rho": 0.79
  },
  "points": [{
    "l": "CHEMBL3402744_300",
    "e": -8.9,
    "r": -7.84,
    "f": -7.77
  }, {
    "l": "CHEMBL3402745_200",
    "e": -9.14,
    "r": -9.02,
    "f": -8.38
  }, {
    "l": "CHEMBL3402747_3400",
    "e": -7.46,
    "r": -5.62,
    "f": -6.08
  }, {
    "l": "CHEMBL3402748_5300",
    "e": -7.2,
    "r": -7.72,
    "f": -7.74
  }, {
    "l": "CHEMBL3402749_500",
    "e": -8.6,
    "r": -8.63,
    "f": -7.75
  }, {
    "l": "CHEMBL3402750_400",
    "e": -8.73,
    "r": -9.71,
    "f": -9.9
  }, {
    "l": "CHEMBL3402751_2100",
    "e": -7.75,
    "r": -5.25,
    "f": -7.69
  }, {
    "l": "CHEMBL3402752_30000",
    "e": -6.17,
    "r": -6.11,
    "f": -6.56
  }, {
    "l": "CHEMBL3402753_200",
    "e": -9.14,
    "r": -10.27,
    "f": -9.99
  }, {
    "l": "CHEMBL3402754_40",
    "e": -10.09,
    "r": -10.56,
    "f": -10.32
  }, {
    "l": "CHEMBL3402755_4200",
    "e": -7.34,
    "r": -9.78,
    "f": -8.36
  }]
}, {
  "id": "cmet-q1o",
  "name": "c-Met (+1)",
  "set": "Merck",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 6,
    "rmse": 1.87,
    "mae": 1.69,
    "r2": 0.18,
    "rho": -0.38
  },
  "fepplus": {
    "n": 6,
    "rmse": 0.63,
    "mae": 0.56,
    "r2": 0.24,
    "rho": 0.55
  },
  "points": [{
    "l": "CHEMBL3402741_400",
    "e": -8.73,
    "r": -11.9,
    "f": -9.52
  }, {
    "l": "CHEMBL3402742_23",
    "e": -10.42,
    "r": -10.88,
    "f": -10.56
  }, {
    "l": "CHEMBL3402743_42",
    "e": -10.06,
    "r": -8.55,
    "f": -9.82
  }, {
    "l": "CHEMBL3402763_90",
    "e": -9.61,
    "r": -11.05,
    "f": -10.37
  }, {
    "l": "CHEMBL3402764_90",
    "e": -9.61,
    "r": -7.81,
    "f": -8.97
  }, {
    "l": "CHEMBL3402765_11-charged-pKa-8.1",
    "e": -10.86,
    "r": -9.1,
    "f": -10.05
  }]
}, {
  "id": "cmet-q1r",
  "name": "c-Met (+1, redocked)",
  "set": "Merck",
  "excluded": true,
  "note": "Same six ligands as c-Met (+1), started from redocked poses; excluded from the averages.",
  "revfep": {
    "n": 6,
    "rmse": 0.7,
    "mae": 0.57,
    "r2": 0.52,
    "rho": 0.58
  },
  "fepplus": {
    "n": 6,
    "rmse": 0.46,
    "mae": 0.38,
    "r2": 0.56,
    "rho": 0.81
  },
  "points": [{
    "l": "CHEMBL3402756_2.7_redocked",
    "e": -11.69,
    "r": -10.68,
    "f": -11.56
  }, {
    "l": "CHEMBL3402757_6.5_redocked",
    "e": -11.17,
    "r": -10.72,
    "f": -10.98
  }, {
    "l": "CHEMBL3402758_10_redocked",
    "e": -10.91,
    "r": -11.01,
    "f": -11.3
  }, {
    "l": "CHEMBL3402759_5.7_redocked",
    "e": -11.25,
    "r": -11.65,
    "f": -11.07
  }, {
    "l": "CHEMBL3402760_1_redocked",
    "e": -12.28,
    "r": -12.03,
    "f": -11.61
  }, {
    "l": "CHEMBL3402762_1_redocked",
    "e": -12.28,
    "r": -13.49,
    "f": -13.04
  }]
}, {
  "id": "eg5-q0",
  "name": "Eg5 (neutral)",
  "set": "Merck",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 9,
    "rmse": 1.32,
    "mae": 1.09,
    "r2": 0.59,
    "rho": 0.54
  },
  "fepplus": {
    "n": 9,
    "rmse": 0.64,
    "mae": 0.55,
    "r2": 0.54,
    "rho": 0.74
  },
  "points": [{
    "l": "CHEMBL1078691",
    "e": -10.25,
    "r": -11.25,
    "f": -10.69
  }, {
    "l": "CHEMBL1078774",
    "e": -10.66,
    "r": -10.98,
    "f": -9.72
  }, {
    "l": "CHEMBL1078998",
    "e": -10.43,
    "r": -10.04,
    "f": -9.97
  }, {
    "l": "CHEMBL1084115",
    "e": -9.5,
    "r": -10.17,
    "f": -9.93
  }, {
    "l": "CHEMBL1085895",
    "e": -10.92,
    "r": -13.53,
    "f": -11.01
  }, {
    "l": "CHEMBL1088740",
    "e": -8.75,
    "r": -6.84,
    "f": -9.16
  }, {
    "l": "CHEMBL1089056",
    "e": -9.5,
    "r": -9.78,
    "f": -9.72
  }, {
    "l": "CHEMBL1089393",
    "e": -8.43,
    "r": -7.35,
    "f": -9.32
  }, {
    "l": "CHEMBL1093088",
    "e": -11.28,
    "r": -9.77,
    "f": -10.2
  }]
}, {
  "id": "eg5-q1",
  "name": "Eg5 (+1)",
  "set": "Merck",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 19,
    "rmse": 2.05,
    "mae": 1.64,
    "r2": 0.07,
    "rho": -0.21
  },
  "fepplus": {
    "n": 19,
    "rmse": 0.94,
    "mae": 0.68,
    "r2": 0.23,
    "rho": 0.46
  },
  "points": [{
    "l": "CHEMBL1077204",
    "e": -11.11,
    "r": -8.34,
    "f": -11.94
  }, {
    "l": "CHEMBL1077227",
    "e": -11.39,
    "r": -10.85,
    "f": -10.69
  }, {
    "l": "CHEMBL1082248",
    "e": -11.7,
    "r": -11.77,
    "f": -11.03
  }, {
    "l": "CHEMBL1082249",
    "e": -11.28,
    "r": -10.94,
    "f": -11.32
  }, {
    "l": "CHEMBL1083517",
    "e": -11.52,
    "r": -14.02,
    "f": -11.78
  }, {
    "l": "CHEMBL1083836",
    "e": -10.92,
    "r": -9.52,
    "f": -10.15
  }, {
    "l": "CHEMBL1084143",
    "e": -11.94,
    "r": -8.73,
    "f": -11.88
  }, {
    "l": "CHEMBL1084431",
    "e": -9.86,
    "r": -13.91,
    "f": -10.19
  }, {
    "l": "CHEMBL1084676",
    "e": -10.78,
    "r": -7.34,
    "f": -10.88
  }, {
    "l": "CHEMBL1084677",
    "e": -11.52,
    "r": -11.34,
    "f": -11.98
  }, {
    "l": "CHEMBL1084678",
    "e": -11.19,
    "r": -11.56,
    "f": -13.63
  }, {
    "l": "CHEMBL1084935",
    "e": -10.6,
    "r": -9.87,
    "f": -8.45
  }, {
    "l": "CHEMBL1085666",
    "e": -10.57,
    "r": -13.77,
    "f": -10.76
  }, {
    "l": "CHEMBL1085692",
    "e": -11.39,
    "r": -12.15,
    "f": -12.35
  }, {
    "l": "CHEMBL1086409",
    "e": -9.93,
    "r": -11.5,
    "f": -10.29
  }, {
    "l": "CHEMBL1086410",
    "e": -11.52,
    "r": -10.99,
    "f": -10.12
  }, {
    "l": "CHEMBL1093087",
    "e": -10.6,
    "r": -12.36,
    "f": -11.15
  }, {
    "l": "CHEMBL1096002",
    "e": -11.19,
    "r": -12.53,
    "f": -10.68
  }, {
    "l": "CHEMBL1096003",
    "e": -11.94,
    "r": -9.48,
    "f": -11.75
  }]
}, {
  "id": "hif2a",
  "name": "HIF-2α",
  "set": "Merck",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 41,
    "rmse": 1.66,
    "mae": 1.33,
    "r2": 0.38,
    "rho": 0.64
  },
  "fepplus": {
    "n": 41,
    "rmse": 0.77,
    "mae": 0.66,
    "r2": 0.61,
    "rho": 0.65
  },
  "points": [{
    "l": "1",
    "e": -11.32,
    "r": -11.03,
    "f": -10.15
  }, {
    "l": "124",
    "e": -10.12,
    "r": -10.14,
    "f": -10.16
  }, {
    "l": "15",
    "e": -11.05,
    "r": -12.43,
    "f": -11.41
  }, {
    "l": "155",
    "e": -10.5,
    "r": -11.26,
    "f": -9.69
  }, {
    "l": "156",
    "e": -9.74,
    "r": -10.61,
    "f": -8.83
  }, {
    "l": "163",
    "e": -11.01,
    "r": -10.14,
    "f": -10.47
  }, {
    "l": "164",
    "e": -6.76,
    "r": -4.46,
    "f": -6.6
  }, {
    "l": "165",
    "e": -9.24,
    "r": -8.31,
    "f": -8.54
  }, {
    "l": "167",
    "e": -10.67,
    "r": -10.86,
    "f": -10.38
  }, {
    "l": "206",
    "e": -10.12,
    "r": -8.47,
    "f": -9.05
  }, {
    "l": "215",
    "e": -10.6,
    "r": -11.49,
    "f": -10.32
  }, {
    "l": "224",
    "e": -9.66,
    "r": -10.17,
    "f": -10.28
  }, {
    "l": "227",
    "e": -10.76,
    "r": -14.16,
    "f": -11.16
  }, {
    "l": "23",
    "e": -7.07,
    "r": -8.45,
    "f": -6.9
  }, {
    "l": "231",
    "e": -10.02,
    "r": -10.82,
    "f": -9.6
  }, {
    "l": "234",
    "e": -10.35,
    "r": -9.65,
    "f": -9.85
  }, {
    "l": "235",
    "e": -10.12,
    "r": -13.1,
    "f": -9.94
  }, {
    "l": "237",
    "e": -9.55,
    "r": -6.99,
    "f": -10.52
  }, {
    "l": "25",
    "e": -10.28,
    "r": -10.72,
    "f": -9.68
  }, {
    "l": "251",
    "e": -10.24,
    "r": -7.38,
    "f": -10.54
  }, {
    "l": "252",
    "e": -9.71,
    "r": -8.88,
    "f": -10.48
  }, {
    "l": "254",
    "e": -9.55,
    "r": -8.95,
    "f": -10.21
  }, {
    "l": "256",
    "e": -10.67,
    "r": -7.55,
    "f": -9.72
  }, {
    "l": "266",
    "e": -10.53,
    "r": -13.65,
    "f": -11.04
  }, {
    "l": "289",
    "e": -10.91,
    "r": -10.6,
    "f": -11.14
  }, {
    "l": "290",
    "e": -10.57,
    "r": -12.12,
    "f": -11.38
  }, {
    "l": "30",
    "e": -10.57,
    "r": -11.12,
    "f": -11.04
  }, {
    "l": "31",
    "e": -8.84,
    "r": -9.43,
    "f": -9.22
  }, {
    "l": "336",
    "e": -9.82,
    "r": -9.47,
    "f": -10.39
  }, {
    "l": "338",
    "e": -10.76,
    "r": -9.58,
    "f": -11.21
  }, {
    "l": "41",
    "e": -9.79,
    "r": -10.19,
    "f": -10.35
  }, {
    "l": "42",
    "e": -8.94,
    "r": -10.52,
    "f": -9.96
  }, {
    "l": "43",
    "e": -8.7,
    "r": -9.91,
    "f": -9.21
  }, {
    "l": "50",
    "e": -9.7,
    "r": -6.27,
    "f": -8.63
  }, {
    "l": "54",
    "e": -9.31,
    "r": -7.26,
    "f": -9.62
  }, {
    "l": "57",
    "e": -10.67,
    "r": -13.44,
    "f": -10.32
  }, {
    "l": "61",
    "e": -10.11,
    "r": -9.64,
    "f": -9.28
  }, {
    "l": "67",
    "e": -10.17,
    "r": -10.67,
    "f": -11.85
  }, {
    "l": "7a",
    "e": -10.0,
    "r": -9.09,
    "f": -8.57
  }, {
    "l": "7b",
    "e": -7.39,
    "r": -5.5,
    "f": -6.34
  }, {
    "l": "84",
    "e": -7.58,
    "r": -9.03,
    "f": -9.37
  }]
}, {
  "id": "pfkfb3",
  "name": "PFKFB3",
  "set": "Merck",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 40,
    "rmse": 1.1,
    "mae": 0.91,
    "r2": 0.35,
    "rho": 0.56
  },
  "fepplus": {
    "n": 40,
    "rmse": 0.99,
    "mae": 0.77,
    "r2": 0.58,
    "rho": 0.76
  },
  "points": [{
    "l": "19",
    "e": -7.55,
    "r": -6.08,
    "f": -7.51
  }, {
    "l": "20",
    "e": -8.52,
    "r": -8.35,
    "f": -8.07
  }, {
    "l": "23",
    "e": -7.91,
    "r": -6.64,
    "f": -8.2
  }, {
    "l": "24",
    "e": -8.87,
    "r": -6.44,
    "f": -9.06
  }, {
    "l": "26",
    "e": -9.17,
    "r": -7.98,
    "f": -9.32
  }, {
    "l": "29",
    "e": -9.65,
    "r": -8.58,
    "f": -9.12
  }, {
    "l": "30_flip_b",
    "e": -9.2,
    "r": -7.3,
    "f": -8.98
  }, {
    "l": "31",
    "e": -9.28,
    "r": -8.42,
    "f": -9.45
  }, {
    "l": "33",
    "e": -8.61,
    "r": -7.5,
    "f": -9.51
  }, {
    "l": "34",
    "e": -9.97,
    "r": -8.93,
    "f": -9.76
  }, {
    "l": "35",
    "e": -10.14,
    "r": -9.85,
    "f": -9.94
  }, {
    "l": "36",
    "e": -7.44,
    "r": -8.57,
    "f": -7.75
  }, {
    "l": "37",
    "e": -10.0,
    "r": -10.52,
    "f": -10.92
  }, {
    "l": "38",
    "e": -8.76,
    "r": -10.19,
    "f": -7.2
  }, {
    "l": "39",
    "e": -7.2,
    "r": -8.49,
    "f": -6.48
  }, {
    "l": "41",
    "e": -7.39,
    "r": -9.21,
    "f": -9.24
  }, {
    "l": "42",
    "e": -8.47,
    "r": -8.45,
    "f": -8.91
  }, {
    "l": "43",
    "e": -8.0,
    "r": -9.93,
    "f": -9.75
  }, {
    "l": "44",
    "e": -9.95,
    "r": -10.09,
    "f": -8.96
  }, {
    "l": "46",
    "e": -7.86,
    "r": -8.02,
    "f": -7.14
  }, {
    "l": "47",
    "e": -8.33,
    "r": -8.09,
    "f": -6.63
  }, {
    "l": "48",
    "e": -8.01,
    "r": -8.86,
    "f": -6.65
  }, {
    "l": "49",
    "e": -7.93,
    "r": -8.29,
    "f": -7.32
  }, {
    "l": "52",
    "e": -10.4,
    "r": -8.64,
    "f": -8.35
  }, {
    "l": "53",
    "e": -7.0,
    "r": -7.71,
    "f": -7.21
  }, {
    "l": "54",
    "e": -8.18,
    "r": -7.42,
    "f": -6.86
  }, {
    "l": "55",
    "e": -10.64,
    "r": -11.52,
    "f": -11.5
  }, {
    "l": "56",
    "e": -10.57,
    "r": -10.61,
    "f": -10.89
  }, {
    "l": "57",
    "e": -10.47,
    "r": -9.92,
    "f": -10.2
  }, {
    "l": "58",
    "e": -9.13,
    "r": -9.7,
    "f": -8.35
  }, {
    "l": "59",
    "e": -8.12,
    "r": -7.54,
    "f": -7.69
  }, {
    "l": "60_adjust",
    "e": -8.14,
    "r": -9.14,
    "f": -7.11
  }, {
    "l": "62_flip",
    "e": -7.69,
    "r": -10.08,
    "f": -7.92
  }, {
    "l": "63",
    "e": -8.56,
    "r": -9.1,
    "f": -9.73
  }, {
    "l": "64",
    "e": -9.42,
    "r": -8.92,
    "f": -9.44
  }, {
    "l": "65",
    "e": -9.49,
    "r": -8.51,
    "f": -10.51
  }, {
    "l": "67",
    "e": -10.24,
    "r": -10.74,
    "f": -11.89
  }, {
    "l": "68",
    "e": -9.78,
    "r": -11.13,
    "f": -12.28
  }, {
    "l": "69",
    "e": -10.71,
    "r": -10.8,
    "f": -11.21
  }, {
    "l": "70",
    "e": -10.3,
    "r": -10.76,
    "f": -10.19
  }]
}, {
  "id": "shp2",
  "name": "SHP2",
  "set": "Merck",
  "excluded": false,
  "note": "1 ligand has no FEP+ prediction.",
  "revfep": {
    "n": 25,
    "rmse": 2.12,
    "mae": 1.82,
    "r2": 0.02,
    "rho": 0.16
  },
  "fepplus": {
    "n": 25,
    "rmse": 1.13,
    "mae": 0.95,
    "r2": 0.48,
    "rho": 0.72
  },
  "points": [{
    "l": "10",
    "e": -7.15,
    "r": -7.43,
    "f": -7.68
  }, {
    "l": "11",
    "e": -6.35,
    "r": -3.64,
    "f": -7.56
  }, {
    "l": "3",
    "e": -5.46,
    "r": -9.34,
    "f": -5.82
  }, {
    "l": "4",
    "e": -5.82,
    "r": -9.87,
    "f": -6.41
  }, {
    "l": "6",
    "e": -8.03,
    "r": -8.85,
    "f": -8.85
  }, {
    "l": "7",
    "e": -7.08,
    "r": -9.9,
    "f": -8.3
  }, {
    "l": "Example_1",
    "e": -8.99,
    "r": -9.59,
    "f": -8.83
  }, {
    "l": "Example_14",
    "e": -9.78,
    "r": -11.38,
    "f": -11.3
  }, {
    "l": "Example_2",
    "e": -8.28,
    "r": -8.99,
    "f": -7.52
  }, {
    "l": "Example_22",
    "e": -9.24,
    "r": -7.74,
    "f": -8.78
  }, {
    "l": "Example_23",
    "e": -8.74,
    "r": -7.88,
    "f": -8.7
  }, {
    "l": "Example_24",
    "e": -9.08,
    "r": -6.5,
    "f": -9.15
  }, {
    "l": "Example_25",
    "e": -8.87,
    "r": -6.34,
    "f": -7.13
  }, {
    "l": "Example_26",
    "e": -8.96,
    "r": -6.83,
    "f": -8.56
  }, {
    "l": "Example_27",
    "e": -8.52,
    "r": -8.9,
    "f": -9.53
  }, {
    "l": "Example_28",
    "e": -8.46,
    "r": -5.9,
    "f": -7.36
  }, {
    "l": "Example_29",
    "e": -7.62,
    "r": -5.68,
    "f": -5.63
  }, {
    "l": "Example_3",
    "e": -8.22,
    "r": -6.27,
    "f": -7.09
  }, {
    "l": "Example_30",
    "e": -9.16,
    "r": -10.74,
    "f": -9.07
  }, {
    "l": "Example_4",
    "e": -8.2,
    "r": -5.84,
    "f": -6.68
  }, {
    "l": "Example_5",
    "e": -7.56,
    "r": -7.8,
    "f": -5.1
  }, {
    "l": "Example_6",
    "e": -9.07,
    "r": -9.75,
    "f": -10.57
  }, {
    "l": "Example_8",
    "e": -9.71,
    "r": -8.65,
    "f": -10.5
  }, {
    "l": "Example_9",
    "e": -8.79,
    "r": -11.84,
    "f": -10.08
  }, {
    "l": "SHP099-1_Example_7",
    "e": -9.75,
    "r": -8.5,
    "f": null
  }, {
    "l": "SHP836-2",
    "e": -6.71,
    "r": -9.44,
    "f": -7.7
  }]
}, {
  "id": "syk-s0",
  "name": "SYK (neutral)",
  "set": "Merck",
  "excluded": false,
  "note": "3 ligands have no FEP+ prediction.",
  "revfep": {
    "n": 35,
    "rmse": 1.04,
    "mae": 0.75,
    "r2": 0.31,
    "rho": 0.52
  },
  "fepplus": {
    "n": 35,
    "rmse": 0.71,
    "mae": 0.55,
    "r2": 0.48,
    "rho": 0.63
  },
  "points": [{
    "l": "CHEMBL3259820",
    "e": -10.57,
    "r": -10.32,
    "f": -10.19
  }, {
    "l": "CHEMBL3264996",
    "e": -10.67,
    "r": -11.71,
    "f": -11.72
  }, {
    "l": "CHEMBL3264999",
    "e": -11.42,
    "r": -11.57,
    "f": null
  }, {
    "l": "CHEMBL3265002",
    "e": -10.61,
    "r": -13.65,
    "f": -12.46
  }, {
    "l": "CHEMBL3265003",
    "e": -10.82,
    "r": -13.65,
    "f": null
  }, {
    "l": "CHEMBL3265004",
    "e": -10.86,
    "r": -13.92,
    "f": null
  }, {
    "l": "CHEMBL3265005",
    "e": -10.33,
    "r": -10.58,
    "f": -10.38
  }, {
    "l": "CHEMBL3265006",
    "e": -9.96,
    "r": -9.77,
    "f": -9.22
  }, {
    "l": "CHEMBL3265008",
    "e": -9.78,
    "r": -10.18,
    "f": -9.98
  }, {
    "l": "CHEMBL3265009",
    "e": -10.3,
    "r": -10.52,
    "f": -10.43
  }, {
    "l": "CHEMBL3265010",
    "e": -10.14,
    "r": -9.81,
    "f": -9.75
  }, {
    "l": "CHEMBL3265011",
    "e": -8.64,
    "r": -6.03,
    "f": -8.55
  }, {
    "l": "CHEMBL3265012",
    "e": -10.55,
    "r": -10.29,
    "f": -9.86
  }, {
    "l": "CHEMBL3265013",
    "e": -9.81,
    "r": -10.19,
    "f": -10.77
  }, {
    "l": "CHEMBL3265014",
    "e": -10.4,
    "r": -9.38,
    "f": -10.62
  }, {
    "l": "CHEMBL3265015",
    "e": -9.48,
    "r": -8.03,
    "f": -8.06
  }, {
    "l": "CHEMBL3265016",
    "e": -11.69,
    "r": -10.15,
    "f": -11.29
  }, {
    "l": "CHEMBL3265017",
    "e": -11.49,
    "r": -11.22,
    "f": -11.6
  }, {
    "l": "CHEMBL3265018",
    "e": -11.19,
    "r": -12.0,
    "f": -11.24
  }, {
    "l": "CHEMBL3265019",
    "e": -10.78,
    "r": -12.77,
    "f": -11.86
  }, {
    "l": "CHEMBL3265020",
    "e": -11.01,
    "r": -10.42,
    "f": -10.26
  }, {
    "l": "CHEMBL3265021",
    "e": -10.89,
    "r": -11.23,
    "f": -10.45
  }, {
    "l": "CHEMBL3265022",
    "e": -10.79,
    "r": -10.84,
    "f": -10.01
  }, {
    "l": "CHEMBL3265023",
    "e": -10.48,
    "r": -11.49,
    "f": -9.91
  }, {
    "l": "CHEMBL3265024",
    "e": -11.79,
    "r": -10.63,
    "f": -11.26
  }, {
    "l": "CHEMBL3265025",
    "e": -10.2,
    "r": -10.82,
    "f": -10.81
  }, {
    "l": "CHEMBL3265026",
    "e": -11.36,
    "r": -9.59,
    "f": -10.4
  }, {
    "l": "CHEMBL3265027",
    "e": -10.33,
    "r": -10.03,
    "f": -10.58
  }, {
    "l": "CHEMBL3265028",
    "e": -11.01,
    "r": -10.49,
    "f": -10.89
  }, {
    "l": "CHEMBL3265029",
    "e": -11.12,
    "r": -10.58,
    "f": -11.25
  }, {
    "l": "CHEMBL3265030_n",
    "e": -11.47,
    "r": -10.86,
    "f": -11.52
  }, {
    "l": "CHEMBL3265031",
    "e": -12.07,
    "r": -11.33,
    "f": -11.71
  }, {
    "l": "CHEMBL3265032",
    "e": -11.06,
    "r": -10.95,
    "f": -11.39
  }, {
    "l": "CHEMBL3265033",
    "e": -11.84,
    "r": -11.63,
    "f": -11.75
  }, {
    "l": "CHEMBL3265034",
    "e": -11.49,
    "r": -10.84,
    "f": -11.78
  }, {
    "l": "CHEMBL3265035",
    "e": -11.08,
    "r": -11.04,
    "f": -11.8
  }, {
    "l": "CHEMBL3265036",
    "e": -10.42,
    "r": -10.88,
    "f": -11.99
  }, {
    "l": "CHEMBL3265037",
    "e": -12.77,
    "r": -11.28,
    "f": -11.8
  }]
}, {
  "id": "syk-sm",
  "name": "SYK (−1)",
  "set": "Merck",
  "excluded": false,
  "note": "1 ligand has no FEP+ prediction.",
  "revfep": {
    "n": 3,
    "rmse": 0.73,
    "mae": 0.67,
    "r2": 0.11,
    "rho": 0.5
  },
  "fepplus": {
    "n": 3,
    "rmse": 0.39,
    "mae": 0.32,
    "r2": 0.79,
    "rho": 0.5
  },
  "points": [{
    "l": "CHEMBL3264995",
    "e": -12.39,
    "r": -11.24,
    "f": -11.91
  }, {
    "l": "CHEMBL3264997",
    "e": -10.98,
    "r": -11.14,
    "f": -11.45
  }, {
    "l": "CHEMBL3264998",
    "e": -11.72,
    "r": -12.15,
    "f": null
  }, {
    "l": "CHEMBL3265001",
    "e": -12.03,
    "r": -12.59,
    "f": -12.05
  }]
}, {
  "id": "tnks2",
  "name": "TNKS2",
  "set": "Merck",
  "excluded": false,
  "note": null,
  "revfep": {
    "n": 27,
    "rmse": 0.99,
    "mae": 0.79,
    "r2": 0.3,
    "rho": 0.45
  },
  "fepplus": {
    "n": 27,
    "rmse": 1.11,
    "mae": 0.82,
    "r2": 0.29,
    "rho": 0.45
  },
  "points": [{
    "l": "1a",
    "e": -8.55,
    "r": -8.25,
    "f": -8.46
  }, {
    "l": "1b",
    "e": -9.94,
    "r": -10.16,
    "f": -9.79
  }, {
    "l": "3a",
    "e": -11.0,
    "r": -12.2,
    "f": -12.21
  }, {
    "l": "3b",
    "e": -11.52,
    "r": -11.15,
    "f": -11.95
  }, {
    "l": "5a",
    "e": -10.76,
    "r": -11.31,
    "f": -11.28
  }, {
    "l": "5b",
    "e": -10.47,
    "r": -11.67,
    "f": -11.98
  }, {
    "l": "5c",
    "e": -9.96,
    "r": -10.9,
    "f": -11.87
  }, {
    "l": "5d",
    "e": -10.89,
    "r": -10.37,
    "f": -10.73
  }, {
    "l": "5e",
    "e": -10.11,
    "r": -9.76,
    "f": -10.89
  }, {
    "l": "5f",
    "e": -10.26,
    "r": -11.09,
    "f": -11.32
  }, {
    "l": "5g",
    "e": -10.81,
    "r": -11.63,
    "f": -11.61
  }, {
    "l": "5h",
    "e": -10.05,
    "r": -10.7,
    "f": -10.39
  }, {
    "l": "5i",
    "e": -12.08,
    "r": -11.89,
    "f": -12.81
  }, {
    "l": "5j",
    "e": -11.08,
    "r": -11.58,
    "f": -11.16
  }, {
    "l": "5k",
    "e": -10.96,
    "r": -10.32,
    "f": -10.49
  }, {
    "l": "5l",
    "e": -10.09,
    "r": -11.75,
    "f": -11.72
  }, {
    "l": "5m",
    "e": -12.69,
    "r": -11.03,
    "f": -12.08
  }, {
    "l": "5n",
    "e": -10.71,
    "r": -10.4,
    "f": -10.7
  }, {
    "l": "5o_adjust",
    "e": -12.04,
    "r": -11.17,
    "f": -11.86
  }, {
    "l": "5p",
    "e": -10.5,
    "r": -10.59,
    "f": -10.58
  }, {
    "l": "7",
    "e": -8.4,
    "r": -8.25,
    "f": -8.41
  }, {
    "l": "8a_neutral",
    "e": -10.81,
    "r": -8.63,
    "f": -9.97
  }, {
    "l": "8b_neutral",
    "e": -11.93,
    "r": -9.45,
    "f": -11.48
  }, {
    "l": "8c_neutral_equitorial",
    "e": -10.12,
    "r": -11.1,
    "f": -7.83
  }, {
    "l": "8d_neutral_equitorial",
    "e": -12.49,
    "r": -11.9,
    "f": -9.53
  }, {
    "l": "8e_adjust_neutral",
    "e": -10.24,
    "r": -11.22,
    "f": -8.72
  }, {
    "l": "8f_neutral",
    "e": -11.81,
    "r": -11.79,
    "f": -10.37
  }]
}];

RevFEP results on two public protein–ligand benchmark sets, the JACS set (Wang et al., 2015) and the Merck set (Schindler et al., 2020), using the curated structures from Ross et al. (2023). The page covers 19 targets and 439 ligands. Every RevFEP number comes from a production [RevFEP](/docs/revfep) run with the default settings in [How the Benchmarks Were Run](#how-the-benchmarks-were-run). Nothing was tuned per target.

<Info>
  **RevFEP** numbers are our own results: Revilico ran every target through the RevFEP pipeline. **FEP+** numbers are not our runs. They are Schrödinger's published FEP+ (release 21-4) predictions for the same ligands, downloaded from their public benchmark repository and scored here with the same metrics, so the two can be compared directly.
</Info>

## FEP benchmark overview

Every target, with the better value in each row in bold. The set and overall averages are means of the per-target statistics, so each target counts once whatever its size. Click a target to open it below.

<FepBenchmarkOverview targets={benchmarkTargets} averages={benchmarkAverages} />

## Benchmark details

Pick a target to plot every ligand's predicted against experimental binding free energy. Click a point or a ligand row to see the molecule and its values.

<FepBenchmarkDetails targets={benchmarkTargets} />

<AccordionGroup>
  <Accordion title="Notes on individual targets">
    * **JNK1:** the RevFEP run treats the two flipped binding poses of 17 molecules as separate ligands (38 entries for 21 molecules). FEP+ combines each pair into one value per molecule, so this target is not a like-for-like comparison and is left out of the averages.
    * **c-Met (+1, redocked):** the same six ligands as c-Met (+1), started from redocked poses instead of the poses supplied with the benchmark. It is shown to illustrate how much the starting pose matters, and is left out of the averages.
    * **Ligands without an FEP+ prediction:** CDK8 (1 ligand), SHP2 (1), SYK neutral (3) and SYK −1 (1) have ligands that are missing from the public FEP+ data. The plot shows them for RevFEP only, and the statistics use only the ligands with both predictions.
  </Accordion>

  <Accordion title="All targets in one figure (RevFEP only)">
    <Frame caption="RevFEP results only (our runs): predicted against experimental binding free energy for every benchmark target. The diagonal is perfect agreement; the shaded band is ±1 kcal/mol.">
      <img src="https://mintcdn.com/revilicoinc/WeD9875fVdNFVD5q/images/revfep-benchmark-parity.png?fit=max&auto=format&n=WeD9875fVdNFVD5q&q=85&s=efbca63fdee2ce74fab278291f015383" alt="RevFEP predicted against experimental binding free energies" width="2250" height="1800" data-path="images/revfep-benchmark-parity.png" />
    </Frame>
  </Accordion>
</AccordionGroup>

## Charts

<Tabs>
  <Tab title="By benchmark set">
    <Frame caption="Averages of the per-target statistics for each benchmark set (the average rows of the overview table). RevFEP numbers are our runs; FEP+ numbers are Schrödinger's published results.">
      <img src="https://mintcdn.com/revilicoinc/2edXI-EuouFEBZx6/images/benchmarks/fep-summary-by-set.png?fit=max&auto=format&n=2edXI-EuouFEBZx6&q=85&s=2227252c75e8a18ae039abadc10611b2" alt="Mean per-target RMSE and Spearman correlation for RevFEP and FEP+ on each benchmark set" width="1665" height="922" data-path="images/benchmarks/fep-summary-by-set.png" />
    </Frame>
  </Tab>

  <Tab title="By target">
    <Frame caption="RMSE of ΔG for each of the 17 averaged targets, sorted by RevFEP RMSE. The dotted line marks 1 kcal/mol; n is the number of ligands with both predictions.">
      <img src="https://mintcdn.com/revilicoinc/2edXI-EuouFEBZx6/images/benchmarks/fep-rmse-by-target.png?fit=max&auto=format&n=2edXI-EuouFEBZx6&q=85&s=736a9e6e1d7ae88e99b972a2c389c195" alt="RMSE per target for RevFEP and FEP+" width="1774" height="1162" data-path="images/benchmarks/fep-rmse-by-target.png" />
    </Frame>
  </Tab>

  <Tab title="All ligands">
    <Frame caption="Predicted against experimental ΔG for all 389 ligands that have both a RevFEP and an FEP+ prediction, across the 17 averaged targets. The shaded band is ±1 kcal/mol. These pooled errors weight large targets more, so they differ from the per-target averages in the Summary.">
      <img src="https://mintcdn.com/revilicoinc/2edXI-EuouFEBZx6/images/benchmarks/fep-parity-revfep-vs-fepplus.png?fit=max&auto=format&n=2edXI-EuouFEBZx6&q=85&s=1c43f1b968b197ef05ec3ac4e91acad5" alt="Predicted against experimental binding free energy, RevFEP and FEP+ side by side" width="1739" height="918" data-path="images/benchmarks/fep-parity-revfep-vs-fepplus.png" />
    </Frame>
  </Tab>

  <Tab title="Error distribution">
    <Frame caption="Share of the same 389 ligands whose predicted ΔG is within a given absolute error of experiment. RevFEP: 59% within 1 kcal/mol and 86% within 2 kcal/mol. FEP+: 76% and 97%.">
      <img src="https://mintcdn.com/revilicoinc/2edXI-EuouFEBZx6/images/benchmarks/fep-error-within.png?fit=max&auto=format&n=2edXI-EuouFEBZx6&q=85&s=c0df95d596a40b00d20fb1f92a5f7f1b" alt="Cumulative share of ligands within a given error for RevFEP and FEP+" width="1515" height="885" data-path="images/benchmarks/fep-error-within.png" />
    </Frame>
  </Tab>
</Tabs>

## How the Benchmarks Were Run

Every target was run through the standard RevFEP pipeline with its default settings. Nothing was tuned per target.

| Setting | Value |
| - | - |
| Engine | RevFEP relative hybrid-topology protocol on OpenMM 8.2 |
| Force fields | Amber ff14SB (protein), OpenFF 2.2.0 (ligands), TIP3P water, 0.15 M NaCl |
| Ligand charges | AM1-BCC |
| Network | LOMAP atom mapping, minimal spanning tree (one path between each pair of ligands) |
| Sampling | 11 λ windows, Hamiltonian replica exchange, 1 ns equilibration and 5 ns production per window |
| Integrator | 2 fs time step, hydrogen bonds constrained, no hydrogen mass repartitioning |
| Repeats | 1 |
| Solvent padding | 1.5 nm |
| Hardware | NVIDIA L40S GPUs on AWS (about 4.3 GPU-hours per edge, median) |

Structures, ligand poses and protonation states are the ones published with the benchmark. RevFEP does not change the net charge of a ligand along an edge, so series that mix charge states (c-Met, Eg5, SYK) were run as separate same-charge subsets. The published FEP+ results include charge-changing edges and run each of these series as one network.

## How the Numbers Are Calculated

<AccordionGroup>
  <Accordion title="ΔG per ligand">
    RevFEP's relative free energies (ΔΔG) are combined into one ΔG per ligand. Because relative calculations fix ΔG only up to a constant, the predictions for each target are shifted so that their mean equals the mean experimental value. The FEP+ values are shifted the same way, over the same ligands.
  </Accordion>

  <Accordion title="Error and correlation metrics">
    * **RMSE** and **MAE** are the root-mean-square and mean absolute errors of ΔG, in kcal/mol. Lower is better.
    * **R²** is the square of the Pearson correlation coefficient. Higher is better.
    * **ρ** is Spearman's rank correlation, which measures how well the ligands are ranked. Higher is better.
  </Accordion>

  <Accordion title="FEP+ reference">
    Ligand-level FEP+ (release 21-4) predictions are taken from Schrödinger's public benchmark repository, matched to RevFEP ligands by name. The experimental values in both data sets agree to within 0.01 kcal/mol for every matched ligand.
  </Accordion>
</AccordionGroup>

## Reading These Results

<AccordionGroup>
  <Accordion title="These are single runs">
    Each edge was simulated once, and a minimal spanning tree has no closed cycles, so these results come without a statistical uncertainty or a cycle-closure check. Repeat runs and networks with cycles would add an error bar and a consistency check.
  </Accordion>

  <Accordion title="Small sets are noisy">
    R² and ρ computed on fewer than about 15 ligands (c-Met, Eg5 neutral, SYK −1, Thrombin) can change a lot when one ligand moves.
  </Accordion>

  <Accordion title="The FEP+ results are not a like-for-like workflow">
    They were produced with a different force field (OPLS4), a different sampling protocol, and per-target corrections for protonation states, tautomers and alternative ring flips that were applied by experts. RevFEP runs every target the same way with no manual corrections.
  </Accordion>

  <Accordion title="Charged series are harder">
    The largest errors are on the +1 series (c-Met, Eg5) and on SHP2. The redocked c-Met run shows that the starting pose alone can move the RMSE from 1.87 to 0.70 kcal/mol for the same ligands.
  </Accordion>

  <Accordion title="Not yet covered">
    BACE from the JACS set, and the other groups in Ross et al. (2023), have not been run with RevFEP.
  </Accordion>
</AccordionGroup>

## Sources

* Wang, L. et al. *Accurate and Reliable Prediction of Relative Ligand Binding Potency in Prospective Drug Discovery by Way of a Modern Free-Energy Calculation Protocol and Force Field.* J. Am. Chem. Soc. 137, 2695–2703 (2015). [doi:10.1021/ja512751q](https://doi.org/10.1021/ja512751q)
* Schindler, C. E. M. et al. *Large-Scale Assessment of Binding Free Energy Calculations in Active Drug Discovery Projects.* J. Chem. Inf. Model. 60, 5457–5474 (2020). [doi:10.1021/acs.jcim.0c00900](https://doi.org/10.1021/acs.jcim.0c00900)
* Ross, G. A. et al. *The maximal and current accuracy of rigorous protein-ligand binding free energy calculations.* Commun. Chem. 6, 222 (2023). [doi:10.1038/s42004-023-01019-9](https://doi.org/10.1038/s42004-023-01019-9)
* FEP+ 21-4 ligand predictions: [schrodinger/public\_binding\_free\_energy\_benchmark](https://github.com/schrodinger/public_binding_free_energy_benchmark), `21_4_results/ligand_predictions/`


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.