AlphaFind v2: Evaluation data, results and reproducibility protocol

This dataset contains the evalulation dataset, code and results reported on in the publication: <i>AlphaFind v2: Similarity Search in AlphaFold DB and TED Domains across Structural Contexts</i> (https://doi.org/10.64898/2026.03.10.710735).The evaluation uses the multi-domain protein selection from https://doi.org/10.6084/m9.figshare.30546650 (<i>afdb-benchmark/</i><i>af-cath-multi-domain-list.tsv</i>) and downloads this data from AlphaFold DB (https://alphafold.ebi.ac.uk/) and TED DB (https://ted.cathdb.info/) API services. The dataset is then saved as <i>afdb-structures/</i> (2050 multidomain protein chains from AlphaFold DB) and <i>afdb-structures-domains/ (</i>4420 TED domains<i> </i>extracted from the 2050 multidomain proteins).Contents<code>alphafind-evaluation-data.zip</code>├── afdb-benchmark/├── afdb-structures/└── afdb-structures-domains/<code>results.zip</code>├── foldseek_results/ # Raw FoldSeek API responses│ └── afdb50_AF-{UNIPROT_ID}-F1-model_v6.json│├── foldseek_results_tmscores/ # FoldSeek results with TM-scores│ └── foldseek_results_tmscores_{UNIPROT_ID}.csv│├── alphafindv1_results/ # AlphaFind v1 search results│ └── {UNIPROT_ID}_chainA_limit{K}.json│├── alphafindv2_results/ # AlphaFind v2 chain search results│ └── {UNIPROT_ID}_chains_k{K}.json│├── alphafindv2_domains_results/ # AlphaFind v2 domain search results│ └── {UNIPROT_ID}_TED{NN}_chains_k{K}.json│├── merizo_results/ # Merizo domain search results│ ├── AF-{UNIPROT_ID}-F1-model_v4_TED{NN}_results.json│ └── AF-{UNIPROT_ID}-F1-model_v4_TED{NN}_search.tsv│├── figures/ # Comparison plots│ ├── chains_comparison_tm.pdf # Chains TM-score boxplot│ ├── chains_comparison_tm.png│ ├── domains_comparison_tm.pdf # Domains TM-score boxplot│ └── domains_comparison_tm.png│├── *_with_timing.csv # Timing data for each method├── foldseek-nresults.csv # Result counts per query└── *_downloads.csv # Download logs<code>statistical_tests.zip</code>├── aggregate_results_chains_statistics.csv├── aggregate_results_chains_with_stats.py├── aggregate_results_domains_statistics.csv└── statistical_tests.md<code>alphafind-v2-evaluation-scripts.zip</code>├── README.md├── aggregate_results.py├── aggregate_results_chains_with_stats.py├── compute-tms.py├── count-foldseek-results.py├── download_data.py├── eval-alphafindv1.py├── eval-alphafindv2.py├── eval-alphafindv2_domains.py├── eval-foldseek.py├── eval-merizo.py├── extract-foldseek.py├── find_domain_outliers.py├── plot_domains_comparison.py├── plot_input_statistics.py├── plot_results_comparison.py├── requirements.txt└── visualize_results.py <code>figures.zip</code> ├── chains_comparison.pdf ├── chains_comparison_time.pdf ├── chains_comparison_tm.pdf ├── domains_comparison_time.pdf └── domains_comparison_tm.pdf ├── cath_domains_per_chain.pdf ├── cath_unique_families_per_chain.pdf ├── chain_atoms_histogram.pdf ├── chain_residues_histogram.pdf ├── domain_atoms_histogram.pdf └── domain_residues_histogram.pdfHow to reproduceThe instructions are also in the <code>README.md </code>of<code> </code><code>alphafind-v2-evaluation-scripts.zip</code><b>Prerequisites</b><br>Python (Originally run on Python 3.10.16)<code>USalign</code> - for TM-score computation, <code>make</code> for USalign compilation<code>git clone https://github.com/pylelab/USalign.git</code><code>cd USalign &amp;&amp; make</code>Python dependencies<code>pip install numpy pandas scipy requests tqdm matplotlib</code><b>Download the data</b><code>python download_data.py</code>This downloads:Protein chain PDB files to <code>afdb-structures/</code>Domain PDB files to <code>afdb-structures-domains/</code>Alternatively, you can use the included <code>alphafind-evaluation-data.zip</code> and just move the subdirectories into the main directory structure: <code>cd alphafind-evaluation-data/ &amp;&amp; mv * ../.</code><b>Run FoldSeek Search</b>Run FoldSeek Server API search first (required to determine result counts for other methods):<code>python eval-foldseek.py</code>Searches against `afdb-50` databaseResults saved to `results/foldseek_results/`Timing saved to `results/foldseek_results_with_timing.csv`<b>Prepare FoldSeek results for TM-Score computation</b><code>python extract-foldseek.py</code>Extracts results from foldseek evaluation to individual CSV files in <code>results/foldseek_results_tmscores/</code><b>Compute TM-Scores</b><code>python compute-tms.py --input-dir results/foldseek_results_tmscores</code>Uses USalign to compute TM-scores for FoldSeek results. The timing is not included in search time.<b>Count the FoldSeek results</b><code>python count-foldseek-results.py</code>Creates <code>foldseek-nresults.csv</code> used to match result counts in AlphaFind queries.<b>Run AlphaFind v1 Search</b><code>python eval-alphafindv1.py</code>Queries specify UniProt ID, chain (A), and limit matching FoldSeek result countsResults saved to `results/alphafindv1_results/`TM-scores returned directly by API<b>Run AlphaFind v2 Search</b>For chains: <code>python eval-alphafindv2.py</code>For domains:<code>python eval-alphafindv2_domains.py</code>Queries use `k` parameter matching baseline result countsTwo timing metrics recorded:- Approximate time: until initial results collected- TM-score time: until exact TM-scores computedResults saved to <code>results/alphafindv2_results/</code> or <code>results/alphafindv2_domains_results/</code><b>Run Merizo Search (Domains)</b><code>python eval-merizo.py</code>Searches against TED databaseResults saved to `results/merizo_results/`TM-scores returned directly (columns: `q_tm`, `t_tm`, `max_tm`)<b>Aggregate Results with Statistical Testing</b>This step produces final summary tables and p-values.<code>cd results/ &amp;&amp; </code><code>python ../aggregate_results_chains_with_stats.py</code><b>Outputs:</b><code>aggregate_results_chains.csv</code> - Chain performance summary<code>aggregate_results_chains_statistics.csv</code> - Chain statistical tests<code>aggregate_results_domains.csv</code> - Domain performance summary<code>aggregate_results_domains_statistics.csv</code> - Domain statistical tests<b>Generate Visualization Plots</b><code>python plot_input_statistics.py</code>From <code>results/</code> directory:<code>cd results/</code><code>python ../plot_results_comparison.py</code> # Chains boxplot<code>python ../plot_domains_comparison.py</code> # Domains boxplotOutputs:<code>input_statistics/*.pdf</code> - Input data histograms<code>results/figures/chains_comparison_tm.pdf</code> - Chains TM-score comparison<code>results/figures/domains_comparison_tm.pdf</code> - Domains TM-score comparison<br><br>