{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T23:07:09Z","timestamp":1787094429469,"version":"3.56.0"},"reference-count":58,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2024,3,6]],"date-time":"2024-03-06T00:00:00Z","timestamp":1709683200000},"content-version":"vor","delay-in-days":44,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,1,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Identifying protein\u2013protein interactions (PPIs) is crucial for deciphering biological pathways. Numerous prediction methods have been developed as cheap alternatives to biological experiments, reporting surprisingly high accuracy estimates. We systematically investigated how much reproducible deep learning models depend on data leakage, sequence similarities and node degree information, and compared them with basic machine learning models. We found that overlaps between training and test sets resulting from random splitting lead to strongly overestimated performances. In this setting, models learn solely from sequence similarities and node degrees. When data leakage is avoided by minimizing sequence similarities between training and test set, performances become random. Moreover, baseline models directly leveraging sequence similarity and network topology show good performances at a fraction of the computational cost. Thus, we advocate that any improvements should be reported relative to baseline methods in the future. Our findings suggest that predicting PPIs remains an unsolved task for proteins showing little sequence similarity to previously studied proteins, highlighting that further experimental research into the \u2018dark\u2019 protein interactome and better computational methods are needed.<\/jats:p>","DOI":"10.1093\/bib\/bbae076","type":"journal-article","created":{"date-parts":[[2024,2,18]],"date-time":"2024-02-18T16:38:12Z","timestamp":1708274292000},"source":"Crossref","is-referenced-by-count":96,"title":["Cracking the black box of deep sequence-based protein\u2013protein interaction prediction"],"prefix":"10.1093","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5812-8013","authenticated-orcid":false,"given":"Judith","family":"Bernett","sequence":"first","affiliation":[{"name":"Data Science in Systems Biology , TUM School of Life Sciences, , Maximus-von-Imhof Forum 3, 85354, Freising , Germany"},{"name":"Technical University of Munich , TUM School of Life Sciences, , Maximus-von-Imhof Forum 3, 85354, Freising , Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8651-750X","authenticated-orcid":false,"given":"David B","family":"Blumenthal","sequence":"additional","affiliation":[{"name":"Biomedical Network Science Lab , Department Artificial Intelligence in Biomedical Engineering, , Werner-von-Siemens-Str. 61, 91052, Erlangen , Germany"},{"name":"Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg , Department Artificial Intelligence in Biomedical Engineering, , Werner-von-Siemens-Str. 61, 91052, Erlangen , Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0941-4168","authenticated-orcid":false,"given":"Markus","family":"List","sequence":"additional","affiliation":[{"name":"Data Science in Systems Biology , TUM School of Life Sciences, , Maximus-von-Imhof Forum 3, 85354, Freising , Germany"},{"name":"Technical University of Munich , TUM School of Life Sciences, , Maximus-von-Imhof Forum 3, 85354, Freising , Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,3,5]]},"reference":[{"key":"2024030617022030100_ref1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2014\/147648","article-title":"Protein-protein interaction detection: methods and analysis","volume":"2014","author":"Srinivasa Rao","year":"2014","journal-title":"Int J Proteomics"},{"key":"2024030617022030100_ref2","article-title":"Comparing two deep learning sequence-based models for protein-protein interaction prediction","author":"Richoux","year":"2019","journal-title":"arXiv preprint arXiv:190106268"},{"issue":"1","key":"2024030617022030100_ref3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12859-017-1700-2","article-title":"Sequence-based prediction of protein protein interaction using a deep-learning algorithm","volume":"18","author":"Sun","year":"2017","journal-title":"BMC Bioinformatics"},{"issue":"14","key":"2024030617022030100_ref4","doi-asserted-by":"crossref","first-page":"i305","DOI":"10.1093\/bioinformatics\/btz328","article-title":"Multifaceted protein\u2013protein interaction prediction based on siamese residual rcnn","volume":"35","author":"Chen","year":"2019","journal-title":"Bioinformatics"},{"issue":"1","key":"2024030617022030100_ref5","first-page":"1","article-title":"Predicting protein-protein interactions from matrix-based protein sequence using convolution neural network and feature-selective rotation forest","volume":"9","author":"Wang","year":"2019","journal-title":"Sci Rep"},{"issue":"8","key":"2024030617022030100_ref6","first-page":"2020","article-title":"Protein-protein interactions prediction based on graph energy and protein sequence information","volume":"25","author":"Da","year":"1841","journal-title":"Molecules"},{"issue":"11","key":"2024030617022030100_ref7","doi-asserted-by":"crossref","first-page":"2373","DOI":"10.3390\/ijms18112373","article-title":"Protein-protein interactions prediction using a novel local conjoint triad descriptor of amino acid sequences","volume":"18","author":"Wang","year":"2017","journal-title":"Int J Mol Sci"},{"issue":"5","key":"2024030617022030100_ref8","doi-asserted-by":"crossref","first-page":"e0125811","DOI":"10.1371\/journal.pone.0125811","article-title":"Predicting protein-protein interactions from primary protein sequences using a novel multi-scale local feature representation scheme and the random forest","volume":"10","author":"You","year":"2015","journal-title":"PloS One"},{"key":"2024030617022030100_ref9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2015\/867516","article-title":"Detecting protein-protein interactions with a novel matrix-based protein sequence representation and support vector machines","volume":"2015","author":"You","year":"2015","journal-title":"Biomed Res Int"},{"issue":"8","key":"2024030617022030100_ref10","first-page":"1","article-title":"Prediction of protein-protein interactions from amino acid sequences with ensemble extreme learning machines and principal component analysis","volume":"14","author":"You","year":"2013","journal-title":"BMC Bioinformatics"},{"issue":"4","key":"2024030617022030100_ref11","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1109\/TNB.2015.2429672","article-title":"Discovering variable-length patterns in protein sequences for protein-protein interaction prediction","volume":"14","author":"Lun","year":"2015","journal-title":"IEEE Trans Nanobioscience"},{"issue":"6","key":"2024030617022030100_ref12","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1021\/acs.jcim.7b00028","article-title":"Deepppi: boosting prediction of protein\u2013protein interactions with deep neural networks","volume":"57","author":"Xiuquan","year":"2017","journal-title":"J Chem Inf Model"},{"key":"2024030617022030100_ref13","doi-asserted-by":"crossref","first-page":"e7126","DOI":"10.7717\/peerj.7126","article-title":"An integration of deep learning with feature embedding for protein\u2013protein interaction prediction","volume":"7","author":"Yao","year":"2019","journal-title":"PeerJ"},{"issue":"1","key":"2024030617022030100_ref14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-020-75467-x","article-title":"Amalgamation of 3d structure and sequence information for protein\u2013protein interaction prediction","volume":"10","author":"Jha","year":"2020","journal-title":"Sci Rep"},{"issue":"4","key":"2024030617022030100_ref15","doi-asserted-by":"crossref","first-page":"820","DOI":"10.1039\/c3mb70486f","article-title":"Ensemble learning prediction of protein\u2013protein interactions using proteins functional annotations","volume":"10","author":"Saha","year":"2014","journal-title":"Mol Biosyst"},{"issue":"1","key":"2024030617022030100_ref16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/bib\/bbx068","article-title":"Protein-protein interaction prediction using a hybrid feature representation and a stacked generalization scheme","volume":"20","author":"Chen","year":"2019","journal-title":"BMC Bioinformatics"},{"key":"2024030617022030100_ref17","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1016\/j.omtn.2020.08.025","article-title":"Conjoint feature representation of go and protein sequence for ppi prediction based on an inception rnn attention network","volume":"22","author":"Zhao","year":"2020","journal-title":"Molecular Therapy-Nucleic Acids"},{"issue":"17","key":"2024030617022030100_ref18","doi-asserted-by":"crossref","first-page":"i802","DOI":"10.1093\/bioinformatics\/bty573","article-title":"Predicting protein\u2013protein interactions through sequence-based deep learning","volume":"34","author":"Hashemifar","year":"2018","journal-title":"Bioinformatics"},{"issue":"1","key":"2024030617022030100_ref19","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1093\/bioinformatics\/btr610","article-title":"Gene ontology-driven inference of protein\u2013protein interactions using inducers","volume":"28","author":"Maetschke","year":"2012","journal-title":"Bioinformatics"},{"issue":"10","key":"2024030617022030100_ref20","doi-asserted-by":"crossref","first-page":"969","DOI":"10.1016\/j.cels.2021.08.010","article-title":"D-script translates genome to phenome with sequence-based, structure-aware, genome-scale predictions of protein-protein interactions","volume":"12","author":"Sledzieski","year":"2021","journal-title":"Cell Syst"},{"issue":"Supplement_1","key":"2024030617022030100_ref21","doi-asserted-by":"crossref","first-page":"i264","DOI":"10.1093\/bioinformatics\/btac258","article-title":"Topsy-turvy: integrating a global view into sequence-based ppi prediction","volume":"38","author":"Singh","year":"2022","journal-title":"Bioinformatics"},{"issue":"6","key":"2024030617022030100_ref22","doi-asserted-by":"crossref","first-page":"454","DOI":"10.2174\/1389202921999200625103936","article-title":"Evolution of sequence-based bioinformatics tools for protein-protein interaction prediction","volume":"21","author":"Khatun","year":"2020","journal-title":"Curr Genomics"},{"issue":"7873","key":"2024030617022030100_ref23","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","article-title":"Highly accurate protein structure prediction with alphafold","volume":"596","author":"Jumper","year":"2021","journal-title":"Nature"},{"key":"2024030617022030100_ref24","article-title":"Protein complex prediction with alphafold-multimer","author":"Evans","year":"2021","journal-title":"BioRxiv"},{"issue":"12","key":"2024030617022030100_ref25","doi-asserted-by":"crossref","first-page":"1134","DOI":"10.1038\/nmeth.2259","article-title":"Flaws in evaluation schemes for pair-input computational predictions","volume":"9","author":"Park","year":"2012","journal-title":"Nat Methods"},{"issue":"10","key":"2024030617022030100_ref26","doi-asserted-by":"crossref","first-page":"1521","DOI":"10.1093\/bioinformatics\/btu857","article-title":"More challenges for machine-learning protein interactions","volume":"31","author":"Hamp","year":"2015","journal-title":"Bioinformatics"},{"key":"2024030617022030100_ref27","doi-asserted-by":"crossref","first-page":"102344","DOI":"10.1016\/j.sbi.2022.102344","article-title":"Recent advances in predicting protein\u2013protein interactions with the aid of artificial intelligence algorithms","volume":"73","author":"Li","year":"2022","journal-title":"Curr Opin Struct Biol"},{"issue":"3","key":"2024030617022030100_ref28","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1038\/s41576-021-00434-9","article-title":"Navigating the pitfalls of applying machine learning in genomics","volume":"23","author":"Whalen","year":"2022","journal-title":"Nat Rev Genet"},{"key":"2024030617022030100_ref29","article-title":"Leakage and the reproducibility crisis in ML-based science","author":"Kapoor","year":"2022","journal-title":"arXiv preprint arXiv:220707048"},{"issue":"4","key":"2024030617022030100_ref30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2382577.2382579","article-title":"Leakage in data mining: formulation, detection, and avoidance","volume":"6","author":"Kaufman","year":"2012","journal-title":"ACM Trans Knowl Discov Data"},{"issue":"1","key":"2024030617022030100_ref31","doi-asserted-by":"crossref","DOI":"10.1038\/s41467-023-37572-z","article-title":"Improving the generalizability of protein-ligand binding predictions with ai-bind","volume":"14","author":"Chatterjee","journal-title":"Nat Commun"},{"issue":"1","key":"2024030617022030100_ref32","first-page":"1","article-title":"Choosing negative examples for the prediction of protein-protein interactions","volume":"7","author":"Ben-Hur","year":"2006","journal-title":"BMC Bioinformatics"},{"issue":"16","key":"2024030617022030100_ref33","doi-asserted-by":"crossref","first-page":"2833","DOI":"10.1002\/pmic.200700131","article-title":"Methods for the detection and analysis of protein\u2013protein interactions","volume":"7","author":"Bergg\u00e5rd","year":"2007","journal-title":"Proteomics"},{"key":"2024030617022030100_ref34","first-page":"912","article-title":"Semi-supervised learning using gaussian fields and harmonic functions","volume-title":"Proceedings of the 20th International conference on Machine learning (ICML-03)","author":"Zhu","year":"2003"},{"key":"2024030617022030100_ref35","article-title":"Learning with local and global consistency","volume":"16","author":"Zhou","year":"2003","journal-title":"Adv Neural Inf Process Syst"},{"issue":"1","key":"2024030617022030100_ref36","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12859-016-1414-x","article-title":"Sprint: ultrafast protein-protein interaction prediction of the entire human interactome","volume":"18","author":"Li","year":"2017","journal-title":"BMC Bioinformatics"},{"issue":"9","key":"2024030617022030100_ref37","doi-asserted-by":"crossref","first-page":"3025","DOI":"10.1093\/nar\/gkn159","article-title":"Using support vector machine combined with auto covariance to predict protein\u2013protein interactions from protein sequences","volume":"36","author":"Guo","year":"2008","journal-title":"Nucleic Acids Res"},{"issue":"10","key":"2024030617022030100_ref38","doi-asserted-by":"crossref","first-page":"4992","DOI":"10.1021\/pr100618t","article-title":"Large-scale prediction of human protein- protein interactions from amino acid sequence based on latent topic features","volume":"9","author":"Pan","year":"2010","journal-title":"J Proteome Res"},{"issue":"D1","key":"2024030617022030100_ref39","doi-asserted-by":"crossref","first-page":"D279","DOI":"10.1093\/nar\/gkt970","article-title":"Simap-the database of all-against-all protein sequence similarities and annotations with new interfaces and increased coverage","volume":"42","author":"Arnold","year":"2014","journal-title":"Nucleic Acids Res"},{"key":"2024030617022030100_ref40","first-page":"1","article-title":"Using weighted sparse representation model combined with discrete cosine transformation to predict protein-protein interactions from protein sequence","volume":"2015","author":"Huang","year":"2015","journal-title":"Biomed Res Int"},{"issue":"1","key":"2024030617022030100_ref41","first-page":"1","article-title":"Using two-dimensional principal component analysis and rotation forest for prediction of protein-protein interactions","volume":"8","author":"Wang","year":"2018","journal-title":"Sci Rep"},{"issue":"1","key":"2024030617022030100_ref42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12859-016-1253-9","article-title":"Predicting protein-protein interactions via multivariate mutual information of protein sequences","volume":"17","author":"Ding","year":"2016","journal-title":"BMC Bioinformatics"},{"issue":"1","key":"2024030617022030100_ref43","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1756-0500-3-145","article-title":"Pred_ppi: a server for predicting protein-protein interactions based on sequence data with probability assignment","volume":"3","author":"Guo","year":"2010","journal-title":"BMC Res Notes"},{"issue":"11","key":"2024030617022030100_ref44","doi-asserted-by":"crossref","first-page":"4337","DOI":"10.1073\/pnas.0607879104","article-title":"Predicting protein\u2013protein interactions based only on sequences information","volume":"104","author":"Shen","year":"2007","journal-title":"Proc Natl Acad Sci"},{"key":"2024030617022030100_ref45","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-981-15-1081-6_1","article-title":"Effect of dimensionality reduction on classification accuracy for protein\u2013protein interaction prediction","volume-title":"Advanced Computing and Intelligent Engineering","author":"Mahapatra","year":"2020"},{"issue":"8","key":"2024030617022030100_ref46","doi-asserted-by":"crossref","first-page":"2269","DOI":"10.1093\/bioinformatics\/btac104","article-title":"Transformergo: predicting protein\u2013protein interactions by modelling the attention between sets of gene ontology terms","volume":"38","author":"Ieremie","year":"2022","journal-title":"Bioinformatics"},{"key":"2024030617022030100_ref47","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1007\/978-3-642-38527-8_16","article-title":"Think locally, act globally: Highly balanced graph partitioning","volume-title":"International Symposium on Experimental Algorithms","author":"Sanders","year":"2013"},{"key":"2024030617022030100_ref48","article-title":"PPI prediction from sequence, gold standard dataset","author":"Bernett","year":"2022","journal-title":"figshare"},{"key":"2024030617022030100_ref49","doi-asserted-by":"crossref","first-page":"D408","DOI":"10.1093\/nar\/gkw985","article-title":"Hippie v2. 0: enhancing meaningfulness and reliability of protein\u2013protein interaction networks","volume":"45","author":"Alanis-Lobato","year":"2016","journal-title":"Nucleic Acids Res"},{"issue":"23","key":"2024030617022030100_ref50","doi-asserted-by":"crossref","first-page":"3150","DOI":"10.1093\/bioinformatics\/bts565","article-title":"CD-HIT: accelerated for clustering the next-generation sequencing data","volume":"28","author":"Limin","year":"2012","journal-title":"Bioinformatics"},{"issue":"9","key":"2024030617022030100_ref51","doi-asserted-by":"crossref","first-page":"609","DOI":"10.1093\/protein\/14.9.609","article-title":"Similarity of phylogenetic trees as indicator of protein\u2013protein interaction","volume":"14","author":"Pazos","year":"2001","journal-title":"Protein Eng"},{"issue":"13","key":"2024030617022030100_ref52","doi-asserted-by":"crossref","first-page":"2166","DOI":"10.1093\/bioinformatics\/btv102","article-title":"Detection of significant protein coevolution","volume":"31","author":"Ochoa","year":"2015","journal-title":"Bioinformatics"},{"issue":"12","key":"2024030617022030100_ref53","doi-asserted-by":"crossref","first-page":"1945","DOI":"10.1093\/bioinformatics\/btv077","article-title":"Evolutionary profiles improve protein\u2013protein interaction prediction from sequence","volume":"31","author":"Hamp","year":"2015","journal-title":"Bioinformatics"},{"issue":"D1","key":"2024030617022030100_ref54","doi-asserted-by":"crossref","first-page":"D309","DOI":"10.1093\/nar\/gkaa768","article-title":"Digger: exploring the functional role of alternative splicing in protein interactions","volume":"49","author":"Louadi","year":"2021","journal-title":"Nucleic Acids Res"},{"issue":"8","key":"2024030617022030100_ref55","first-page":"100275","article-title":"Illuminating the dark protein-protein interactome. Cell reports","volume":"2","author":"Tabar","year":"2022","journal-title":"Methods"},{"issue":"D1","key":"2024030617022030100_ref56","doi-asserted-by":"crossref","first-page":"D396","DOI":"10.1093\/nar\/gkt1079","article-title":"Negatome 2.0: a database of non-interacting proteins derived by literature mining, manual annotation and protein structure analysis","volume":"42","author":"Blohm","year":"2014","journal-title":"Nucleic Acids Res"},{"key":"2024030617022030100_ref57","first-page":"321","article-title":"Learning with local and global consistency","volume-title":"Advances in Neural Information Processing Systems 16 [Neural Information Processing Systems, NIPS 2003, December 8\u201313, 2003, Vancouver and Whistler, British Columbia, Canada]","author":"Zhou","year":"2003"},{"key":"2024030617022030100_ref58","doi-asserted-by":"crossref","first-page":"1128","DOI":"10.1038\/s41592-021-01241-0","article-title":"The AIMe registry for artificial intelligence in biomedical research","volume":"18","author":"Matschinske","year":"2021","journal-title":"Nat Methods"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/25\/2\/bbae076\/56860721\/bbae076.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/25\/2\/bbae076\/56860721\/bbae076.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,6]],"date-time":"2024-03-06T12:17:14Z","timestamp":1709727434000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbae076\/7621029"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,22]]},"references-count":58,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,1,22]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbae076","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.01.18.524543","asserted-by":"object"}]},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,3,1]]},"published":{"date-parts":[[2024,1,22]]},"article-number":"bbae076"}}