{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T04:31:42Z","timestamp":1775190702111,"version":"3.50.1"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1009492","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,3,17]],"date-time":"2022-03-17T00:00:00Z","timestamp":1647475200000}}],"reference-count":23,"publisher":"Public Library of Science (PLoS)","issue":"3","license":[{"start":{"date-parts":[[2022,3,7]],"date-time":"2022-03-07T00:00:00Z","timestamp":1646611200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSF-Simons Center for the Mathematical and Statistical Analysis of Biology","award":["1764269"],"award-info":[{"award-number":["1764269"]}]},{"DOI":"10.13039\/100000051","name":"National Human Genome Research Institute","doi-asserted-by":"publisher","award":["R01-HG009116"],"award-info":[{"award-number":["R01-HG009116"]}],"id":[{"id":"10.13039\/100000051","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>\n                    Biological sequence families contain many sequences that are very similar to each other because they are related by evolution, so the strategy for splitting data into separate training and test sets is a nontrivial choice in benchmarking sequence analysis methods. A random split is insufficient because it will yield test sequences that are closely related or even identical to training sequences. Adapting ideas from independent set graph algorithms, we describe two new methods for splitting sequence data into dissimilar training and test sets. These algorithms input a sequence family and produce a split in which each test sequence is less than\n                    <jats:italic>p<\/jats:italic>\n                    % identical to any individual training sequence. These algorithms successfully split more families than a previous approach, enabling construction of more diverse benchmark datasets.\n                  <\/jats:p>","DOI":"10.1371\/journal.pcbi.1009492","type":"journal-article","created":{"date-parts":[[2022,3,7]],"date-time":"2022-03-07T13:44:38Z","timestamp":1646660678000},"page":"e1009492","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":16,"title":["Constructing benchmark test sets for biological sequence analysis using independent set algorithms"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8281-8161","authenticated-orcid":true,"given":"Samantha","family":"Petti","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6676-4706","authenticated-orcid":true,"given":"Sean R.","family":"Eddy","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2022,3,7]]},"reference":[{"key":"pcbi.1009492.ref001","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1016\/j.sbi.2011.03.005","article-title":"Protein Sequence Comparison and Fold Recognition: Progress and Good-Practice Benchmarking","volume":"21","author":"J S\u00f6ding","year":"2011","journal-title":"Curr Opin Struct Biol"},{"key":"pcbi.1009492.ref002","doi-asserted-by":"crossref","first-page":"831","DOI":"10.1093\/bib\/bbv082","article-title":"Correct Machine Learning on Protein Sequences: A Peer-Reviewing Perspective","volume":"17","author":"I Walsh","year":"2015","journal-title":"Brief Bioinform"},{"key":"pcbi.1009492.ref003","doi-asserted-by":"crossref","first-page":"659","DOI":"10.1038\/s41580-019-0176-5","article-title":"Setting the Standards for Machine Learning in Biology","volume":"20","author":"DT Jones","year":"2019","journal-title":"Nat Rev Mol Cell Bio"},{"key":"pcbi.1009492.ref004","article-title":"DOME: Recommendations for Supervised Machine Learning Validation in Biology","author":"ELIXIR Machine Learning Focus Group","year":"2021","journal-title":"Nat Methods"},{"key":"pcbi.1009492.ref005","unstructured":"Arpit D, Jastrzebski S, Ballas N, Krueger D, Bengio E, Kanwal MS, et al. A closer look at memorization in deep networks. In: Proc Int Conf Mach Learn. Proc Mach Learn Res; 2017. p. 233\u2013242."},{"key":"pcbi.1009492.ref006","doi-asserted-by":"crossref","first-page":"1335","DOI":"10.1093\/bioinformatics\/btp157","article-title":"Infernal 1.0: Inference of RNA Alignments","volume":"25","author":"EP Nawrocki","year":"2009","journal-title":"Bioinformatics"},{"key":"pcbi.1009492.ref007","doi-asserted-by":"crossref","first-page":"e1002195","DOI":"10.1371\/journal.pcbi.1002195","article-title":"Accelerated profile HMM searches","volume":"7","author":"SR Eddy","year":"2011","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1009492.ref008","doi-asserted-by":"crossref","first-page":"1315","DOI":"10.1038\/s41592-019-0598-1","article-title":"Unified rational protein engineering with sequence-based deep representation learning","volume":"16","author":"EC Alley","year":"2019","journal-title":"Nat Methods"},{"key":"#cr-split#-pcbi.1009492.ref009.1","doi-asserted-by":"crossref","unstructured":"Bileschi ML, Belanger D, Bryant DH, Sanderson T, Carter B, Sculley D, et al. Using deep learning to annotate the protein universe;BioRxiv [Preprint]. 2019 bioRxiv 626507 [Posted 2019 July 15","DOI":"10.1101\/626507"},{"key":"#cr-split#-pcbi.1009492.ref009.2","unstructured":"cited 2021 July 5]: [28 p.]. Available from: https:\/\/www.biorxiv.org\/content\/10.1101\/626507v4.full.pdf."},{"key":"pcbi.1009492.ref010","doi-asserted-by":"crossref","first-page":"e1008085","DOI":"10.1371\/journal.pcbi.1008085","article-title":"Remote homology search with hidden Potts models","volume":"16","author":"GW Wilburn","year":"2020","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1009492.ref011","doi-asserted-by":"crossref","first-page":"062409","DOI":"10.1103\/PhysRevE.102.062409","article-title":"Aligning biological sequences by exploiting residue conservation and coevolution","volume":"102","author":"AP Muntoni","year":"2020","journal-title":"Phys Rev E"},{"key":"pcbi.1009492.ref012","doi-asserted-by":"crossref","first-page":"1496","DOI":"10.1073\/pnas.1914677117","article-title":"Improved protein structure prediction using predicted interresidue orientations","volume":"117","author":"J Yang","year":"2020","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pcbi.1009492.ref013","doi-asserted-by":"crossref","DOI":"10.1073\/pnas.2016239118","article-title":"Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences","volume":"118","author":"A Rives","year":"2021","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pcbi.1009492.ref014","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1007\/978-1-4684-2001-2_9","volume-title":"Complexity of computer computations","author":"RM Karp","year":"1972"},{"key":"pcbi.1009492.ref015","doi-asserted-by":"crossref","unstructured":"Blelloch GE, Fineman JT, Shun J. Greedy sequential maximal independent set and matching are parallel on average. In: Proceedings of the Twenty-Fourth annual ACM symposium on Parallelism in Algorithms and Architectures; 2012. p. 308\u2013317.","DOI":"10.1145\/2312005.2312058"},{"key":"pcbi.1009492.ref016","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1007\/s00446-010-0121-5","article-title":"An optimal bit complexity randomized distributed MIS algorithm","volume":"23","author":"Y M\u00e9tivier","year":"2011","journal-title":"Distributed Computing"},{"key":"pcbi.1009492.ref017","first-page":"D427","volume":"47","author":"S El-Gebali","year":"2019","journal-title":"The Pfam Protein Families Database in 2019"},{"key":"pcbi.1009492.ref018","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1142\/9781848165632_0019","volume-title":"Genome Informatics 2009: Genome Informatics Series","author":"SR Eddy","year":"2009"},{"key":"pcbi.1009492.ref019","doi-asserted-by":"crossref","first-page":"3389","DOI":"10.1093\/nar\/25.17.3389","article-title":"Gapped BLAST and PSI-BLAST: A New Generation of Protein Database Search Programs","volume":"25","author":"SF Altschul","year":"1997","journal-title":"Nucleic Acids Res"},{"key":"pcbi.1009492.ref020","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1038\/nmeth.3176","article-title":"Fast and sensitive protein alignment using DIAMOND","volume":"12","author":"B Buchfink","year":"2015","journal-title":"Nat Methods"},{"key":"pcbi.1009492.ref021","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1089\/cmb.1998.5.479","article-title":"Homology detection via family pairwise search","volume":"5","author":"WN Grundy","year":"1998","journal-title":"J Comput Biol"},{"key":"pcbi.1009492.ref022","unstructured":"Shen Z, Liu J, Zhang X, Xu R, Yu H, Cui P. Towards Out-of-Distribution Generalization: A Survey. arXiv. 2021;p. https:\/\/arxiv.org\/abs\/2108.13624."}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1009492","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,3,17]],"date-time":"2022-03-17T00:00:00Z","timestamp":1647475200000}},{"DOI":"10.1371\/journal.pcbi.1010971","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2023,3,8]],"date-time":"2023-03-08T00:00:00Z","timestamp":1678233600000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009492","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,3,17]],"date-time":"2022-03-17T14:58:49Z","timestamp":1647529129000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009492"}},"subtitle":[],"editor":[{"given":"Maricel G","family":"Kann","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2022,3,7]]},"references-count":23,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,3,7]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1009492","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.09.29.462285","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,7]]}}}