{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T13:48:46Z","timestamp":1782308926462,"version":"3.54.5"},"reference-count":29,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T00:00:00Z","timestamp":1780444800000},"content-version":"vor","delay-in-days":2,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2021YFF0704300"],"award-info":[{"award-number":["2021YFF0704300"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["W2511070"],"award-info":[{"award-number":["W2511070"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["624B2089"],"award-info":[{"award-number":["624B2089"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["32241027"],"award-info":[{"award-number":["32241027"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62472034"],"award-info":[{"award-number":["62472034"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["12501589"],"award-info":[{"award-number":["12501589"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Foundation of Ningxia Province","award":["2026AAC040016"],"award-info":[{"award-number":["2026AAC040016"]}]},{"name":"Foundation of Ningxia Province","award":["2024BEH04022"],"award-info":[{"award-number":["2024BEH04022"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>With the advancement of cryo-electron microscopy (cryo-EM) into the atomic resolution era, accurate C\u03b1 atom modeling has become essential for macromolecular structure determination. However, existing evaluation systems overly rely on full-atom metrics and lack a dedicated, comprehensive benchmark for assessing C\u03b1 prediction modules within automated modeling tools.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>To address this gap, we establish a rigorous benchmark to evaluate the C\u03b1 prediction performance of four prominent deep learning-based methods (ModelAngelo, DeepMainMast, EModelX, and CryoAtom) across multiple dimensions. We construct a diverse dataset covering a wide range of resolutions (1\u20138\u2009\u00c5), molecular weights, and noise levels. A novel evaluation framework is introduced, incorporating multi-threshold RMSD-based metrics (1\u20133\u2009\u00c5) alongside advanced point-cloud similarity measures (Chamfer Distance, Earth Mover\u2019s Distance) for quantitative and nuanced assessment. Our results reveal that method performance is highly dependent on the chosen evaluation criteria and intrinsic data characteristics. ModelAngelo excels under loose thresholds with high-quality data but shows sensitivity to resolution degradation; CryoAtom demonstrates notable computational efficiency, however, its completeness-oriented design leads to a certain loss of precision; EModelX demonstrates balanced generalization across varied conditions; DeepMainMast achieves high localization accuracy under stringent criteria but incurs a high computational cost.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>This work provides a reproducible, C\u03b1-centric evaluation framework to guide method development and advance automated cryo-EM structure determination. The source code for the benchmark and evaluation metrics is freely available at https:\/\/github.com\/zhtianz\/Benchmarking\\_CA.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btag350","type":"journal-article","created":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T11:43:44Z","timestamp":1779882224000},"source":"Crossref","is-referenced-by-count":0,"title":["Benchmarking deep learning methods for C\n                    <i>\u03b1<\/i>\n                    atom prediction in cryo-EM density maps"],"prefix":"10.1093","volume":"42","author":[{"given":"Tian","family":"Zhang","sequence":"first","affiliation":[{"name":"Research Center for Mathematics and Interdisciplinary Sciences; Cheeloo College of Medicine, Qilu Hospital (Qingdao), Shandong University , Qingdao 266237,","place":["China"]},{"name":"College of Medical Information and Engineering, Ningxia Medical University , Yinchuan 750004,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhe","family":"Liu","sequence":"additional","affiliation":[{"name":"Research Center for Mathematics and Interdisciplinary Sciences; Cheeloo College of Medicine, Qilu Hospital (Qingdao), Shandong University , Qingdao 266237,","place":["China"]},{"name":"College of Medical Information and Engineering, Ningxia Medical University , Yinchuan 750004,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yiqing","family":"Ma","sequence":"additional","affiliation":[{"name":"Research Center for Mathematics and Interdisciplinary Sciences; Cheeloo College of Medicine, Qilu Hospital (Qingdao), Shandong University , Qingdao 266237,","place":["China"]},{"name":"College of Medical Information and Engineering, Ningxia Medical University , Yinchuan 750004,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chenjie","family":"Feng","sequence":"additional","affiliation":[{"name":"College of Medical Information and Engineering, Ningxia Medical University , Yinchuan 750004,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4761-6526","authenticated-orcid":false,"given":"Renmin","family":"Han","sequence":"additional","affiliation":[{"name":"Research Center for Mathematics and Interdisciplinary Sciences; Cheeloo College of Medicine, Qilu Hospital (Qingdao), Shandong University , Qingdao 266237,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,6,2]]},"reference":[{"key":"2026062409123210000_btag350-B1","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1016\/j.tibs.2014.10.005","article-title":"How cryo-em is revolutionizing structural biology","volume":"40","author":"Bai","year":"2015","journal-title":"Trends Biochem Sci"},{"key":"2026062409123210000_btag350-B2","doi-asserted-by":"publisher","first-page":"109168","DOI":"10.1016\/j.patcog.2022.109168","article-title":"Rethinking local and global feature representation for dense prediction","volume":"135","author":"Chen","year":"2023","journal-title":"Pattern Recognit"},{"key":"2026062409123210000_btag350-B3","doi-asserted-by":"publisher","first-page":"8808","DOI":"10.1038\/s41467-024-53116-5","article-title":"Protein complex structure modeling by cross-modal alignment between cryo-em maps and protein sequences","volume":"15","author":"Chen","year":"2024","journal-title":"Nat Commun"},{"key":"2026062409123210000_btag350-B4","doi-asserted-by":"publisher","first-page":"e01597\u201318","DOI":"10.1128\/JVI.01597-18","article-title":"A 2.8-angstrom-resolution cryo-electron microscopy structure of human parechovirus 3 in complex with fab from a neutralizing antibody","volume":"93","author":"Domanska","year":"2019","journal-title":"J Virol"},{"key":"2026062409123210000_btag350-B5","doi-asserted-by":"publisher","first-page":"652","DOI":"10.1109\/TPAMI.2019.2938758","article-title":"Res2net: a new multi-scale backbone architecture","volume":"43","author":"Gao","year":"2021","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2026062409123210000_btag350-B6","doi-asserted-by":"publisher","first-page":"132","DOI":"10.3390\/biom13010132","article-title":"Improving protein\u2013ligand interaction modeling with cryo-em data, templates, and deep learning in 2021 ligand model challenge","volume":"13","author":"Giri","year":"2023","journal-title":"Biomolecules"},{"key":"2026062409123210000_btag350-B7","doi-asserted-by":"publisher","first-page":"5511","DOI":"10.1038\/s41467-024-49647-6","article-title":"De novo atomic protein structure modeling for cryoem density maps using 3D transformer and HMM","volume":"15","author":"Giri","year":"2024","journal-title":"Nat Commun"},{"key":"2026062409123210000_btag350-B8","first-page":"770","author":"He","year":"2016"},{"key":"2026062409123210000_btag350-B9","doi-asserted-by":"publisher","first-page":"1055","DOI":"10.1109\/ICASSP40776.2020.9053405","author":"Huang","year":"2020"},{"key":"2026062409123210000_btag350-B10","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1038\/s41586-024-07215-4","article-title":"Automated model building and protein identification in cryo-em maps","volume":"628","author":"Jamali","year":"2024","journal-title":"Nature"},{"key":"2026062409123210000_btag350-B11","doi-asserted-by":"publisher","first-page":"1443","DOI":"10.1126\/science.1251652","article-title":"Biochemistry. the resolution revolution","volume":"343","author":"K\u00fchlbrandt","year":"2014","journal-title":"Science"},{"key":"2026062409123210000_btag350-B12","doi-asserted-by":"publisher","author":"Lee","year":"2017","DOI":"10.48550\/arXiv.1706.00120"},{"key":"2026062409123210000_btag350-B13","first-page":"2117","author":"Lin","year":"2017"},{"key":"2026062409123210000_btag350-B14","doi-asserted-by":"publisher","first-page":"545","DOI":"10.1016\/j.str.2019.07.018","article-title":"Collection of continuous rotation microed data from ion beam-milled crystals of any size","volume":"27","author":"Martynowycz","year":"2019","journal-title":"Structure"},{"key":"2026062409123210000_btag350-B15","doi-asserted-by":"publisher","first-page":"1698","DOI":"10.1016\/j.cell.2016.05.040","article-title":"Breaking cryo-EM resolution barriers to facilitate drug discovery","volume":"165","author":"Merk","year":"2016","journal-title":"Cell"},{"key":"2026062409123210000_btag350-B16","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1038\/s41586-020-2829-5","article-title":"Single-particle cryo-EM at atomic resolution","volume":"587","author":"Nakane","year":"2020","journal-title":"Nature"},{"key":"2026062409123210000_btag350-B17","doi-asserted-by":"publisher","first-page":"677","DOI":"10.1016\/j.molcel.2015.02.019","article-title":"Cryo-em: a unique tool for the visualization of macromolecular complexity","volume":"58","author":"Nogales","year":"2015","journal-title":"Mol Cell"},{"key":"2026062409123210000_btag350-B18","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1038\/s41592-020-0731-1","article-title":"Measurement of atom resolvability in cryo-EM maps with q-scores","volume":"17","author":"Pintilie","year":"2020","journal-title":"Nat Methods"},{"key":"2026062409123210000_btag350-B19","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1007\/978-3-319-24574-4_28","volume-title":"Medical Image Computing and Computer-Assisted Intervention \u2013 MICCAI 2015, Volume 9351 of Lecture Notes in Computer Science","author":"Ronneberger","year":"2015"},{"key":"2026062409123210000_btag350-B20","doi-asserted-by":"publisher","first-page":"890","DOI":"10.1016\/j.drudis.2013.10.027","article-title":"Modelling three-dimensional protein structures for applications in drug design","volume":"19","author":"Schmidt","year":"2014","journal-title":"Drug Discov Today"},{"key":"2026062409123210000_btag350-B21","doi-asserted-by":"publisher","first-page":"4282","DOI":"10.1038\/s41598-020-60598-y","article-title":"Deep learning to predict protein backbone structure from high-resolution cryo-em density maps","volume":"10","author":"Si","year":"2020","journal-title":"Sci Rep"},{"key":"2026062409123210000_btag350-B22","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1038\/s41594-025-01713-3","article-title":"Cryoatom improves model building for cryo-em","volume":"33","author":"Su","year":"2026","journal-title":"Nat Struct Mol Biol"},{"key":"2026062409123210000_btag350-B23","doi-asserted-by":"publisher","first-page":"e2102592","DOI":"10.1002\/advs.202102592","article-title":"Improved protein structure prediction using a new multi-scale network and homologous templates","volume":"8","author":"Su","year":"2021","journal-title":"Adv Sci (Weinh)"},{"key":"2026062409123210000_btag350-B24","doi-asserted-by":"publisher","first-page":"1618","DOI":"10.1038\/s41467-018-04053-7","article-title":"De novo main-chain modeling for EM maps using MAINMAST","volume":"9","author":"Terashi","year":"2018","journal-title":"Nat Commun"},{"key":"2026062409123210000_btag350-B25","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1038\/s41592-023-02099-0","article-title":"Deepmainmast: integrated protocol of protein structure modeling for cryo-em with deep learning and structure prediction","volume":"21","author":"Terashi","year":"2024","journal-title":"Nat Methods"},{"key":"2026062409123210000_btag350-B26","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1152\/physiol.00041.2005","article-title":"Cryo-em and single particles","volume":"21","author":"Wang","year":"2006","journal-title":"Physiology (Bethesda)"},{"key":"2026062409123210000_btag350-B27","doi-asserted-by":"publisher","first-page":"1739","DOI":"10.1038\/s41592-023-02038-z","article-title":"CryoREAD: de novo structure modeling for nucleic acids in cryo-EM maps using deep learning","volume":"20","author":"Wang","year":"2023","journal-title":"Nat Methods"},{"key":"2026062409123210000_btag350-B28","first-page":"2117"},{"key":"2026062409123210000_btag350-B29","doi-asserted-by":"publisher","first-page":"4985","DOI":"10.1038\/s41467-018-07489-z","article-title":"Structures of coxsackievirus a10 unveil the molecular mechanisms of receptor binding and viral uncoating","volume":"9","author":"Zhu","year":"2018","journal-title":"Nat Commun"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btag350\/68459215\/btag350.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/42\/6\/btag350\/68459215\/btag350.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/42\/6\/btag350\/68459215\/btag350.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T13:12:38Z","timestamp":1782306758000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btag350\/8700507"}},"subtitle":[],"editor":[{"given":"Lenore","family":"Cowen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,6,1]]},"references-count":29,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2026,6,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btag350","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,6]]},"published":{"date-parts":[[2026,6,1]]},"article-number":"btag350"}}