{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T13:09:51Z","timestamp":1789650591728,"version":"4.0.0"},"reference-count":45,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T00:00:00Z","timestamp":1789603200000},"content-version":"vor","delay-in-days":16,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["72274152"],"award-info":[{"award-number":["72274152"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62402376"],"award-info":[{"award-number":["62402376"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62572389"],"award-info":[{"award-number":["62572389"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62602511"],"award-info":[{"award-number":["62602511"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Natural Science Basic Research Program of Shaanxi","award":["2026JC-JCQN-122"],"award-info":[{"award-number":["2026JC-JCQN-122"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>As bioinformatics software is increasingly applied across a broader range of scenarios and the rapid development of large language models (LLMs) further lowers the barriers to software use and development, the composition of the bioinformatics research community is undergoing substantial change. Consequently, a growing number of researchers require a deeper understanding of methodological details and software behavior. In this context, systematically analyzing the relationship between paper descriptions and code implementations is emerging as an important new challenge in the field. To address this challenge, we introduce paper-code consistency analysis as a new research perspective and construct BioCon, the first benchmark dataset for paper-code consistency analysis in bioinformatics. Furthermore, we develop a unified cross-modal analysis framework to systematically investigate this problem from three perspectives: sentence-level detection, cross-modal retrieval, and project-level assessment. Experimental results demonstrate that the proposed framework can effectively model the semantic relationships between scientific publications and software implementations. Further case studies reveal that paper-code inconsistency is not a single phenomenon but arises from multiple underlying causes, among which Author-Perceived Non-Essential Details represents the most prevalent category. These findings suggest that paper-code consistency analysis is not merely a technical problem but also raises broader discussions regarding knowledge dissemination, the boundaries of code disclosure, and community norms. We hope that this work will encourage the bioinformatics community to re-examine the relationship between scientific publications and software implementations while providing a foundation for future research in paper-code consistency analysis.<\/jats:p>","DOI":"10.1093\/bib\/bbag509","type":"journal-article","created":{"date-parts":[[2026,9,4]],"date-time":"2026-09-04T11:19:45Z","timestamp":1788520785000},"source":"Crossref","is-referenced-by-count":0,"title":["Do papers tell the whole story? A benchmark and framework for uncovering hidden implementation gaps in bioinformatics"],"prefix":"10.1093","volume":"27","author":[{"given":"Tianxiang","family":"Xu","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]},{"name":"Shaanxi Engineering Research Center of Medical and Health Big Data, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoyan","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]},{"name":"Shaanxi Engineering Research Center of Medical and Health Big Data, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xin","family":"Lai","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]},{"name":"Shaanxi Engineering Research Center of Medical and Health Big Data, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xin","family":"Lian","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]},{"name":"Shaanxi Engineering Research Center of Medical and Health Big Data, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sizhe","family":"Dang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]},{"name":"Shaanxi Engineering Research Center of Medical and Health Big Data, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hangyu","family":"Cheng","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]},{"name":"Shaanxi Engineering Research Center of Medical and Health Big Data, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3862-6557","authenticated-orcid":false,"given":"Jiayin","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]},{"name":"Shaanxi Engineering Research Center of Medical and Health Big Data, Xi\u2019an Jiaotong University , 710049, Xi\u2019an ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,9,17]]},"reference":[{"key":"2026091708300284900_ref1","doi-asserted-by":"crossref","first-page":"100083","DOI":"10.59717\/j.xinn-life.2024.100083","article-title":"Bioinformatics software development: principles and future directions","volume":"2","author":"Ma","year":"2024","journal-title":"Innov Life"},{"key":"2026091708300284900_ref2","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1093\/bib\/bbx100","article-title":"A global perspective on evolving bioinformatics and data science training needs","volume":"20","author":"Attwood","year":"2019","journal-title":"Brief Bioinform"},{"key":"2026091708300284900_ref3","article-title":"Bioinformatics in the age of big data: Leveraging computational tools for biological discoveries","volume":"14","author":"Liu","year":"2024","journal-title":"Comput Mol Biol"},{"key":"2026091708300284900_ref4","doi-asserted-by":"crossref","first-page":"e2066","DOI":"10.7717\/peerj-cs.2066","article-title":"Analytical code sharing practices in biomedical research","volume":"10","author":"Sharma","year":"2024","journal-title":"PeerJ Comput Sci"},{"key":"2026091708300284900_ref5","doi-asserted-by":"crossref","first-page":"e70243","DOI":"10.1002\/mco2.70243","article-title":"Current bioinformatics tools in precision oncology","volume":"6","author":"Wolde","year":"2025","journal-title":"MedComm"},{"key":"2026091708300284900_ref6","doi-asserted-by":"crossref","first-page":"bbaf329","DOI":"10.1093\/bib\/bbaf329","article-title":"Advancing genome-based precision medicine: a review on machine learning applications for rare genetic disorders","volume":"26","author":"Abbas","year":"2025","journal-title":"Brief Bioinform"},{"key":"2026091708300284900_ref7","doi-asserted-by":"crossref","first-page":"1666","DOI":"10.1038\/s41591-021-01533-0","article-title":"Alphafold heralds a data-driven revolution in biology and medicine","volume":"27","author":"Thornton","year":"2021","journal-title":"Nat Med"},{"key":"2026091708300284900_ref8","doi-asserted-by":"crossref","first-page":"btae464","DOI":"10.1093\/bioinformatics\/btae464","article-title":"Fairsoft\u2014a practical implementation of fair principles for research software","volume":"40","author":"Mart\u00edn","year":"2024","journal-title":"Bioinformatics"},{"key":"2026091708300284900_ref9","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-023-10396-7","article-title":"An empirical study on software understandability and its dependence on code characteristics","volume":"28","author":"Lavazza","year":"2023","journal-title":"Empirical Softw Eng"},{"key":"2026091708300284900_ref10","doi-asserted-by":"crossref","first-page":"bbad375","DOI":"10.1093\/bib\/bbad375","article-title":"The five pillars of computational reproducibility: bioinformatics and beyond","volume":"24","author":"Ziemann","year":"2023","journal-title":"Brief Bioinform"},{"key":"2026091708300284900_ref11","doi-asserted-by":"crossref","first-page":"1381","DOI":"10.1093\/comjnl\/bxad067","article-title":"Improving science that uses code","volume":"67","author":"Thimbleby","year":"2024","journal-title":"Comput J"},{"key":"2026091708300284900_ref12","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","article-title":"Highly accurate protein structure prediction with alphafold","volume":"596","author":"Jumper","year":"2021","journal-title":"Nature"},{"key":"2026091708300284900_ref13","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1038\/s41587-023-01767-y","article-title":"Dictionary learning for integrative, multimodal and scalable single-cell analysis","volume":"42","author":"Hao","year":"2024","journal-title":"Nat Biotechnol"},{"key":"2026091708300284900_ref14","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1093\/bioinformatics\/bts635","article-title":"Star: ultrafast universal rna-seq aligner","volume":"29","author":"Dobin","year":"2013","journal-title":"Bioinformatics"},{"key":"2026091708300284900_ref15","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1186\/s13059-024-03343-2","article-title":"Genomic reproducibility in the bioinformatics era","volume":"25","author":"Baykal","year":"2024","journal-title":"Genome Biol"},{"key":"2026091708300284900_ref16","article-title":"Code representation learning at scale","author":"Zhang","year":"2024","journal-title":"The Twelfth International Conference on Learning Representations"},{"key":"2026091708300284900_ref17","doi-asserted-by":"publisher","first-page":"3088","DOI":"10.1109\/TSE.2025.3609876","article-title":"An empirical study of exploring the capabilities of large language models in code learning","volume":"51","author":"Liu","year":"2025","journal-title":"IEEE Trans Softw Eng"},{"key":"2026091708300284900_ref18","article-title":"Code representation pre-training with complements from program executions","volume-title":"Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track","author":"Huang","year":"2024"},{"key":"2026091708300284900_ref19","doi-asserted-by":"crossref","first-page":"100804","DOI":"10.1016\/j.patter.2023.100804","article-title":"Leakage and the reproducibility crisis in machine-learning-based science","volume":"4","author":"Kapoor","year":"2023","journal-title":"Patterns"},{"key":"2026091708300284900_ref20","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1038\/s41597-022-01143-6","article-title":"A large-scale study on research code quality and execution","volume":"9","author":"Trisovic","year":"2022","journal-title":"Sci Data"},{"key":"2026091708300284900_ref21","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2512.00651","article-title":"Large language models for software engineering: a reproducibility crisis","author":"Siddiq","year":"2025"},{"key":"2026091708300284900_ref22","doi-asserted-by":"crossref","first-page":"141102","DOI":"10.1007\/s11432-025-4670-0","article-title":"A survey on large language models for software engineering","volume":"69","author":"Zhang","year":"2026","journal-title":"Sci China Inf Sci"},{"key":"2026091708300284900_ref23","doi-asserted-by":"publisher","DOI":"10.1145\/3821535","article-title":"Pseudobridge: pseudo code as the bridge for better semantic and logic alignment in code retrieval","author":"Li","journal-title":"ACM Trans Softw Eng Methodol"},{"key":"2026091708300284900_ref24","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1909.09436","article-title":"Codesearchnet challenge: evaluating the state of semantic code search","author":"Husain","year":"2019"},{"key":"2026091708300284900_ref25","doi-asserted-by":"publisher","DOI":"10.1145\/3817056","article-title":"Towards realistic project-level code generation via multi-agent collaboration and semantic architecture modeling","author":"Zhao","journal-title":"ACM Trans Softw Eng Methodol"},{"key":"2026091708300284900_ref26","doi-asserted-by":"crossref","first-page":"7443","DOI":"10.18653\/v1\/2023.acl-long.411","article-title":"Large language models meet NL2Code: a survey","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Zan","year":"2023"},{"key":"2026091708300284900_ref27","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2025.3641225","article-title":"Reaching software quality for bioinformatics applications: how far are we","volume":"52","author":"Zhu","journal-title":"IEEE Trans Softw Eng"},{"key":"2026091708300284900_ref28","doi-asserted-by":"publisher","DOI":"10.1101\/2022.03.10.483804","article-title":"Empirical study on software and process quality in bioinformatics tools","author":"Ferenc","year":"2022"},{"key":"2026091708300284900_ref29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pcbi.1011920","article-title":"For long-term sustainable software in bioinformatics","volume":"20","author":"Coelho","year":"2024","journal-title":"PLoS Comput Biol"},{"key":"2026091708300284900_ref30","doi-asserted-by":"crossref","first-page":"7212","DOI":"10.18653\/v1\/2022.acl-long.499","article-title":"Unixcoder: unified cross-modal pre-training for code representation","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Guo","year":"2022"},{"key":"2026091708300284900_ref31","doi-asserted-by":"crossref","first-page":"1069","DOI":"10.18653\/v1\/2023.emnlp-main.68","article-title":"Codet5+: open code large language models for code understanding and generation","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Wang","year":"2023"},{"key":"2026091708300284900_ref32","first-page":"1536","article-title":"Codebert: a pre-trained model for programming and natural languages","volume":"2020","author":"Feng","journal-title":"Find Assoc Comput Linguist: EMNLP 2020"},{"key":"2026091708300284900_ref33","first-page":"2980","article-title":"Focal loss for dense object detection","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Lin","year":"2017"},{"key":"2026091708300284900_ref34","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2505.09388","article-title":"Qwen3 technical report","author":"Yang","year":"2025"},{"key":"2026091708300284900_ref35","article-title":"Codegen: an open large language model for code with multi-turn program synthesis","author":"Nijkamp","journal-title":"The Eleventh International Conference on Learning Representations"},{"key":"2026091708300284900_ref36","first-page":"38","article-title":"Transformers: state-of-the-art natural language processing","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations","author":"Wolf","year":"2020"},{"key":"2026091708300284900_ref37","doi-asserted-by":"crossref","first-page":"78368","DOI":"10.1109\/ACCESS.2021.3084050","article-title":"The Matthews correlation coefficient (mcc) is more informative than cohen\u2019s kappa and brier score in binary classification assessment","volume":"9","author":"Chicco","year":"2021","journal-title":"IEEE Access"},{"key":"2026091708300284900_ref38","first-page":"1321","article-title":"On calibration of modern neural networks","volume-title":"International Conference on Machine Learning","author":"Guo","year":"2017"},{"key":"2026091708300284900_ref39","doi-asserted-by":"crossref","first-page":"btag122","DOI":"10.1093\/bioinformatics\/btag122","article-title":"Scalable analysis of whole slide spatial proteomics with harpy","volume":"42","author":"Rombaut","year":"2026","journal-title":"Bioinformatics"},{"key":"2026091708300284900_ref40","doi-asserted-by":"crossref","first-page":"btag073","DOI":"10.1093\/bioinformatics\/btag073","article-title":"Insitupy: a framework for histology-guided, multi-sample analysis of single-cell spatial omics data","volume":"42","author":"Wirth","year":"2026","journal-title":"Bioinformatics"},{"key":"2026091708300284900_ref41","doi-asserted-by":"crossref","first-page":"btag067","DOI":"10.1093\/bioinformatics\/btag067","article-title":"Prediction of bacterial protein\u2013compound interactions with only positive samples","volume":"42","author":"Kim","year":"2026","journal-title":"Bioinformatics"},{"key":"2026091708300284900_ref42","doi-asserted-by":"crossref","first-page":"btag088","DOI":"10.1093\/bioinformatics\/btag088","article-title":"A deep learning framework for comprehensive prediction of human rna g-quadruplex-binding proteins","volume":"42","author":"Rosignoli","year":"2026","journal-title":"Bioinformatics"},{"key":"2026091708300284900_ref43","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1186\/s12916-020-01590-1","article-title":"Evidence of unexplained discrepancies between planned and conducted statistical analyses: a review of randomised trials","volume":"18","author":"Cro","year":"2020","journal-title":"BMC Med"},{"key":"2026091708300284900_ref44","doi-asserted-by":"crossref","first-page":"btag116","DOI":"10.1093\/bioinformatics\/btag116","article-title":"Terminatornet: comprehensive identification of intrinsic transcription terminators in bacteria","volume":"42","author":"Tjaden","year":"2026","journal-title":"Bioinformatics"},{"key":"2026091708300284900_ref45","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-026-03987-2","article-title":"Self-supervised pretraining with nuspire unlocks nuclear morphology-driven insights in spatial omics","volume":"27","author":"Hua","year":"2026","journal-title":"Genome Biol"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/27\/5\/bbag509\/71188535\/bbag509.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/27\/5\/bbag509\/71188535\/bbag509.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T12:30:14Z","timestamp":1789648214000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbag509\/8812210"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,1]]},"references-count":45,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbag509","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,9]]},"published":{"date-parts":[[2026,9,1]]},"article-number":"bbag509"}}