{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T15:04:47Z","timestamp":1789571087656,"version":"build-2803163510"},"reference-count":66,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T00:00:00Z","timestamp":1789516800000},"content-version":"vor","delay-in-days":15,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Bioinformatics software plays a critical role in clinical applications such as cancer screening and genetic disease diagnosis, where comprehensive quality management is essential for ensuring the accuracy and reliability of downstream analysis. However, current validation practices rely heavily on manually designed simulation experiments, which are labor-intensive and limited in their ability to systematically identify potential quality risks under certain scenarios. In this study, we first construct a benchmark by simulating subtle implementation-level defects in bioinformatics programs. We then propose BioTester, an oracle-based automated testing framework that integrates software testing techniques to support more comprehensive quality assessment of bioinformatics software. BioTester integrates retrieval-augmented LLMs with a differential testing strategy to address the long-standing oracle problem in bioinformatics software testing, demonstrating superior defect-detection performance over existing methods on the constructed benchmark. Finally, applying BioTester to real-world bioinformatics software demonstrates its practical effectiveness and highlights the value of automated testing as a generalizable complement to existing validation practices for improving software reliability in biomedical applications.<\/jats:p>","DOI":"10.1093\/bib\/bbag452","type":"journal-article","created":{"date-parts":[[2026,9,12]],"date-time":"2026-09-12T11:42:16Z","timestamp":1789213336000},"source":"Crossref","is-referenced-by-count":0,"title":["BioTester: an AI-driven automated testing framework for identifying potential quality risks in bioinformatics software"],"prefix":"10.1093","volume":"27","author":[{"given":"Xin","family":"Lian","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Xi'an Jiaotong University , Xi'an 710049, Shaanxi ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiayin","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xi'an Jiaotong University , Xi'an 710049, Shaanxi ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoyan","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xi'an Jiaotong University , Xi'an 710049, Shaanxi ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sizhe","family":"Dang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xi'an Jiaotong University , Xi'an 710049, Shaanxi ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tianxiang","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Xi'an Jiaotong University , Xi'an 710049, Shaanxi ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,9,16]]},"reference":[{"key":"2026091610320149800_ref1","doi-asserted-by":"crossref","first-page":"e1004873","DOI":"10.1371\/journal.pcbi.1004873","article-title":"CNVkit: genome-wide copy number detection and visualization from targeted DNA sequencing","volume":"12","author":"Eric Talevich","year":"2016","journal-title":"PLoS Comput Biol"},{"key":"2026091610320149800_ref2","doi-asserted-by":"crossref","first-page":"974","DOI":"10.1101\/gr.114876.110","article-title":"CNVnator: an approach to discover, genotype, and characterize typical and atypical CNVs from family and population genome sequencing","volume":"21","author":"Abyzov","year":"2011","journal-title":"Genome Res"},{"key":"2026091610320149800_ref3","doi-asserted-by":"crossref","first-page":"gix115","DOI":"10.1093\/gigascience\/gix115","article-title":"CNVcaller: highly efficient and widely applicable software for detecting copy number variations in large populations","volume":"6","author":"Wang","year":"2017","journal-title":"Gigascience"},{"key":"2026091610320149800_ref4","doi-asserted-by":"crossref","first-page":"bbac375","DOI":"10.1093\/bib\/bbac375","article-title":"PEcnv: accurate and efficient detection of copy number variations of various lengths","volume":"23","author":"Wang","year":"2022","journal-title":"Brief Bioinform"},{"key":"2026091610320149800_ref5","doi-asserted-by":"crossref","first-page":"2223","DOI":"10.1093\/bioinformatics\/btt375","article-title":"A comparative analysis of algorithms for somatic SNV detection in cancer","volume":"29","author":"Roberts","year":"2013","journal-title":"Bioinformatics"},{"key":"2026091610320149800_ref6","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1038\/nmeth.3835","article-title":"Monovar: single-nucleotide variant detection in single cells","volume":"13","author":"Zafar","year":"2016","journal-title":"Nat Methods"},{"key":"2026091610320149800_ref7","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1038\/s41586-020-2287-8","article-title":"A structural variation reference for medical and population genetics","volume":"581","author":"Collins","year":"2020","journal-title":"Nature"},{"key":"2026091610320149800_ref8","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1186\/s13059-022-02737-4","article-title":"PerSVade: personalized structural variant detection in any species of interest","volume":"23","author":"Schikora-Tamarit","year":"2022","journal-title":"Genome Biol"},{"key":"2026091610320149800_ref9","doi-asserted-by":"crossref","first-page":"bbae648","DOI":"10.1093\/bib\/bbae648","article-title":"THOR: a TMB heterogeneity-adaptive optimization model predicts immunotherapy response using clonal genomic features in group-structured data","volume":"26","author":"Wang","year":"2025","journal-title":"Brief Bioinform"},{"key":"2026091610320149800_ref10","doi-asserted-by":"crossref","first-page":"1151755","DOI":"10.3389\/fimmu.2023.1151755","article-title":"TMBserval: a statistical explainable learning model reveals weighted tumor mutation burden better categorizing therapeutic benefits","volume":"14","author":"Wang","year":"2023","journal-title":"Front Immunol"},{"key":"2026091610320149800_ref11","doi-asserted-by":"crossref","first-page":"1514295","DOI":"10.3389\/fimmu.2024.1514295","article-title":"TMBocelot: an omnibus statistical control model optimizing the TMB thresholds with systematic measurement errors","volume":"15","author":"Lai","year":"2025","journal-title":"Front Immunol"},{"key":"2026091610320149800_ref12","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1007\/s12551-015-0177-3","article-title":"How to test bioinformatics software?","volume":"7","author":"Kamali","year":"2015","journal-title":"Biophys Rev"},{"key":"2026091610320149800_ref13","doi-asserted-by":"crossref","first-page":"273","DOI":"10.12688\/f1000research.10750.2","article-title":"General guidelines for biomedical software development","volume":"6","author":"Silva","year":"2017","journal-title":"F1000Research"},{"key":"2026091610320149800_ref14","doi-asserted-by":"crossref","first-page":"e0205898","DOI":"10.1371\/journal.pone.0205898","article-title":"A large-scale analysis of bioinformatics code on github","volume":"13","author":"Russell","year":"2018","journal-title":"PLoS One"},{"key":"2026091610320149800_ref15","first-page":"2022","article-title":"Empirical study on software and process quality in bioinformatics tools","author":"Ferenc","year":"2022","journal-title":"bioRxiv"},{"key":"2026091610320149800_ref16","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1109\/TSE.2025.3641225","article-title":"Reaching software quality for bioinformatics applications: how far are we?","volume":"52","author":"Zhu","year":"2026","journal-title":"IEEE Trans Softw Eng"},{"key":"2026091610320149800_ref17","doi-asserted-by":"crossref","first-page":"S15","DOI":"10.1186\/1471-2105-15-S16-S15","article-title":"Verification and validation of bioinformatics software without a gold standard: a case study of BWA and Bowtie","volume":"15","author":"Giannoulatou","year":"2014","journal-title":"BMC Bioinformatics"},{"key":"2026091610320149800_ref18","doi-asserted-by":"crossref","first-page":"e839","DOI":"10.7717\/peerj-cs.839","article-title":"Improving bioinformatics software quality through incorporation of software engineering practices","volume":"8","author":"Noor","year":"2022","journal-title":"PeerJ Comput Sci"},{"key":"2026091610320149800_ref19","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1093\/bioinformatics\/btm566","article-title":"The SBML discrete stochastic models test suite","volume":"24","author":"Evans","year":"2008","journal-title":"Bioinformatics"},{"key":"2026091610320149800_ref20","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1093\/bib\/bbn050","article-title":"Biochemical simulations: stochastic, approximate stochastic and hybrid approaches","volume":"10","author":"Pahle","year":"2009","journal-title":"Brief Bioinform"},{"key":"2026091610320149800_ref21","first-page":"1","article-title":"Metamorphic testing: a review of challenges and opportunities","volume":"51","author":"Chen","year":"2018","journal-title":"ACM Comput Surv"},{"key":"2026091610320149800_ref22","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1186\/1471-2105-10-24","article-title":"An innovative approach for testing bioinformatics programs using metamorphic testing","volume":"10","author":"Chen","year":"2009","journal-title":"BMC Bioinformatics"},{"key":"2026091610320149800_ref23","doi-asserted-by":"crossref","DOI":"10.1145\/2896971.2896975","article-title":"A cloud-based framework for applying metamorphic testing to a bioinformatics pipeline","volume-title":"Proceedings of the 1st International Workshop on Metamorphic Testing","author":"Troup"},{"key":"2026091610320149800_ref24","article-title":"Metamorphic testing for quality assurance of protein function prediction tools","volume-title":"2019 IEEE International Conference On Artificial Intelligence Testing (AITest)","author":"Shahri"},{"key":"2026091610320149800_ref25","doi-asserted-by":"crossref","DOI":"10.1145\/2931037.2931061","article-title":"Automatic generation of oracles for exceptional behaviors","volume-title":"Proceedings of the 25th international symposium on software testing and analysis","author":"Goffi"},{"key":"2026091610320149800_ref26","doi-asserted-by":"crossref","DOI":"10.1109\/ASE56229.2023.00089","article-title":"Nuances are the key: unlocking ChatGPT to find failure-inducing tests with differential prompting","volume-title":"2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Li"},{"key":"2026091610320149800_ref27","doi-asserted-by":"crossref","first-page":"107565","DOI":"10.1016\/j.infsof.2024.107565","article-title":"A3Test: assertion-augmented automated test case generation","volume":"176","author":"Alagarsamy","year":"2024","journal-title":"Inf Softw Technol"},{"key":"2026091610320149800_ref28","author":"","journal-title":"Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics"},{"key":"2026091610320149800_ref29","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv Neural Inf Proces Syst"},{"key":"2026091610320149800_ref30","article-title":"Fast and accurate alignment of long bisulfite-seq reads","author":"Pedersen"},{"key":"2026091610320149800_ref31","doi-asserted-by":"crossref","first-page":"giae020","DOI":"10.1093\/gigascience\/giae020","article-title":"Hecatomb: an integrated software platform for viral metagenomics","volume":"13","author":"Roach","year":"2024","journal-title":"GigaScience"},{"key":"2026091610320149800_ref32","doi-asserted-by":"crossref","first-page":"1103","DOI":"10.1038\/s41592-020-00971-x","article-title":"metaFlye: scalable long-read metagenome assembly using repeat graphs","volume":"17","author":"Kolmogorov","year":"2020","journal-title":"Nat Methods"},{"key":"2026091610320149800_ref33","doi-asserted-by":"crossref","first-page":"btaf592","DOI":"10.1093\/bioinformatics\/btaf592","article-title":"ZIPcnv: accurate and efficient inference of copy number variations from shallow whole-genome sequencing","volume":"41","author":"Xue","year":"2025","journal-title":"Bioinformatics"},{"key":"2026091610320149800_ref34","doi-asserted-by":"crossref","first-page":"e15","DOI":"10.1093\/nar\/gku1196","article-title":"Rapid phylogenetic analysis of large samples of recombinant bacterial whole genome sequences using Gubbins","volume":"43","author":"Croucher","year":"2015","journal-title":"Nucleic Acids Res"},{"key":"2026091610320149800_ref35","doi-asserted-by":"crossref","first-page":"btac757","DOI":"10.1093\/bioinformatics\/btac757","article-title":"GSEApy: a comprehensive package for performing gene set enrichment analysis in Python","volume":"39","author":"Fang","year":"2023","journal-title":"Bioinformatics"},{"key":"2026091610320149800_ref36","doi-asserted-by":"crossref","first-page":"4177","DOI":"10.1038\/s41596-021-00573-7","article-title":"NovoSpaRc: flexible spatial reconstruction of single-cell gene expression with optimal transport","volume":"16","author":"Moriel","year":"2021","journal-title":"Nat Protoc"},{"key":"2026091610320149800_ref37","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1109\/TSE.2010.62","article-title":"An analysis and survey of the development of mutation testing","volume":"37","author":"Jia","year":"2010","journal-title":"IEEE Trans Softw Eng"},{"key":"2026091610320149800_ref38","doi-asserted-by":"crossref","DOI":"10.1109\/DSA63982.2024.00030","article-title":"An analysis and comparison of mutation testing tools for Python","volume-title":"2024 11th International Conference on Dependable Systems and Their Applications (DSA)","author":"Diallo"},{"key":"2026091610320149800_ref39","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2024.findings-acl.943","article-title":"Retrieval-augmented retrieval: large language models are strong zero-shot retriever","volume-title":"Findings of the Association for Computational Linguistics ACL","author":"Shen","year":"2024"},{"key":"2026091610320149800_ref40","article-title":"Retrieve anything to augment large language models","author":"Zhang"},{"key":"2026091610320149800_ref41","article-title":"The Faiss library","author":"Douze","year":"2024"},{"key":"2026091610320149800_ref42","article-title":"CodeT5: identifier-aware unified pre-trained encoder-decoder models for code understanding and generation","author":"Wang","journal-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing"},{"key":"2026091610320149800_ref43","article-title":"InferCNV of the trinity CTAT project","author":"Tickle","year":"2019"},{"key":"2026091610320149800_ref44","doi-asserted-by":"crossref","first-page":"giab074","DOI":"10.1093\/gigascience\/giab074","article-title":"CNVpytor: a tool for copy number variation detection and analysis from read depth and allele imbalance in whole-genome sequencing","volume":"10","author":"Suvakov","year":"2021","journal-title":"Gigascience"},{"key":"2026091610320149800_ref45","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/s13059-017-1382-0","article-title":"SCANPY: large-scale single-cell gene expression data analysis","volume":"19","author":"Alexander Wolf","year":"2018","journal-title":"Genome Biol"},{"key":"2026091610320149800_ref46","article-title":"CNV: copy number variant detection from targeted DNA sequencing","author":"Talevich","year":"2016"},{"key":"2026091610320149800_ref47","article-title":"CNVpytor: python extension of CNVnator for CNV detection","author":"Abyzov"},{"key":"2026091610320149800_ref48","article-title":"InferCNV of the trinity CTAT project","author":"Broad Institute"},{"key":"2026091610320149800_ref49","article-title":"SCANPY: single-cell analysis in Python","author":"Wolf","year":"2018"},{"key":"2026091610320149800_ref50","article-title":"CNVkit documentation","author":"Talevich"},{"key":"2026091610320149800_ref51","article-title":"SCANPY documentation","author":"Alexander Wolf"},{"key":"2026091610320149800_ref52","article-title":"InferCNVpy documentation","author":"Zhang"},{"key":"2026091610320149800_ref53","article-title":"CNVpytor documentation","author":"Abyzov"},{"key":"2026091610320149800_ref54","article-title":"Program synthesis with large language models","author":"Austin"},{"key":"2026091610320149800_ref55","article-title":"CodeSearchNet challenge: evaluating the state of semantic code search","author":"Husain"},{"key":"2026091610320149800_ref56","volume":"645","author":"","journal-title":"Nature"},{"key":"2026091610320149800_ref57","first-page":"3","article-title":"LoRA: Low-rank adaptation of large language models","volume":"1","author":"Hu","year":"2022","journal-title":"ICLR"},{"key":"2026091610320149800_ref58","article-title":"GPT-4o system card","author":"Hurst"},{"key":"2026091610320149800_ref59","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1109\/TSE.2023.3334955","article-title":"An empirical evaluation of using large language models for automated unit test generation","volume":"50","author":"Sch\u00e4fer","year":"2023","journal-title":"IEEE Trans Softw Eng"},{"key":"2026091610320149800_ref60","author":"","journal-title":"Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering"},{"key":"2026091610320149800_ref61","author":"","journal-title":"Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE)"},{"key":"2026091610320149800_ref62","doi-asserted-by":"crossref","DOI":"10.1109\/ASE56229.2023.00181","article-title":"An empirical study on fine-tuning large language models of code for automated program repair","volume-title":"2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Huang"},{"key":"2026091610320149800_ref63","doi-asserted-by":"crossref","DOI":"10.1109\/ICSE48619.2023.00129","article-title":"Automated program repair in the era of large pre-trained language models","volume-title":"2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE)","author":"Xia"},{"key":"2026091610320149800_ref64","doi-asserted-by":"crossref","first-page":"e1014322","DOI":"10.1371\/journal.pcbi.1014322","article-title":"Energy transfer leaves fingerprints in cyanine photoswitching behavior","volume":"22","author":"Ebert","year":"2026","journal-title":"PLoS Comput Biol"},{"key":"2026091610320149800_ref65","volume":"22","author":"","journal-title":"PLoS Comput Biol"},{"key":"2026091610320149800_ref66","volume":"22","author":"","journal-title":"PLoS Comput Biol"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/27\/5\/bbag452\/71123541\/bbag452.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/27\/5\/bbag452\/71123541\/bbag452.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T14:32:14Z","timestamp":1789569134000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbag452\/8802170"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,1]]},"references-count":66,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbag452","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,9]]},"published":{"date-parts":[[2026,9,1]]},"article-number":"bbag452"}}