{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T23:14:08Z","timestamp":1787008448757,"version":"build-2736575974"},"reference-count":87,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2025,2,6]],"date-time":"2025-02-06T00:00:00Z","timestamp":1738800000000},"content-version":"vor","delay-in-days":76,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,11,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Large language models (LLMs) leverage factual knowledge from pretraining. Yet this knowledge remains incomplete and sometimes challenging to retrieve\u2014especially in scientific domains not extensively covered in pretraining datasets and where information is still evolving. Here, we focus on genomics and bioinformatics. We confirm and expand upon issues with plain ChatGPT functioning as a bioinformatics assistant. Poor data retrieval and hallucination lead ChatGPT to err, as do incorrect sequence manipulations. To address this, we propose a system basing LLM outputs on up-to-date, authoritative facts and facilitating LLM-guided data analysis. Specifically, we introduce NagGPT, a middleware tool to insert between LLMs and databases, designed to bridge gaps in LLM knowledge and usage of database application programming interfaces. NagGPT proxies LLM-generated database queries, with special handling of incorrect queries. It acts as a gatekeeper between query responses and the LLM prompt, redirecting large responses to files but providing a synthesized snippet and injecting comments to steer the LLM. A companion OpenAI custom GPT, Genomics Fetcher-Analyzer, connects ChatGPT with NagGPT. It steers ChatGPT to generate and run Python code, performing bioinformatics tasks on data dynamically retrieved from a dozen common genomics databases (e.g. NCBI, Ensembl, UniProt, WormBase, and FlyBase). We implement partial mitigations for encountered challenges: detrimental interactions between code generation style and data analysis, confusion between database identifiers, and hallucination of both data and actions taken. Our results identify avenues to augment ChatGPT as a bioinformatics assistant and, more broadly, to improve factual accuracy and instruction following of unmodified LLMs.<\/jats:p>","DOI":"10.1093\/bib\/bbaf045","type":"journal-article","created":{"date-parts":[[2025,2,5]],"date-time":"2025-02-05T22:54:16Z","timestamp":1738796056000},"source":"Crossref","is-referenced-by-count":9,"title":["Steering veridical large language model analyses by correcting and enriching generated database queries: first steps toward ChatGPT bioinformatics"],"prefix":"10.1093","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4771-1857","authenticated-orcid":false,"given":"Olivier","family":"Cinquin","sequence":"first","affiliation":[{"name":"Department of Developmental and Cell Biology, Center for Complex Biological Systems, University of California at Irvine , 4203 McGaugh Hall, Irvine, CA 92697 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,2,6]]},"reference":[{"key":"2025020603540014300_ref1","first-page":"15696","volume-title":"Proceedings of the 40th International Conference on Machine Learning, in PMLR","author":"Kandpal","year":"2023"},{"key":"2025020603540014300_ref2","doi-asserted-by":"crossref","first-page":"311","DOI":"10.18653\/v1\/2024.naacl-long.18","volume-title":"Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)","author":"Sun","year":"2024"},{"key":"2025020603540014300_ref3","doi-asserted-by":"publisher","DOI":"10.1145\/3703155","article-title":"A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions","volume":"43","author":"Huang","journal-title":"ACM Trans Inf Syst"},{"key":"2025020603540014300_ref4","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1007\/s10439-023-03306-x","article-title":"A domain-specific next-generation large language model (LLM) or ChatGPT is required for biomedical engineering and research","volume":"52","author":"Pal","year":"2024","journal-title":"Ann Biomed Eng"},{"key":"2025020603540014300_ref5","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","volume-title":"Adv Neural Inf Process Syst","author":"Lewis"},{"key":"2025020603540014300_ref6","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.","author":"De Cao","year":"2021"},{"key":"2025020603540014300_ref7","author":"Sinitsin","year":"2020"},{"key":"2025020603540014300_ref8","doi-asserted-by":"publisher","first-page":"1812","DOI":"10.1093\/jamia\/ocad259","article-title":"Improving large language models for clinical named entity recognition via prompt engineering","volume":"31","author":"Hu","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"2025020603540014300_ref9","author":"Wu","year":"2023"},{"key":"2025020603540014300_ref10","author":"Razniewski","year":"2021"},{"key":"2025020603540014300_ref11","author":"Hatakeyama-Sato","year":"2023"},{"key":"2025020603540014300_ref12","author":"He","year":"2024"},{"key":"2025020603540014300_ref13","doi-asserted-by":"publisher","DOI":"10.1002\/qub2.67","article-title":"Bioinformatics and biomedical informatics with ChatGPT: Year one review","volume":"12","author":"Wang","year":"2024","journal-title":"Quant Biol"},{"key":"2025020603540014300_ref14","doi-asserted-by":"publisher","first-page":"e1011511","DOI":"10.1371\/journal.pcbi.1011511","article-title":"Evaluating a large language model\u2019s ability to solve programming exercises from an introductory bioinformatics course","volume":"19","author":"Piccolo","year":"2023","journal-title":"PLoS Comput Biol"},{"key":"2025020603540014300_ref15","doi-asserted-by":"publisher","first-page":"105","DOI":"10.15302\/J-QB-023-0327","article-title":"Empowering beginners in bioinformatics with ChatGPT","volume":"11","author":"Shue","year":"2023","journal-title":"Quant Biol"},{"key":"2025020603540014300_ref16","doi-asserted-by":"publisher","author":"Jansen","year":"2023","DOI":"10.1101\/2023.12.11.571140"},{"key":"2025020603540014300_ref17","doi-asserted-by":"publisher","first-page":"754","DOI":"10.1007\/s10439-023-03324-9","article-title":"Code interpreter for bioinformatics: Are we there yet?","volume":"52","author":"Wang","year":"2024","journal-title":"Ann Biomed Eng"},{"key":"2025020603540014300_ref18","author":"Bubeck","year":"2023"},{"key":"2025020603540014300_ref19","doi-asserted-by":"publisher","first-page":"e2723","DOI":"10.1002\/smr.2723","article-title":"LLMs for science: Usage for code generation and data analysis","volume":"37","author":"Nejjar","year":"2024","journal-title":"J Softw Evol Process"},{"key":"2025020603540014300_ref20","article-title":"An LLM compiler for parallel function calling","volume-title":"Proceedings of the 41st International Conference on Machine Learning","author":"Kim","year":"2024"},{"key":"2025020603540014300_ref21","doi-asserted-by":"publisher","first-page":"D988","DOI":"10.1093\/nar\/gkab1049","article-title":"Ensembl 2022","volume":"50","author":"Cunningham","year":"2022","journal-title":"Nucleic Acids Res"},{"key":"2025020603540014300_ref22","doi-asserted-by":"publisher","first-page":"D20","DOI":"10.1093\/nar\/gkab1112","article-title":"Database resources of the national center for biotechnology information","volume":"50","author":"Sayers","year":"2022","journal-title":"Nucleic Acids Res"},{"key":"2025020603540014300_ref23","doi-asserted-by":"publisher","first-page":"D523","DOI":"10.1093\/nar\/gkac1052","article-title":"UniProt: The universal protein knowledgebase in 2023","volume":"51","author":"The UniProt Consortium","year":"2023","journal-title":"Nucleic Acids Res"},{"key":"2025020603540014300_ref24","doi-asserted-by":"publisher","first-page":"gkz920","DOI":"10.1093\/nar\/gkz920","article-title":"WormBase: A modern model organism information resource","volume":"48","author":"Harris","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2025020603540014300_ref25","doi-asserted-by":"publisher","first-page":"D899","DOI":"10.1093\/nar\/gkaa1026","article-title":"FlyBase: Updates to the Drosophila melanogaster knowledge base","volume":"49","author":"Larkin","year":"2021","journal-title":"Nucleic Acids Res"},{"key":"2025020603540014300_ref26","doi-asserted-by":"publisher","first-page":"D26","DOI":"10.1093\/nar\/gkl993","article-title":"Entrez gene: Gene-centered information at NCBI","volume":"35","author":"Maglott","year":"2007","journal-title":"Nucleic Acids Res"},{"key":"2025020603540014300_ref27","article-title":"Gene therapy allows an 11-year-old boy to hear for the first time","author":"Kolata","year":"2024","journal-title":"The New York Times"},{"key":"2025020603540014300_ref28","author":"Uniprot","year":"2024"},{"key":"2025020603540014300_ref29","doi-asserted-by":"publisher","first-page":"1422","DOI":"10.1093\/bioinformatics\/btp163","article-title":"Biopython: Freely available python tools for computational molecular biology and bioinformatics","volume":"25","author":"Cock","year":"2009","journal-title":"Bioinformatics"},{"key":"2025020603540014300_ref30","doi-asserted-by":"publisher","first-page":"1928","DOI":"10.1093\/bioinformatics\/btz795","article-title":"Kalign 3: Multiple sequence alignment of large datasets","volume":"36","author":"Lassmann","year":"2020","journal-title":"Bioinformatics"},{"key":"2025020603540014300_ref31","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1038\/s41586-020-2308-7","article-title":"The mutational constraint spectrum quantified from variation in 141,456 humans","volume":"581","author":"Karczewski","year":"2020","journal-title":"Nature"},{"key":"2025020603540014300_ref32","doi-asserted-by":"publisher","first-page":"405","DOI":"10.1145\/1376616.1376660","volume-title":"Proceedings of the 2008 ACM SIGMOD international conference on Management of data","author":"He","year":"2008"},{"key":"2025020603540014300_ref33","first-page":"RFC8259","article-title":"The JavaScript object notation (JSON)","author":"Bray","year":"2017","journal-title":"Data Interchange Format"},{"key":"2025020603540014300_ref34","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1016\/0022-2836(82)90515-0","article-title":"A simple method for displaying the hydropathic character of a protein","volume":"157","author":"Kyte","year":"1982","journal-title":"J Mol Biol"},{"key":"2025020603540014300_ref35","doi-asserted-by":"publisher","first-page":"btae075","DOI":"10.1093\/bioinformatics\/btae075","article-title":"GeneGPT: Augmenting large language models with domain tools for improved access to biomedical information","volume":"40","author":"Jin","year":"2024","journal-title":"Bioinformatics"},{"key":"2025020603540014300_ref36","article-title":"Leveraging large language models to build and execute computational workflows","volume-title":"Proceedings of WORKS \u201923: 18th Workshop on Workflows in Support of Large-Scale Science","author":"Duque","year":"2023"},{"key":"2025020603540014300_ref37","doi-asserted-by":"publisher","first-page":"814","DOI":"10.1126\/science.1231535","article-title":"Troubling trends in scientific software use","volume":"340","author":"Joppa","year":"2013","journal-title":"Science"},{"key":"2025020603540014300_ref38","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1007\/s12551-015-0177-3","article-title":"How to test bioinformatics software?","volume":"7","author":"Kamali","year":"2015","journal-title":"Biophys Rev"},{"key":"2025020603540014300_ref39","doi-asserted-by":"publisher","first-page":"469","DOI":"10.1145\/3632620.3671116","volume-title":"Proceedings of the 2024 ACM Conference on International Computing Education Research","author":"Prather","year":"2024"},{"key":"2025020603540014300_ref40","author":"Passi","year":"2022"},{"key":"2025020603540014300_ref41","doi-asserted-by":"publisher","first-page":"1248705","DOI":"10.3389\/feduc.2024.1248705","article-title":"ChatGPT in a programming course: Benefits and limitations","volume":"9","author":"Bringula","year":"2024","journal-title":"Front Educ"},{"key":"2025020603540014300_ref42","doi-asserted-by":"publisher","first-page":"100147","DOI":"10.1016\/j.caeai.2023.100147","article-title":"The effect of generative artificial intelligence (AI)-based tool use on students\u2019 computational thinking skills, programming self-efficacy and motivation","volume":"4","author":"Yilmaz","year":"2023","journal-title":"Comput Educ Artif Intell"},{"key":"2025020603540014300_ref43","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3544548.3580919","volume-title":"Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems","author":"Kazemitabaar","year":"2023"},{"key":"2025020603540014300_ref44","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1515\/labmed-2024-0034","article-title":"Dear ChatGPT \u2013 Can you teach me how to program an app for laboratory medicine?","volume":"48","author":"Meyer","year":"2024","journal-title":"J Lab Med"},{"key":"2025020603540014300_ref45","doi-asserted-by":"publisher","DOI":"10.1016\/j.acra.2024.09.042","article-title":"Large language models can help with biostatistics and coding needed in radiology research","volume":"S1076633224006913","author":"Ghosh","year":"2024","journal-title":"Acad Radiol"},{"key":"2025020603540014300_ref46","author":"AI4Science MR","year":"2023"},{"key":"2025020603540014300_ref47","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1016\/j.ymeth.2024.03.012","article-title":"Experimental and computational approaches for membrane protein insertion and topology determination","volume":"226","author":"Duart","year":"2024","journal-title":"Methods"},{"key":"2025020603540014300_ref48","doi-asserted-by":"publisher","first-page":"qzae002","DOI":"10.1093\/gpbjnl\/qzae002","article-title":"On the responsible use of Chatbots in bioinformatics","volume":"22","author":"Hu","year":"2024","journal-title":"Genomics Proteomics Bioinformatics"},{"key":"2025020603540014300_ref49","first-page":"1","article-title":"Validating AI-generated code with live programming","author":"Ferdowsi","year":"2024","journal-title":"Proceedings of the CHI Conference on Human Factors in Computing Systems"},{"key":"2025020603540014300_ref50","article-title":"Prompts first, finally","author":"Reeves","year":"2024"},{"key":"2025020603540014300_ref51","volume-title":"The Impact of AI on Developer Productivity: Evidence from GitHub Copilot","author":"Peng","year":"2023"},{"key":"2025020603540014300_ref52","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1126\/science.adh2586","article-title":"Experimental evidence on the productivity effects of generative artificial intelligence","volume":"381","author":"Noy","year":"2023","journal-title":"Science"},{"key":"2025020603540014300_ref53","doi-asserted-by":"publisher","first-page":"273","DOI":"10.12688\/f1000research.10750.2","article-title":"General guidelines for biomedical software development","volume":"6","author":"Silva","year":"2017","journal-title":"F1000Res"},{"key":"2025020603540014300_ref54","doi-asserted-by":"publisher","first-page":"e839","DOI":"10.7717\/peerj-cs.839","article-title":"Improving bioinformatics software quality through incorporation of software engineering practices","volume":"8","author":"Noor","year":"2022","journal-title":"PeerJ Comput Sci"},{"key":"2025020603540014300_ref55","doi-asserted-by":"publisher","first-page":"btae632","DOI":"10.1093\/bioinformatics\/btae632","article-title":"Improving bioinformatics software quality through teamwork","volume":"40","author":"Ferenc","year":"2024","journal-title":"Bioinformatics"},{"key":"2025020603540014300_ref56","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1145\/3643795.3648396","volume-title":"LLM4Code '24: Proceedings of the 1st International Workshop on Large Language Models for Code","author":"Bhatia","year":"2024"},{"key":"2025020603540014300_ref57","volume-title":"No more Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generation","author":"Yuan","year":"2024"},{"key":"2025020603540014300_ref58","doi-asserted-by":"publisher","first-page":"468","DOI":"10.1126\/science.1240474","article-title":"Atypical combinations and scientific impact","volume":"342","author":"Uzzi","year":"2013","journal-title":"Science"},{"key":"2025020603540014300_ref59","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1038\/s41586-019-0941-9","article-title":"Large teams develop and small teams disrupt science and technology","volume":"566","author":"Wu","year":"2019","journal-title":"Nature"},{"key":"2025020603540014300_ref60","doi-asserted-by":"publisher","DOI":"10.1007\/s43681-024-00493-8","article-title":"The ethics of using artificial intelligence in scientific research: New guidance needed for a new tool","author":"Resnik","year":"2024","journal-title":"AI Ethics"},{"key":"2025020603540014300_ref61","doi-asserted-by":"publisher","first-page":"S15","DOI":"10.1186\/1471-2105-15-S16-S15","article-title":"Verification and validation of bioinformatics software without a gold standard: A case study of BWA and bowtie","volume":"15","author":"Giannoulatou","year":"2014","journal-title":"BMC Bioinformatics"},{"key":"2025020603540014300_ref62","author":"name no","year":"2024"},{"key":"2025020603540014300_ref63","author":"Gentleman","year":"2008"},{"key":"2025020603540014300_ref64","author":"Zheng","year":"2024"},{"key":"2025020603540014300_ref65","doi-asserted-by":"publisher","first-page":"i266","DOI":"10.1093\/bioinformatics\/btae230","article-title":"BioCoder: A benchmark for bioinformatics code generation with large language models","volume":"40","author":"Tang","year":"2024","journal-title":"Bioinformatics"},{"key":"2025020603540014300_ref66","article-title":"Toolformer: language models can teach themselves to use tools","volume-title":"Adv Neural Inf Process Syst","author":"Schick","year":"2023"},{"key":"2025020603540014300_ref67","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2024.emnlp-demo.9","article-title":"TinyAgent: function calling at the edge","volume-title":"Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations","author":"Erdogan","year":"2024"},{"key":"2025020603540014300_ref68","doi-asserted-by":"publisher","first-page":"18030","DOI":"10.1609\/aaai.v38i16.29759","article-title":"Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum","volume":"38","author":"Gao","year":"2024","journal-title":"AAAI"},{"key":"2025020603540014300_ref69","author":"Liu","year":"2024"},{"key":"2025020603540014300_ref70","author":"Qin","year":"2023"},{"key":"2025020603540014300_ref71","author":"Liu","year":"2024"},{"key":"2025020603540014300_ref72","author":"Patil","year":"2023"},{"key":"2025020603540014300_ref73","author":"Singh","year":"2024"},{"key":"2025020603540014300_ref74","author":"Zhuang","year":"2023"},{"key":"2025020603540014300_ref75","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1145\/3643795.3648384","volume-title":"LLM4Code '24: Proceedings of the 1st International Workshop on Large Language Models for Code","author":"Koziolek","year":"2024"},{"key":"2025020603540014300_ref76","first-page":"1","article-title":"We need structured output: Towards user-centered constraints on large language model output","author":"Liu","year":"2024","journal-title":"Extended Abstracts of the CHI Conference on Human Factors in Computing Systems"},{"key":"2025020603540014300_ref77","author":"Willard","year":"2023"},{"key":"2025020603540014300_ref78","author":"Beurer-Kellner","year":"2024"},{"key":"2025020603540014300_ref79","doi-asserted-by":"publisher","first-page":"bbac409","DOI":"10.1093\/bib\/bbac409","article-title":"BioGPT: Generative pre-trained transformer for biomedical text generation and mining","volume":"23","author":"Luo","year":"2022","journal-title":"Brief Bioinform"},{"key":"2025020603540014300_ref80","doi-asserted-by":"publisher","first-page":"bbad535","DOI":"10.1093\/bib\/bbad535","article-title":"ChIP-GPT: A managed large language model for robust data extraction from biomedical database records","volume":"25","author":"Cinquin","year":"2024","journal-title":"Brief Bioinform"},{"key":"2025020603540014300_ref81","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1145\/3605764.3623985","volume-title":"Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security","author":"Greshake","year":"2023"},{"key":"2025020603540014300_ref82","author":"OpenAI","year":"2024"},{"key":"2025020603540014300_ref83","doi-asserted-by":"publisher","first-page":"e53164","DOI":"10.2196\/53164","article-title":"Hallucination rates and reference accuracy of ChatGPT and bard for systematic reviews: Comparative analysis","volume":"26","author":"Chelli","year":"2024","journal-title":"J Med Internet Res"},{"key":"2025020603540014300_ref84","author":"OpenAI","year":"2024"},{"key":"2025020603540014300_ref85","author":"Guan","year":"2024"},{"key":"2025020603540014300_ref86","author":"Hughes","year":"2024"},{"key":"2025020603540014300_ref87","doi-asserted-by":"publisher","DOI":"10.1162\/qss_a_00327","article-title":"The strain on scientific publishing","volume":"5","author":"Hanson","year":"2024","journal-title":"Quant Sci Stud"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/1\/bbaf045\/61772497\/bbaf045.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/1\/bbaf045\/61772497\/bbaf045.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,5]],"date-time":"2025-02-05T22:54:25Z","timestamp":1738796065000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbaf045\/8002976"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,22]]},"references-count":87,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,11,22]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbaf045","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,1]]},"published":{"date-parts":[[2024,11,22]]},"article-number":"bbaf045"}}