{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T18:13:31Z","timestamp":1767636811345,"version":"3.48.0"},"reference-count":26,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2025,12,12]],"date-time":"2025-12-12T00:00:00Z","timestamp":1765497600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["U01 AG066833"],"award-info":[{"award-number":["U01 AG066833"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["2R01CA151610"],"award-info":[{"award-number":["2R01CA151610"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R21CA280458"],"award-info":[{"award-number":["R21CA280458"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R00CA256519"],"award-info":[{"award-number":["R00CA256519"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Glazer Foundation Award"},{"name":"Samuel Oschin Cancer Institute Research Development Fund"},{"name":"Jim and Eleanor Department of Surgery Research Award"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,1,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Transcription factors (TFs) and their target genes form regulatory networks that control gene expression and influence diverse biological processes and disease outcomes. Although multiple computational methods and curated databases have been developed to identify TF\u2013target interactions, they often require specialized expertise. Large language models (LLMs) chatbots offer a more accessible alternative for querying TF\u2013target interactions. In this study, we benchmarked four prominent LLMs, Anthropic\u2019s Claude 3.5 Sonnet, Google\u2019s Gemini 1.0 Pro, OpenAI\u2019s GPT-4o, and Meta\u2019s Llama3 8b, using 8432 literature-curated human TF\u2013target interactions. We examined four regulatory categories: bidirectional, ambiguous, self-regulated, and unidirectional interactions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Under single-turn queries, Claude 3.5 Sonnet and GPT-4o outperformed the others, with balanced accuracies reaching 50.0\u202f\u00b1\u202f7.6% (GPT-4o, self-regulated) and 48.2\u202f\u00b1\u202f1.0% (Claude 3.5 Sonnet, unidirectional). Zero-temperature settings generally enhanced reproducibility, and multi-turn prompting improved performance for most models, increasing Claude 3.5 Sonnet\u2019s accuracy on self-regulated pairs by 32.6%. Excluding TF\u2013target pairs with all unknown regulation types also generally improved accuracy, with unidirectional regulation reaching near 70% balanced accuracy in some cases. We also benchmarked Anthropic\u2019s Claude 3.5 Sonnet, Google\u2019s Gemini 2.0 Flash, OpenAI\u2019s GPT-4o, and Meta\u2019s Llama3 using 5148 experimentally derived TF\u2013target interactions. Claude 3.5 Sonnet consistently outperformed the other models across conditions. Our findings highlight that prompt engineering and strategic use of model parameters consistently influence LLM chatbots\u2019 performance on TF\u2013target identifications. This study establishes a benchmarking framework and demonstrates the potential of pre-trained general-purpose LLMs to support regulatory biology research, especially for researchers without extensive computational expertise.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>The literature-based TF\u2013target interactions ground truth were obtained from TRRUST v2 human dataset (www.grnpedia.org\/trrust). The experimental derived TF\u2013target interactions ground truth were obtained from TFLink Home Sapiens small-scale interaction table (https:\/\/tflink.net\/). Processed TF\u2013target interactions data and the analytical pipeline has been compiled as an interactive Python notebook file and is available at https:\/\/github.com\/pengpclab\/LLM-TF-interactions.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btaf653","type":"journal-article","created":{"date-parts":[[2025,12,9]],"date-time":"2025-12-09T12:50:54Z","timestamp":1765284654000},"source":"Crossref","is-referenced-by-count":0,"title":["Benchmarking large language models for identifying transcription factor regulatory interactions"],"prefix":"10.1093","volume":"42","author":[{"given":"Lake","family":"Noel","sequence":"first","affiliation":[{"name":"Department of Computational Biomedicine, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]}]},{"given":"Yi-Wen","family":"Hsiao","sequence":"additional","affiliation":[{"name":"Department of Computational Biomedicine, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]}]},{"given":"Yimeng","family":"He","sequence":"additional","affiliation":[{"name":"Department of Computational Biomedicine, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]},{"name":"Biomedical Imaging Research Institute, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]}]},{"given":"Andrew","family":"Hung","sequence":"additional","affiliation":[{"name":"Department of Urology, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]}]},{"given":"Xiaojiang","family":"Cui","sequence":"additional","affiliation":[{"name":"Department of Surgery, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]}]},{"given":"Edward","family":"Ray","sequence":"additional","affiliation":[{"name":"Department of Surgery, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]}]},{"given":"Jason H","family":"Moore","sequence":"additional","affiliation":[{"name":"Department of Computational Biomedicine, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5004-4322","authenticated-orcid":false,"given":"Pei-Chen","family":"Peng","sequence":"additional","affiliation":[{"name":"Department of Computational Biomedicine, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0498-3131","authenticated-orcid":false,"given":"Xiuzhen","family":"Huang","sequence":"additional","affiliation":[{"name":"Department of Computational Biomedicine, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]}]}],"member":"286","published-online":{"date-parts":[[2025,12,12]]},"reference":[{"key":"2026010509224196200_btaf653-B1","doi-asserted-by":"crossref","first-page":"vbac016","DOI":"10.1093\/bioadv\/vbac016","article-title":"decoupleR: ensemble of computational methods to infer biological activities from omics data","volume":"2","author":"Badia-I-Mompel","year":"2022","journal-title":"Bioinform Adv"},{"key":"2026010509224196200_btaf653-B2","doi-asserted-by":"crossref","first-page":"lqad083","DOI":"10.1093\/nargab\/lqad083","article-title":"TGPred: efficient methods for predicting target genes of a transcription factor by integrating statistics, machine learning and optimization","volume":"5","author":"Cao","year":"2023","journal-title":"NAR Genom Bioinform"},{"key":"2026010509224196200_btaf653-B3","first-page":"200336","article-title":"Claude 2.0 large language model: tackling a real-world classification problem with a new iterative prompt engineering approach","volume":"21","author":"Caruccio","year":"2024","journal-title":"Intell Syst Appl"},{"key":"2026010509224196200_btaf653-B4","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1186\/1471-2164-14-84","article-title":"An integrated approach to characterize transcription factor and microRNA regulatory networks involved in Schwann cell response to peripheral nerve injury","volume":"14","author":"Chang","year":"2013","journal-title":"BMC Genomics"},{"key":"2026010509224196200_btaf653-B5","doi-asserted-by":"crossref","first-page":"D1016","DOI":"10.1093\/nar\/gkae1080","article-title":"Harmonizome 3.0: integrated knowledge about genes and proteins from diverse multi-omics resources","volume":"53","author":"Diamant","year":"2025","journal-title":"Nucleic Acids Res"},{"key":"2026010509224196200_btaf653-B6","doi-asserted-by":"crossref","first-page":"1363","DOI":"10.1101\/gr.240663.118","article-title":"Benchmark and integration of resources for the estimation of human transcription factor activities","volume":"29","author":"Garcia-Alonso","year":"2019","journal-title":"Genome Res"},{"key":"2026010509224196200_btaf653-B7","doi-asserted-by":"crossref","first-page":"D380","DOI":"10.1093\/nar\/gkx1013","article-title":"TRRUST v2: an expanded reference database of human and mouse transcriptional regulatory interactions","volume":"46","author":"Han","year":"2018","journal-title":"Nucleic Acids Res"},{"key":"2026010509224196200_btaf653-B8","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1186\/s40561-024-00310-z","article-title":"Google gemini as a next generation AI educational tool: a review of emerging educational technology","volume":"11","author":"Imran","year":"2024","journal-title":"Smart Learn Environ"},{"key":"2026010509224196200_btaf653-B9","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.gde.2017.02.007","article-title":"Transcription factor-DNA binding: beyond binding site motifs","volume":"43","author":"Inukai","year":"2017","journal-title":"Curr Opin Genet Dev"},{"key":"2026010509224196200_btaf653-B10","doi-asserted-by":"crossref","first-page":"2817","DOI":"10.1007\/s00330-023-10213-1","article-title":"ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports","volume":"34","author":"Jeblick","year":"2022","journal-title":"Eur Radiol"},{"key":"2026010509224196200_btaf653-B11","doi-asserted-by":"crossref","first-page":"D104","DOI":"10.1093\/nar\/gkaa1057","article-title":"GTRD: an integrated view of transcription regulation","volume":"49","author":"Kolmykov","year":"2021","journal-title":"Nucleic Acids Res"},{"key":"2026010509224196200_btaf653-B12","doi-asserted-by":"crossref","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digit Health"},{"key":"2026010509224196200_btaf653-B13","doi-asserted-by":"crossref","first-page":"650","DOI":"10.1016\/j.cell.2018.01.029","article-title":"The human transcription factors","volume":"172","author":"Lambert","year":"2018","journal-title":"Cell"},{"year":"2025","author":"Lee","key":"2026010509224196200_btaf653-B14"},{"key":"2026010509224196200_btaf653-B15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3233547.3233551","author":"Lim","year":"2018"},{"key":"2026010509224196200_btaf653-B16","doi-asserted-by":"publisher","DOI":"10.1093\/database\/baac083","article-title":"TFLink: an integrated gateway to access transcription factor-target gene interactions for multiple species","volume":"2022","author":"Liska","year":"2022","journal-title":"Database (Oxford)"},{"key":"2026010509224196200_btaf653-B17","doi-asserted-by":"crossref","first-page":"1237","DOI":"10.1093\/jamia\/ocad072","article-title":"Using AI-generated suggestions from ChatGPT to optimize clinical decision support","volume":"30","author":"Liu","year":"2023","journal-title":"J Am Med Inform Assoc"},{"key":"2026010509224196200_btaf653-B18","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1101\/gr.097378.109","article-title":"Gene regulatory networks and the role of robustness and stochasticity in the control of gene expression","volume":"21","author":"Macneil","year":"2011","journal-title":"Genome Res"},{"key":"2026010509224196200_btaf653-B19","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1186\/s13040-023-00339-9","article-title":"ChatGPT and large language models in academia: opportunities and challenges","volume":"16","author":"Meyer","year":"2023","journal-title":"BioData Min"},{"key":"2026010509224196200_btaf653-B20","doi-asserted-by":"crossref","first-page":"10934","DOI":"10.1093\/nar\/gkad841","article-title":"Expanding the coverage of regulons from high-confidence prior knowledge for accurate estimation of transcription factor activities","volume":"51","author":"M\u00fcller-Dott","year":"2023","journal-title":"Nucleic Acids Res"},{"year":"2024","author":"OpenAI","key":"2026010509224196200_btaf653-B21"},{"key":"2026010509224196200_btaf653-B22","doi-asserted-by":"crossref","first-page":"e1011511","DOI":"10.1371\/journal.pcbi.1011511","article-title":"Evaluating a large language model\u2019s ability to solve programming exercises from an introductory bioinformatics course","volume":"19","author":"Piccolo","year":"2023","journal-title":"PLoS Comput Biol"},{"year":"2023","author":"Touvron","key":"2026010509224196200_btaf653-B23"},{"key":"2026010509224196200_btaf653-B24","doi-asserted-by":"crossref","first-page":"e2000034","DOI":"10.1002\/pmic.202000034","article-title":"Transcription factors: bridge between cell signaling and gene regulation","volume":"21","author":"Weidem\u00fcller","year":"2021","journal-title":"Proteomics"},{"first-page":"9","year":"2024","author":"Yin","key":"2026010509224196200_btaf653-B25"},{"key":"2026010509224196200_btaf653-B26","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1016\/j.gpb.2019.09.006","article-title":"hTFtarget: a comprehensive database for regulations of human transcription factors and their targets","volume":"18","author":"Zhang","year":"2020","journal-title":"Genomics Proteomics Bioinformatics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaf653\/65858172\/btaf653.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaf653\/65858172\/btaf653.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaf653\/65858172\/btaf653.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T14:22:50Z","timestamp":1767622970000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btaf653\/8378393"}},"subtitle":[],"editor":[{"given":"Janet","family":"Kelso","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2025,12,12]]},"references-count":26,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1,2]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btaf653","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"type":"print","value":"1367-4803"},{"type":"electronic","value":"1367-4811"}],"subject":[],"published-other":{"date-parts":[[2026,1]]},"published":{"date-parts":[[2025,12,12]]},"article-number":"btaf653"}}