{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,3]],"date-time":"2026-05-03T01:38:36Z","timestamp":1777772316541,"version":"3.51.4"},"reference-count":50,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T00:00:00Z","timestamp":1773014400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["1R01LM014239"],"award-info":[{"award-number":["1R01LM014239"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000049","name":"National Institute on Aging","doi-asserted-by":"publisher","award":["R01AG080429"],"award-info":[{"award-number":["R01AG080429"]}],"id":[{"id":"10.13039\/100000049","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,5,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Automated literature screening in biomedical research is often hindered by domain shifts and scarcity of labeled data, which limit model accuracy and generalizability. While large language models (LLMs) perform well in zero-shot settings, they often fail to capture complex, domain-specific reasoning patterns. To address this limitation, this study investigates whether an interactive, weakly supervised learning framework combining GPT (generative pre-trained transformer)\u2019s fine-tuning adaptability with DeepSeek\u2019s reasoning capabilities can improve literature screening performance across biomedical domains.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>We developed an active learning framework that leverages model disagreement between GPT-4o and DeepSeek to improve literature screening performance. This process began with a labeled corpus of 6331 articles on large language models, from which a model disagreement analysis was performed to identify cases where GPT-4o misclassified and DeepSeek produced correct predictions. Three GPT variants\u2014GPT-4o, GPT-4o-mini, and GPT-4.1-nano, were fine-tuned under standard supervised learning settings using these disagreement-based samples. Fine-tuning prompts incorporated classification labels and, when available, rationale traces generated by DeepSeek to provide reasoning-augmented weak supervision. Model performance was evaluated on an independent benchmark set of 291 annotated articles across 10 topic queries in cancer immunotherapy and LLMs in medicine, using standard evaluation metrics, with recall as the primary measure.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Fine-tuning GPT models using disagreement-based examples significantly improved performance. GPT-4o-mini achieved the best overall results after fine-tuning, especially with the highest F1 score (0.93, P &amp;lt; .001) and recall (0.95, P &amp;lt; .001). Across the biomedical topics, fine-tuned models consistently outperformed their zero-shot counterparts without increasing reviewer workload.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>These findings demonstrate the effectiveness of disagreement-driven active learning in enhancing GPT-based biomedical literature screening. Lightweight models like GPT-4o-mini benefit most from targeted, reasoning-enriched training, highlighting their suitability for scalable deployment.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>This study introduces an interactive active learning framework that leverages fine-tuned LLMs with reasoning capabilities to enhance literature screening. The approach offers a scalable solution to more efficient and reliable information retrieval in systematic reviews.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocag014","type":"journal-article","created":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T12:27:36Z","timestamp":1772540856000},"page":"1026-1036","source":"Crossref","is-referenced-by-count":0,"title":["Interactive active learning for literature screening: finetuning GPT with DeepSeek reasoning for cross-domain generalization"],"prefix":"10.1093","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-8784-1745","authenticated-orcid":false,"given":"Yiming","family":"Li","sequence":"first","affiliation":[{"name":"Department of Medicine, Harvard Medical School , Boston, MA 02115,","place":["United States"]},{"name":"Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women\u2019s Hospital , Boston, MA 02115,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joseph M","family":"Plasek","sequence":"additional","affiliation":[{"name":"Department of Medicine, Harvard Medical School , Boston, MA 02115,","place":["United States"]},{"name":"Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women\u2019s Hospital , Boston, MA 02115,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3713-3264","authenticated-orcid":false,"given":"Xinsong","family":"Du","sequence":"additional","affiliation":[{"name":"Department of Medicine, Harvard Medical School , Boston, MA 02115,","place":["United States"]},{"name":"Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women\u2019s Hospital , Boston, MA 02115,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yifei","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Brandeis University , Waltham, MA 02453,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhengyang","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Brandeis University , Waltham, MA 02453,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John","family":"Lian","sequence":"additional","affiliation":[{"name":"Department of Medicine, Harvard Medical School , Boston, MA 02115,","place":["United States"]},{"name":"Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women\u2019s Hospital , Boston, MA 02115,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ya-Wen","family":"Chuang","sequence":"additional","affiliation":[{"name":"Division of Nephrology, Department of Internal Medicine, Taichung Veterans General Hospital , Taichung 407219,","place":["Taiwan"]},{"name":"Department of Post-Baccalaureate Medicine, College of Medicine, National Chung Hsing University , Taichung 402202,","place":["Taiwan"]},{"name":"School of Medicine, College of Medicine, China Medical University , Taichung 404328,","place":["Taiwan"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pengyu","family":"Hong","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Brandeis University , Waltham, MA 02453,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter C","family":"Hou","sequence":"additional","affiliation":[{"name":"Division of Emergency Critical Care Medicine, Department of Emergency Medicine, Brigham and Women\u2019s Hospital , Boston, MA 02115,","place":["United States"]},{"name":"Department of Emergency Medicine, Harvard Medical School, Boston, MA 02115,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Li","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Medicine, Harvard Medical School , Boston, MA 02115,","place":["United States"]},{"name":"Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women\u2019s Hospital , Boston, MA 02115,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2026,3,9]]},"reference":[{"key":"2026042913023043000_ocag014-B1","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1016\/j.mcpdig.2024.05.023","article-title":"Future of evidence synthesis: automated, living, and interactive systematic reviews and meta-analyses","volume":"2","author":"Riaz","year":"2024","journal-title":"Mayo Clin Proc Digit Health"},{"key":"2026042913023043000_ocag014-B2","doi-asserted-by":"publisher","first-page":"e012545","DOI":"10.1136\/bmjopen-2016-012545","article-title":"Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry","volume":"7","author":"Borah","year":"2017","journal-title":"BMJ Open"},{"key":"2026042913023043000_ocag014-B3","doi-asserted-by":"publisher","first-page":"D20","DOI":"10.1093\/nar\/gkae979","article-title":"Database resources of the National Center for Biotechnology Information in 2025","volume":"53","author":"Sayers","year":"2025","journal-title":"Nucleic Acids Res."},{"key":"2026042913023043000_ocag014-B4","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1002\/jrsm.1589","article-title":"In-depth evaluation of machine learning methods for semi-automating article screening in a systematic review of mechanistic literature","volume":"14","author":"Kebede","year":"2023","journal-title":"Res Synth Methods"},{"key":"2026042913023043000_ocag014-B5","doi-asserted-by":"publisher","first-page":"3047","DOI":"10.1007\/s11192-020-03648-6","article-title":"Automatic document screening of medical literature using word and text embeddings in an active learning setting","volume":"125","author":"Carvallo","year":"2020","journal-title":"Scientometrics"},{"key":"2026042913023043000_ocag014-B6","doi-asserted-by":"publisher","first-page":"3","DOI":"10.21203\/rs.3.rs-4876692\/v1","article-title":"VaxBot-HPV: a GPT-based chatbot for answering HPV vaccine-related questions","volume":"3","author":"Tao","year":"2024","journal-title":"Res Sq"},{"issue":"2","key":"2026042913023043000_ocag014-B7","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1093\/jamia\/ocaf179","article-title":"AcuKG: a comprehensive knowledge graph for medical acupuncture","volume":"33","author":"Li","year":"2026","journal-title":"J Am Med Inform Assoc"},{"key":"2026042913023043000_ocag014-B8","doi-asserted-by":"publisher","author":"Hu","DOI":"10.48550\/arXiv.2303.16416"},{"key":"2026042913023043000_ocag014-B9","doi-asserted-by":"publisher","first-page":"104621","DOI":"10.1016\/j.jbi.2024.104621","article-title":"Artificial intelligence-powered pharmacovigilance: a review of machine and deep learning in clinical text-based adverse drug event detection for benchmark datasets","volume":"152","author":"Li","year":"2024","journal-title":"J Biomed Inform"},{"key":"2026042913023043000_ocag014-B10","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1186\/s13326-024-00318-x","article-title":"Mapping vaccine names in clinical trials to vaccine ontology using cascaded fine-tuned domain-specific language models","volume":"15","author":"Li","year":"2024","journal-title":"J Biomed Semantics"},{"key":"2026042913023043000_ocag014-B11","doi-asserted-by":"publisher","first-page":"104735","DOI":"10.1016\/j.jbi.2024.104735","article-title":"Improving tabular data extraction in scanned laboratory reports using deep learning models","volume":"159","author":"Li","year":"2024","journal-title":"J Biomed Inform"},{"key":"2026042913023043000_ocag014-B12","doi-asserted-by":"publisher","first-page":"104930","DOI":"10.1016\/j.jbi.2025.104930","article-title":"Exploring multimodal large language models on transthoracic echocardiogram (TTE) tasks for cardiovascular decision support","volume":"171","author":"Li","year":"2025","journal-title":"J Biomed Inform"},{"key":"2026042913023043000_ocag014-B13","doi-asserted-by":"publisher","first-page":"e48996","DOI":"10.2196\/48996","article-title":"Automated paper screening for clinical reviews using large language models: data analysis study","volume":"26","author":"Guo","year":"2024","journal-title":"J Med Internet Res."},{"key":"2026042913023043000_ocag014-B14","doi-asserted-by":"publisher","first-page":"2030","DOI":"10.1093\/jamia\/ocae129","article-title":"RefAI: a GPT-powered retrieval-augmented generative tool for biomedical literature recommendation and summarization","volume":"31","author":"Li","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"2026042913023043000_ocag014-B15","doi-asserted-by":"publisher","first-page":"2622","DOI":"10.1093\/jamia\/ocae233","article-title":"Relation extraction using large language models: a case study on acupuncture point locations","volume":"31","author":"Li","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"2026042913023043000_ocag014-B16","doi-asserted-by":"publisher","first-page":"2146","DOI":"10.3390\/app15042146","article-title":"Fine-tuning large language models for ontology engineering: a comparative analysis of GPT-4 and Mistral","volume":"15","author":"Doumanas","year":"2025","journal-title":"Appl Sci"},{"key":"2026042913023043000_ocag014-B17","author":"Li","year":"2024"},{"key":"2026042913023043000_ocag014-B18","doi-asserted-by":"publisher","first-page":"1726","DOI":"10.3390\/healthcare12171726","article-title":"Custom GPTs enhancing performance and evidence compared with GPT-3.5, GPT-4, and GPT-4o? A study on the Emergency Medicine Specialist Examination","volume":"12","author":"Liu","year":"2024","journal-title":"Healthcare (Basel)"},{"key":"2026042913023043000_ocag014-B19","doi-asserted-by":"publisher","author":"Li","year":"2025","DOI":"10.21203\/rs.3.rs-6201919\/v1"},{"key":"2026042913023043000_ocag014-B20","doi-asserted-by":"publisher","first-page":"718","DOI":"10.3233\/SHTI250934","article-title":"PheCatcher: leveraging LLM-generated synthetic data for automated phenotype definition extraction from biomedical literature","volume":"329","author":"Hu","year":"2025","journal-title":"Stud Health Technol Inform"},{"key":"2026042913023043000_ocag014-B21","doi-asserted-by":"publisher","first-page":"ooaf005","DOI":"10.1093\/jamiaopen\/ooaf005","article-title":"VaxBot-HPV: a GPT-based chatbot for answering HPV vaccine-related questions","volume":"8","author":"Li","year":"2025","journal-title":"JAMIA Open"},{"key":"2026042913023043000_ocag014-B22"},{"key":"2026042913023043000_ocag014-B23","doi-asserted-by":"publisher","first-page":"1071","DOI":"10.1093\/jamia\/ocaf063","article-title":"The emergence of large language models as tools in literature reviews: a large language model-assisted systematic review","volume":"32","author":"Scherbakov","year":"2025","journal-title":"J Am Med Inform Assoc"},{"key":"2026042913023043000_ocag014-B24","doi-asserted-by":"publisher","first-page":"e43023","DOI":"10.7759\/cureus.43023","article-title":"Streamlining systematic reviews: harnessing large language models for quality assessment and risk-of-bias evaluation","volume":"15","author":"Nashwan","year":"2023","journal-title":"Cureus"},{"key":"2026042913023043000_ocag014-B25","doi-asserted-by":"publisher","first-page":"106205","DOI":"10.1016\/j.ijmedinf.2025.106205","article-title":"AI-assisted literature screening: a hybrid approach using large language models and retrieval-augmented generation","volume":"207","author":"Li","year":"2026","journal-title":"Int J Med Inform"},{"key":"2026042913023043000_ocag014-B26","author":"Akinseloyin","year":"2026"},{"key":"2026042913023043000_ocag014-B27","doi-asserted-by":"publisher","first-page":"e0300919","DOI":"10.1371\/journal.pone.0300919","article-title":"AE-GPT: using large language models to extract adverse events from surveillance reports\u2014a use case with influenza vaccine adverse events","volume":"19","author":"Li","year":"2024","journal-title":"PLoS One"},{"key":"2026042913023043000_ocag014-B28","doi-asserted-by":"publisher","first-page":"104789","DOI":"10.1016\/j.jbi.2025.104789","article-title":"Improving entity recognition using ensembles of deep learning and fine-tuned large language models: a case study on adverse event extraction from VAERS and social media","volume":"163","author":"Li","year":"2025","journal-title":"J Biomed Inform"},{"key":"2026042913023043000_ocag014-B29","first-page":"748","author":"Lu","year":"2024"},{"key":"2026042913023043000_ocag014-B30","doi-asserted-by":"crossref","first-page":"728","DOI":"10.1109\/PASSAT\/SocialCom.2011.193","volume-title":"2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing","author":"Zhao","year":"2011"},{"key":"2026042913023043000_ocag014-B31","author":"Zhao","year":"2013"},{"key":"2026042913023043000_ocag014-B32","first-page":"406","volume-title":"Proceedings of the Sixteenth International Conference on Machine Learning","author":"Thompson","year":"1999"},{"key":"2026042913023043000_ocag014-B33","first-page":"350","volume-title":"Proceedings of the Fifteenth International Conference on Machine Learning","author":"McCallum","year":"1998"},{"key":"2026042913023043000_ocag014-B34","volume-title":"Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC\u201910).","author":"Ambati","year":"2010"},{"key":"2026042913023043000_ocag014-B35","doi-asserted-by":"publisher","first-page":"ocaf233","DOI":"10.1101\/2024.08.11.24311828","article-title":"Testing and evaluation of generative large language models in electronic health record applications: a systematic review","author":"Du","year":"2025","journal-title":"JAm Med Inform Assoc"},{"key":"2026042913023043000_ocag014-B36","doi-asserted-by":"publisher","first-page":"106091","DOI":"10.1016\/j.ijmedinf.2025.106091","article-title":"Performance and improvement strategies for adapting generative large language models for electronic health record applications: a systematic review","volume":"205","author":"Du","year":"2026","journal-title":"Int J Med Inform"},{"key":"2026042913023043000_ocag014-B37","doi-asserted-by":"publisher","first-page":"104867","DOI":"10.1016\/j.jbi.2025.104867","article-title":"A comparative study of recent large language models on generating hospital discharge summaries for lung cancer patients","volume":"168","author":"Li","year":"2025","journal-title":"J Biomed Inform"},{"key":"2026042913023043000_ocag014-B38","doi-asserted-by":"publisher","author":"Siddiky","year":"2025","DOI":"10.20944\/preprints202502.0066.v1"},{"key":"2026042913023043000_ocag014-B39","doi-asserted-by":"publisher","first-page":"7782","DOI":"10.3390\/app14177782","article-title":"Putting GPT-4o to the sword: a comprehensive evaluation of language, vision, speech, and multimodal proficiency","volume":"14","author":"Shahriar","year":"2024","journal-title":"Appl Sci"},{"key":"2026042913023043000_ocag014-B40","doi-asserted-by":"publisher","author":"Joshi","DOI":"10.20944\/preprints202503.1887.v1"},{"key":"2026042913023043000_ocag014-B41","doi-asserted-by":"publisher","author":"Neha","DOI":"10.36227\/techrxiv.173896582.25938392\/v1"},{"key":"2026042913023043000_ocag014-B42","doi-asserted-by":"publisher","author":"Liu","DOI":"10.48550\/arxiv.2502.09100"},{"key":"2026042913023043000_ocag014-B43","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1038\/s41586-025-09422-z2025","article-title":"DeepSeek-R1: incentivizing reasoning capability in LLMs via reinforcement learning","volume":"645","author":"DeepSeek","year":"2025","journal-title":"Nature"},{"key":"2026042913023043000_ocag014-B44","doi-asserted-by":"publisher","author":"Ye","DOI":"10.48550\/arxiv.2506.01257"},{"key":"2026042913023043000_ocag014-B45","doi-asserted-by":"publisher","author":"Abraham","DOI":"10.31219\/osf.io\/9r7e3_v1"},{"key":"2026042913023043000_ocag014-B46","doi-asserted-by":"publisher","first-page":"872","DOI":"10.1109\/JAS.2025.125498","article-title":"Exploring DeepSeek: a survey on advances, applications, challenges and future directions","volume":"12","author":"Deng","year":"2025","journal-title":"IEEE\/CAA J Autom Sinica"},{"key":"2026042913023043000_ocag014-B47","author":"K\u00e4yhk\u00f6","year":"2025"},{"key":"2026042913023043000_ocag014-B48","doi-asserted-by":"publisher","DOI":"10.48550\/arxiv.2501.14326"},{"key":"2026042913023043000_ocag014-B49","author":"OpenAI News","year":"2025"},{"key":"2026042913023043000_ocag014-B50","doi-asserted-by":"publisher","author":"Yu","DOI":"10.48550\/arxiv.2506.03637"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/33\/5\/1026\/67280244\/ocag014.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/33\/5\/1026\/67280244\/ocag014.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T17:02:39Z","timestamp":1777482159000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/33\/5\/1026\/8512191"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,9]]},"references-count":50,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,3,9]]},"published-print":{"date-parts":[[2026,5,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocag014","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,5]]},"published":{"date-parts":[[2026,3,9]]}}}