{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T10:11:11Z","timestamp":1776161471928,"version":"3.50.1"},"reference-count":63,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T00:00:00Z","timestamp":1710979200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01AG078154"],"award-info":[{"award-number":["R01AG078154"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01LM013519"],"award-info":[{"award-number":["R01LM013519"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["1K99LM01402"],"award-info":[{"award-number":["1K99LM01402"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Intramural Research Program of the National Library of Medicine"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,3,29]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Motivation<\/jats:title><jats:p>Large Language Models (LLMs) have the potential to revolutionize the field of Natural Language Processing, excelling not only in text generation and reasoning tasks but also in their ability for zero\/few-shot learning, swiftly adapting to new tasks with minimal fine-tuning. LLMs have also demonstrated great promise in biomedical and healthcare applications. However, when it comes to Named Entity Recognition (NER), particularly within the biomedical domain, LLMs fall short of the effectiveness exhibited by fine-tuned domain-specific models. One key reason is that NER is typically conceptualized as a sequence labeling task, whereas LLMs are optimized for text generation and reasoning tasks.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We developed an instruction-based learning paradigm that transforms biomedical NER from a sequence labeling task into a generation task. This paradigm is end-to-end and streamlines the training and evaluation process by automatically repurposing pre-existing biomedical NER datasets. We further developed BioNER-LLaMA using the proposed paradigm with LLaMA-7B as the foundational LLM. We conducted extensive testing on BioNER-LLaMA across three widely recognized biomedical NER datasets, consisting of entities related to diseases, chemicals, and genes. The results revealed that BioNER-LLaMA consistently achieved higher F1-scores ranging from 5% to 30% compared to the few-shot learning capabilities of GPT-4 on datasets with different biomedical entities. We show that a general-domain LLM can match the performance of rigorously fine-tuned PubMedBERT models and PMC-LLaMA, biomedical-specific language model. Our findings underscore the potential of our proposed paradigm in developing general-domain LLMs that can rival SOTA performances in multi-task, multi-domain scenarios in biomedical and health applications.<\/jats:p><\/jats:sec><jats:sec><jats:title>Availability and implementation<\/jats:title><jats:p>Datasets and other resources are available at https:\/\/github.com\/BIDS-Xu-Lab\/BioNER-LLaMA.<\/jats:p><\/jats:sec>","DOI":"10.1093\/bioinformatics\/btae163","type":"journal-article","created":{"date-parts":[[2024,3,22]],"date-time":"2024-03-22T03:21:30Z","timestamp":1711077690000},"source":"Crossref","is-referenced-by-count":89,"title":["Advancing entity recognition in biomedicine via instruction tuning of large language models"],"prefix":"10.1093","volume":"40","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6919-1122","authenticated-orcid":false,"given":"Vipina K","family":"Keloth","sequence":"first","affiliation":[{"name":"Section of Biomedical Informatics and Data Science, School of Medicine, Yale University , New Haven, CT-06510, United States"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2413-5918","authenticated-orcid":false,"given":"Yan","family":"Hu","sequence":"additional","affiliation":[{"name":"McWilliams School of Biomedical Informatics, University of Texas Health Science at Houston , Houston, TX-77030, United States"}]},{"given":"Qianqian","family":"Xie","sequence":"additional","affiliation":[{"name":"Section of Biomedical Informatics and Data Science, School of Medicine, Yale University , New Haven, CT-06510, United States"}]},{"given":"Xueqing","family":"Peng","sequence":"additional","affiliation":[{"name":"Section of Biomedical Informatics and Data Science, School of Medicine, Yale University , New Haven, CT-06510, United States"}]},{"given":"Yan","family":"Wang","sequence":"additional","affiliation":[{"name":"Section of Biomedical Informatics and Data Science, School of Medicine, Yale University , New Haven, CT-06510, United States"}]},{"given":"Andrew","family":"Zheng","sequence":"additional","affiliation":[{"name":"William P. Clements High School , Sugar Land, TX-77479, United States"}]},{"given":"Melih","family":"Selek","sequence":"additional","affiliation":[{"name":"Stephen F. Austin High School , Sugar Land, TX-77498, United States"}]},{"given":"Kalpana","family":"Raja","sequence":"additional","affiliation":[{"name":"Section of Biomedical Informatics and Data Science, School of Medicine, Yale University , New Haven, CT-06510, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5094-7321","authenticated-orcid":false,"given":"Chih Hsuan","family":"Wei","sequence":"additional","affiliation":[{"name":"National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health , Bethesda, MD-20894, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1268-7239","authenticated-orcid":false,"given":"Qiao","family":"Jin","sequence":"additional","affiliation":[{"name":"National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health , Bethesda, MD-20894, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9998-916X","authenticated-orcid":false,"given":"Zhiyong","family":"Lu","sequence":"additional","affiliation":[{"name":"National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health , Bethesda, MD-20894, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6036-1516","authenticated-orcid":false,"given":"Qingyu","family":"Chen","sequence":"additional","affiliation":[{"name":"Section of Biomedical Informatics and Data Science, School of Medicine, Yale University , New Haven, CT-06510, United States"},{"name":"National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health , Bethesda, MD-20894, United States"}]},{"given":"Hua","family":"Xu","sequence":"additional","affiliation":[{"name":"Section of Biomedical Informatics and Data Science, School of Medicine, Yale University , New Haven, CT-06510, United States"}]}],"member":"286","published-online":{"date-parts":[[2024,3,21]]},"reference":[{"key":"2024040900182630800_btae163-B1","author":"Achiam","year":"2023"},{"key":"2024040900182630800_btae163-B2","author":"Agrawal","year":"2022"},{"key":"2024040900182630800_btae163-B3","author":"Ashok","year":"2023"},{"key":"2024040900182630800_btae163-B4","author":"Beltagy"},{"key":"2024040900182630800_btae163-B5","author":"Biderman","year":"2023"},{"key":"2024040900182630800_btae163-B6","author":"Chaudhry","year":"2022"},{"key":"2024040900182630800_btae163-B7","author":"Chen","year":"2023"},{"key":"2024040900182630800_btae163-B8","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1186\/s12859-019-3321-4","article-title":"Biomedical named entity recognition using deep neural networks with contextual information","volume":"20","author":"Cho","year":"2019","journal-title":"BMC Bioinformatics"},{"key":"2024040900182630800_btae163-B9","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1186\/s12859-017-1776-8","article-title":"A neural network multi-task learning approach to biomedical named entity recognition","volume":"18","author":"Crichton","year":"2017","journal-title":"BMC Bioinformatics"},{"key":"2024040900182630800_btae163-B10","doi-asserted-by":"crossref","first-page":"3539","DOI":"10.1093\/bioinformatics\/bty356","article-title":"D3NER: biomedical named entity recognition using CRF-biLSTM improved with fine-tuned embeddings of various linguistic information","volume":"34","author":"Dang","year":"2018","journal-title":"Bioinformatics"},{"key":"2024040900182630800_btae163-B11","author":"Dettmers"},{"key":"2024040900182630800_btae163-B12","author":"Dettmers"},{"key":"2024040900182630800_btae163-B13","author":"Devlin"},{"key":"2024040900182630800_btae163-B14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jbi.2013.12.006","article-title":"NCBI disease corpus: a resource for disease name recognition and concept normalization","volume":"47","author":"Do\u011fan","year":"2014","journal-title":"J Biomed Inform"},{"key":"2024040900182630800_btae163-B15","author":"Fang","year":"2023"},{"key":"2024040900182630800_btae163-B16","first-page":"707","article-title":"Toward information extraction: identifying protein names from biological papers","author":"Fukuda","year":"1998","journal-title":"Pac Symp Biocomput"},{"key":"2024040900182630800_btae163-B17","author":"Gilardi"},{"key":"2024040900182630800_btae163-B18","doi-asserted-by":"crossref","first-page":"e45312","DOI":"10.2196\/45312","article-title":"How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment","volume":"9","author":"Gilson","year":"2023","journal-title":"JMIR Med Educ"},{"key":"2024040900182630800_btae163-B19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3458754","article-title":"Domain-specific language model pretraining for biomedical natural language processing","volume":"3","author":"Gu","year":"2021","journal-title":"ACM Trans Comput Healthcare"},{"key":"2024040900182630800_btae163-B20","author":"Hendy","year":"2023"},{"key":"2024040900182630800_btae163-B21","author":"Hu"},{"key":"2024040900182630800_btae163-B22","doi-asserted-by":"crossref","first-page":"ocad259","DOI":"10.1093\/jamia\/ocad259","article-title":"Improving large language models for clinical named entity recognition via prompt engineering","author":"Hu","year":"2024","journal-title":"J Am Med Inf Assoc"},{"key":"2024040900182630800_btae163-B23","author":"Huang","year":"2015"},{"key":"2024040900182630800_btae163-B24","doi-asserted-by":"crossref","first-page":"baac102","DOI":"10.1093\/database\/baac102","article-title":"NLM-Chem-BC7: manually annotated full-text resources for chemical entity annotation and indexing in biomedical articles","volume":"2022","author":"Islamaj","year":"2022","journal-title":"Database"},{"key":"2024040900182630800_btae163-B25","doi-asserted-by":"crossref","first-page":"103779","DOI":"10.1016\/j.jbi.2021.103779","article-title":"NLM-Gene, a richly annotated gold standard dataset for gene entities that addresses ambiguity and multi-species gene recognition","volume":"118","author":"Islamaj","year":"2021","journal-title":"J Biomed Inform"},{"key":"2024040900182630800_btae163-B26","author":"Ji","year":"2023"},{"key":"2024040900182630800_btae163-B27","author":"Khambete","year":"2021"},{"key":"2024040900182630800_btae163-B28","first-page":"22199","article-title":"Large language models are zero-shot reasoners","volume":"35","author":"Kojima","year":"2022","journal-title":"Adv Neural Inf Process Syst"},{"key":"2024040900182630800_btae163-B29","author":"Ko\u0161prdi\u0107","year":"2023"},{"key":"2024040900182630800_btae163-B30","author":"Labrak","year":"2023"},{"key":"2024040900182630800_btae163-B31","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"2024040900182630800_btae163-B32","doi-asserted-by":"crossref","first-page":"baw068","DOI":"10.1093\/database\/baw068","article-title":"BioCreative V CDR task corpus: a resource for chemical disease relation extraction","volume":"2016","author":"Li","year":"2016","journal-title":"Database"},{"key":"2024040900182630800_btae163-B33","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1186\/s12911-022-01967-7","article-title":"A comparative study of pre-trained language models for named entity recognition in clinical trial eligibility criteria from multiple corpora","volume":"22","author":"Li","year":"2022","journal-title":"BMC Med Inform Decis Mak"},{"key":"2024040900182630800_btae163-B34","author":"Loukas"},{"key":"2024040900182630800_btae163-B35","doi-asserted-by":"crossref","first-page":"btad310","DOI":"10.1093\/bioinformatics\/btad310","article-title":"AIONER: all-in-one scheme-based biomedical named entity recognition using deep learning","volume":"39","author":"Luo","year":"2023","journal-title":"Bioinformatics"},{"key":"2024040900182630800_btae163-B36","author":"Manakhimova","year":"2023"},{"key":"2024040900182630800_btae163-B37","author":"Margatina"},{"key":"2024040900182630800_btae163-B38","author":"Morwal"},{"key":"2024040900182630800_btae163-B39","author":"Peng"},{"key":"2024040900182630800_btae163-B40","author":"Ponomareva","year":"2007"},{"key":"2024040900182630800_btae163-B41","doi-asserted-by":"crossref","first-page":"104062","DOI":"10.1016\/j.jbi.2022.104062","article-title":"Effects of data and entity ablation on multitask learning models for biomedical entity recognition","volume":"130","author":"Rodriguez","year":"2022","journal-title":"J Biomed Inform"},{"key":"2024040900182630800_btae163-B42","doi-asserted-by":"crossref","first-page":"3191","DOI":"10.1093\/bioinformatics\/bti475","article-title":"ABNER: an open source tool for automatically tagging genes, proteins and other entity names in text","volume":"21","author":"Settles","year":"2005","journal-title":"Bioinformatics"},{"key":"2024040900182630800_btae163-B43","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.patrec.2021.02.008","article-title":"Self-attention-based conditional random fields latent variables model for sequence labeling","volume":"145","author":"Shao","year":"2021","journal-title":"Pattern Recognit Lett"},{"key":"2024040900182630800_btae163-B44","doi-asserted-by":"crossref","first-page":"S2","DOI":"10.1186\/gb-2008-9-s2-s2","article-title":"Overview of BioCreative II gene mention recognition","volume":"9 Suppl 2","author":"Smith","year":"2008","journal-title":"Genome Biol"},{"key":"2024040900182630800_btae163-B45","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1561\/2200000013","article-title":"An introduction to conditional random fields","volume":"4","author":"Sutton","year":"2012","journal-title":"Found Trends Mach Learn"},{"key":"2024040900182630800_btae163-B46","author":"Touvron","year":"2023"},{"key":"2024040900182630800_btae163-B47","author":"Touvron","year":"2023"},{"key":"2024040900182630800_btae163-B48","doi-asserted-by":"crossref","first-page":"S11","DOI":"10.1186\/1471-2105-7-S5-S11","article-title":"NERBio: using selected word conjunctions, term normalization, and global patterns to improve biomedical named entity recognition","volume":"7","author":"Tsai","year":"2006","journal-title":"BMC Bioinformatics"},{"key":"2024040900182630800_btae163-B49","author":"Tsuruoka","year":"2003"},{"key":"2024040900182630800_btae163-B50","author":"Wang"},{"key":"2024040900182630800_btae163-B51","author":"Wang","year":"2023"},{"key":"2024040900182630800_btae163-B52","author":"Wang","year":"2023"},{"key":"2024040900182630800_btae163-B53","doi-asserted-by":"crossref","first-page":"1745","DOI":"10.1093\/bioinformatics\/bty869","article-title":"Cross-type biomedical named entity recognition with deep multi-task learning","volume":"35","author":"Wang","year":"2019","journal-title":"Bioinformatics"},{"key":"2024040900182630800_btae163-B54","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1093\/bioinformatics\/btz528","article-title":"HUNER: improving biomedical NER with pretraining","volume":"36","author":"Weber","year":"2020","journal-title":"Bioinformatics"},{"key":"2024040900182630800_btae163-B55","author":"Wei"},{"key":"2024040900182630800_btae163-B56","author":"Wei","year":"2023"},{"key":"2024040900182630800_btae163-B57","author":"Wu","year":"2023"},{"key":"2024040900182630800_btae163-B58","doi-asserted-by":"crossref","first-page":"1105","DOI":"10.1109\/TCBB.2020.3020016","article-title":"Relation extraction from biomedical and clinical text: unified multitask learning framework","volume":"19","author":"Yadav","year":"2022","journal-title":"IEEE\/ACM Trans Comput Biol Bioinform"},{"key":"2024040900182630800_btae163-B59","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1016\/j.compbiolchem.2008.03.008","article-title":"Exploiting the performance of dictionary-based bio-entity name recognition in biomedical literature","volume":"32","author":"Yang","year":"2008","journal-title":"Comput Biol Chem"},{"key":"2024040900182630800_btae163-B60","author":"Zhang","year":"2023"},{"key":"2024040900182630800_btae163-B61","author":"Zhao"},{"key":"2024040900182630800_btae163-B62","author":"Zhou","year":"2021"},{"key":"2024040900182630800_btae163-B63","author":"Zhou","year":"2023"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btae163\/57056267\/btae163.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/40\/4\/btae163\/57186045\/btae163.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/40\/4\/btae163\/57186045\/btae163.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T17:09:59Z","timestamp":1731604199000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btae163\/7633405"}},"subtitle":[],"editor":[{"given":"Jonathan","family":"Wren","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2024,3,21]]},"references-count":63,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,3,29]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btae163","relation":{},"ISSN":["1367-4811"],"issn-type":[{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,4,1]]},"published":{"date-parts":[[2024,3,21]]},"article-number":"btae163"}}