{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T03:35:27Z","timestamp":1787888127250,"version":"build-2784847793"},"reference-count":41,"publisher":"Oxford University Press (OUP)","issue":"9","license":[{"start":{"date-parts":[[2024,6,21]],"date-time":"2024-06-21T00:00:00Z","timestamp":1718928000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008460","name":"National Center for Complementary and Integrative Health","doi-asserted-by":"publisher","award":["R01AT009457"],"award-info":[{"award-number":["R01AT009457"]}],"id":[{"id":"10.13039\/100008460","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000049","name":"National Institute on Aging","doi-asserted-by":"publisher","award":["R01AG078154"],"award-info":[{"award-number":["R01AG078154"]}],"id":[{"id":"10.13039\/100000049","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["R01CA287413"],"award-info":[{"award-number":["R01CA287413"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>To investigate the demonstration in large language models (LLMs) for biomedical relation extraction. This study introduces a framework comprising three types of adaptive tuning methods to assess their impacts and effectiveness.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>Our study was conducted in two phases. Initially, we analyzed a range of demonstration components vital for LLMs\u2019 biomedical data capabilities, including task descriptions and examples, experimenting with various combinations. Subsequently, we introduced the LLM instruction-example adaptive prompting (LEAP) framework, including instruction adaptive tuning, example adaptive tuning, and instruction-example adaptive tuning methods. This framework aims to systematically investigate both adaptive task descriptions and adaptive examples within the demonstration. We assessed the performance of the LEAP framework on the DDI, ChemProt, and BioRED datasets, employing LLMs such as Llama2-7b, Llama2-13b, and MedLLaMA_13B.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Our findings indicated that Instruction + Options + Example and its expanded form substantially improved F1 scores over the standard Instruction + Options mode for zero-shot LLMs. The LEAP framework, particularly through its example adaptive prompting, demonstrated superior performance over conventional instruction tuning across all models. Notably, the MedLLAMA_13B model achieved an exceptional F1 score of 95.13 on the ChemProt dataset using this method. Significant improvements were also observed in the DDI 2013 and BioRED datasets, confirming the method\u2019s robustness in sophisticated data extraction scenarios.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>The LEAP framework offers a compelling strategy for enhancing LLM training strategies, steering away from extensive fine-tuning towards more dynamic and contextually enriched prompting methodologies, showcasing in biomedical relation extraction.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocae147","type":"journal-article","created":{"date-parts":[[2024,6,3]],"date-time":"2024-06-03T19:10:38Z","timestamp":1717441838000},"page":"2010-2018","source":"Crossref","is-referenced-by-count":26,"title":["LEAP: LLM instruction-example adaptive prompting framework for biomedical relation extraction"],"prefix":"10.1093","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6524-5506","authenticated-orcid":false,"given":"Huixue","family":"Zhou","sequence":"first","affiliation":[{"name":"Institute for Health Informatics, University of Minnesota, Minneapolis, MN 55455, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mingchen","family":"Li","sequence":"additional","affiliation":[{"name":"Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8808-8371","authenticated-orcid":false,"given":"Yongkang","family":"Xiao","sequence":"additional","affiliation":[{"name":"Institute for Health Informatics, University of Minnesota, Minneapolis, MN 55455, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-4322-6753","authenticated-orcid":false,"given":"Han","family":"Yang","sequence":"additional","affiliation":[{"name":"Institute for Health Informatics, University of Minnesota, Minneapolis, MN 55455, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8258-3585","authenticated-orcid":false,"given":"Rui","family":"Zhang","sequence":"additional","affiliation":[{"name":"Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,6,21]]},"reference":[{"key":"2024082207523660700_ocae147-B1","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.jbi.2017.11.011","article-title":"Clinical information extraction applications: a literature review","volume":"77","author":"Wang","year":"2018","journal-title":"J Biomed Inform"},{"issue":"1","key":"2024082207523660700_ocae147-B2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3458754","article-title":"Domain-specific language model pretraining for biomedical natural language processing","volume":"3","author":"Gu","year":"2022","journal-title":"ACM Trans Comput Healthc"},{"issue":"4","key":"2024082207523660700_ocae147-B3","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"2024082207523660700_ocae147-B4","author":"Roy","year":"2021"},{"key":"2024082207523660700_ocae147-B5","author":"Beltagy","year":"2019"},{"key":"2024082207523660700_ocae147-B6","author":"Yasunaga","year":"2022"},{"issue":"2","key":"2024082207523660700_ocae147-B7","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1093\/jamia\/ocad216","article-title":"Complementary and integrative health information in the literature: its lexicon and named entity recognition","volume":"31","author":"Zhou","year":"2023","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2024082207523660700_ocae147-B8","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1038\/s43856-023-00370-1","article-title":"The future landscape of large language models in medicine","volume":"3","author":"Clusmann","year":"2023","journal-title":"Commun Med"},{"issue":"11","key":"2024082207523660700_ocae147-B9","first-page":"688","article-title":"Using large language models in psychology","volume":"2","author":"Demszky","year":"2023","journal-title":"Nat Rev Psychol"},{"key":"2024082207523660700_ocae147-B10","author":"Li","year":"2023"},{"issue":"2","key":"2024082207523660700_ocae147-B11","doi-asserted-by":"crossref","first-page":"e0000205","DOI":"10.1371\/journal.pdig.0000205","article-title":"ChatGPT passing USMLE shines a spotlight on the flaws of medical education","volume":"2","author":"Mbakwe","year":"2023","journal-title":"PLOS Digit Health"},{"issue":"7972","key":"2024082207523660700_ocae147-B12","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"issue":"1","key":"2024082207523660700_ocae147-B13","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1038\/s41746-023-00896-7","article-title":"Evaluating large language models on medical evidence summarization","volume":"6","author":"Tang","year":"2023","journal-title":"NPJ Digit Med"},{"issue":"8","key":"2024082207523660700_ocae147-B14","doi-asserted-by":"crossref","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat Med"},{"key":"2024082207523660700_ocae147-B15","author":"Zhang","year":"2023"},{"key":"2024082207523660700_ocae147-B16","author":"Lou","year":"2023"},{"key":"2024082207523660700_ocae147-B17","author":"Wei","year":"2021"},{"key":"2024082207523660700_ocae147-B18","author":"Li","year":"2023"},{"key":"2024082207523660700_ocae147-B19","author":"Prasad","year":"2023"},{"key":"2024082207523660700_ocae147-B20","author":"Zhou","year":"2022"},{"key":"2024082207523660700_ocae147-B21","author":"Wei","year":"2022"},{"key":"2024082207523660700_ocae147-B22","author":"Wan","year":"2023"},{"key":"2024082207523660700_ocae147-B23","author":"Chen","year":"2023"},{"key":"2024082207523660700_ocae147-B24","author":"Wadhwa","year":"2023"},{"key":"2024082207523660700_ocae147-B25","author":"Meng","year":"2023"},{"key":"2024082207523660700_ocae147-B26","author":"Xu","year":"2023"},{"key":"2024082207523660700_ocae147-B27","author":"Gao","year":"2023"},{"key":"2024082207523660700_ocae147-B28","author":"Rubin","year":"2021"},{"key":"2024082207523660700_ocae147-B29","author":"Liu","year":"2022"},{"key":"2024082207523660700_ocae147-B30","author":"Lester","year":"2021"},{"key":"2024082207523660700_ocae147-B31","author":"Liu","year":"2021"},{"key":"2024082207523660700_ocae147-B32","author":"Schick","year":"2021"},{"key":"2024082207523660700_ocae147-B33","author":"Liu","year":"2021"},{"issue":"5","key":"2024082207523660700_ocae147-B34","doi-asserted-by":"crossref","first-page":"914","DOI":"10.1016\/j.jbi.2013.07.011","article-title":"The DDI corpus: an annotated corpus with pharmacological substances and drug\u2013drug interactions","volume":"46","author":"Herrero-Zazo","year":"2013","journal-title":"J Biomed Inform"},{"key":"2024082207523660700_ocae147-B35","first-page":"141","author":"Krallinger"},{"issue":"5","key":"2024082207523660700_ocae147-B36","doi-asserted-by":"crossref","DOI":"10.1093\/bib\/bbac282","article-title":"BioRED: a rich biomedical relation extraction dataset","volume":"23","author":"Luo","year":"2022","journal-title":"Brief Bioinform"},{"key":"2024082207523660700_ocae147-B37","author":"Chaoyi-wu\/MedLLaMA_13B Hugging Face","year":"."},{"key":"2024082207523660700_ocae147-B38","author":"Touvron","year":"2023"},{"key":"2024082207523660700_ocae147-B39","author":"Hu","year":"2021"},{"key":"2024082207523660700_ocae147-B40","doi-asserted-by":"crossref","first-page":"104630","DOI":"10.1016\/j.jbi.2024.104630","article-title":"Model tuning or prompt Tuning? a study of large language models for clinical concept and relation extraction","volume":"153","author":"Peng","year":"2024","journal-title":"J Biomed Inform"},{"key":"2024082207523660700_ocae147-B41","author":"Peng","year":"2023"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/9\/2010\/58868346\/ocae147.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/9\/2010\/58868346\/ocae147.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,22]],"date-time":"2024-08-22T09:35:27Z","timestamp":1724319327000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/9\/2010\/7696965"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,21]]},"references-count":41,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2024,6,21]]},"published-print":{"date-parts":[[2024,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae147","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,9]]},"published":{"date-parts":[[2024,6,21]]}}}