{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,10]],"date-time":"2026-05-10T11:11:09Z","timestamp":1778411469675,"version":"3.51.4"},"reference-count":82,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,1,19]],"date-time":"2025-01-19T00:00:00Z","timestamp":1737244800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,19]],"date-time":"2025-01-19T00:00:00Z","timestamp":1737244800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000060","name":"U.S. Department of Health & Human Services | NIH | National Institute of Allergy and Infectious Diseases","doi-asserted-by":"publisher","award":["1R01AI17812101"],"award-info":[{"award-number":["1R01AI17812101"]}],"id":[{"id":"10.13039\/100000060","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Large language models (LLMs) with retrieval-augmented generation (RAG) have improved information extraction over previous methods, yet their reliance on embeddings often leads to inefficient retrieval. We introduce CLinical Entity Augmented Retrieval (CLEAR), a RAG pipeline that retrieves information using entities. We compared CLEAR to embedding RAG and full-note approaches for extracting 18 variables using six LLMs across 20,000 clinical notes. Average F1 scores were 0.90, 0.86, and 0.79; inference times were 4.95, 17.41, and 20.08\u2009s per note; average model queries were 1.68, 4.94, and 4.18 per note; and average input tokens were 1.1k, 3.8k, and 6.1k per note for CLEAR, embedding RAG, and full-note approaches, respectively. In conclusion, CLEAR utilizes clinical entities for information retrieval and achieves &gt;70% reduction in token usage and inference time with improved performance compared to modern methods.<\/jats:p>","DOI":"10.1038\/s41746-024-01377-1","type":"journal-article","created":{"date-parts":[[2025,1,19]],"date-time":"2025-01-19T05:21:01Z","timestamp":1737264061000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":45,"title":["Clinical entity augmented retrieval for clinical information extraction"],"prefix":"10.1038","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0246-2180","authenticated-orcid":false,"given":"Ivan","family":"Lopez","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3426-9289","authenticated-orcid":false,"given":"Akshay","family":"Swaminathan","sequence":"additional","affiliation":[]},{"given":"Karthik","family":"Vedula","sequence":"additional","affiliation":[]},{"given":"Sanjana","family":"Narayanan","sequence":"additional","affiliation":[]},{"given":"Fateme","family":"Nateghi Haredasht","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3738-9569","authenticated-orcid":false,"given":"Stephen P.","family":"Ma","sequence":"additional","affiliation":[]},{"given":"April S.","family":"Liang","sequence":"additional","affiliation":[]},{"given":"Steven","family":"Tate","sequence":"additional","affiliation":[]},{"given":"Manoj","family":"Maddali","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2601-0173","authenticated-orcid":false,"given":"Robert Joseph","family":"Gallo","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9385-7158","authenticated-orcid":false,"given":"Nigam H.","family":"Shah","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4387-8740","authenticated-orcid":false,"given":"Jonathan H.","family":"Chen","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,1,19]]},"reference":[{"key":"1377_CR1","first-page":"97","volume":"9","author":"MK Ross","year":"2014","unstructured":"Ross, M. K., Wei, W. & Ohno-Machado, L. Big data and the electronic health record. Yearb. Med. Inform. 9, 97\u2013104 (2014).","journal-title":"Yearb. Med. Inform."},{"key":"1377_CR2","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1055\/s-0038-1638592","volume":"17","author":"SM Meystre","year":"2008","unstructured":"Meystre, S. M., Savova, G. K., Kipper-Schuler, K. C. & Hurdle, J. F. Extracting information from textual documents in the electronic health record: a review of recent research. Yearb. Med. Inform. 17, 128\u2013144 (2008).","journal-title":"Yearb. Med. Inform."},{"key":"1377_CR3","doi-asserted-by":"publisher","unstructured":"Wornow, M. et al. Zero-shot clinical trial patient matching with LLMs. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2402.05125 (2024).","DOI":"10.48550\/arXiv.2402.05125"},{"key":"1377_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-023-00957-x","volume":"6","author":"E Alsentzer","year":"2023","unstructured":"Alsentzer, E. et al. Zero-shot interpretable phenotyping of postpartum hemorrhage using large language models. Npj Digit. Med. 6, 1\u201310 (2023).","journal-title":"Npj Digit. Med."},{"key":"1377_CR5","doi-asserted-by":"publisher","first-page":"S79","DOI":"10.7326\/M19-0873","volume":"172","author":"A Callahan","year":"2020","unstructured":"Callahan, A., Shah, N. H. & Chen, J. H. Research and reporting considerations for observational studies using electronic health record data. Ann. Intern. Med. 172, S79\u2013S84 (2020).","journal-title":"Ann. Intern. Med."},{"key":"1377_CR6","first-page":"1067","volume":"2023","author":"I Lopez","year":"2024","unstructured":"Lopez, I. et al. Predicting premature discontinuation of medication for opioid use disorder from electronic medical records. Amia. Annu. Symp. Proc. 2023, 1067\u20131076 (2024).","journal-title":"Amia. Annu. Symp. Proc."},{"key":"1377_CR7","doi-asserted-by":"publisher","first-page":"358","DOI":"10.1093\/bib\/bbm045","volume":"8","author":"P Zweigenbaum","year":"2007","unstructured":"Zweigenbaum, P., Demner-Fushman, D., Yu, H. & Cohen, K. B. Frontiers of biomedical text mining: current progress. Brief. Bioinform. 8, 358\u2013375 (2007).","journal-title":"Brief. Bioinform."},{"key":"1377_CR8","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1016\/j.jbi.2017.11.011","volume":"77","author":"Y Wang","year":"2018","unstructured":"Wang, Y. et al. Clinical information extraction applications: a literature review. J. Biomed. Inform. 77, 34\u201349 (2018).","journal-title":"J. Biomed. Inform."},{"key":"1377_CR9","doi-asserted-by":"publisher","unstructured":"Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K. & Dyer, C. Neural Architectures for Named Entity Recognition. in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (eds. Knight, K., Nenkova, A. & Rambow, O.) 260\u2013270 https:\/\/doi.org\/10.18653\/v1\/N16-1030 (Association for Computational Linguistics, San Diego, California, 2016).","DOI":"10.18653\/v1\/N16-1030"},{"key":"1377_CR10","unstructured":"K\u00e5geb\u00e4ck, M. & Salomonsson, H. Word Sense Disambiguation using a Bidirectional LSTM. in Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex - V) (eds. Zock, M., Lenci, A. & Evert, S.) 51\u201356 (The COLING 2016 Organizing Committee, Osaka, Japan, 2016)."},{"key":"1377_CR11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-022-00730-6","volume":"5","author":"H Wu","year":"2022","unstructured":"Wu, H. et al. A survey on clinical natural language processing in the United Kingdom from 2007 to 2022. Npj Digit. Med. 5, 1\u201315 (2022).","journal-title":"Npj Digit. Med."},{"key":"1377_CR12","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1136\/amiajnl-2014-002902","volume":"22","author":"K Jung","year":"2015","unstructured":"Jung, K. et al. Functional evaluation of out-of-the-box text-mining tools for data-mining tasks. J. Am. Med. Inform. Assoc. JAMIA 22, 121\u2013131 (2015).","journal-title":"J. Am. Med. Inform. Assoc. JAMIA"},{"key":"1377_CR13","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1146\/annurev-biodatasci-030421-030931","volume":"4","author":"B Percha","year":"2021","unstructured":"Percha, B. Modern clinical text mining: a guide and review. Annu. Rev. Biomed. Data Sci. 4, 165\u2013187 (2021).","journal-title":"Annu. Rev. Biomed. Data Sci."},{"key":"1377_CR14","doi-asserted-by":"publisher","first-page":"e2312538","DOI":"10.1001\/jamanetworkopen.2023.12538","volume":"6","author":"AR Agarwal","year":"2023","unstructured":"Agarwal, A. R., Prichett, L., Jain, A. & Srikumaran, U. Assessment of use of ICD-9 and ICD-10 codes for social determinants of health in the US, 2011-2021. JAMA Netw. Open 6, e2312538 (2023).","journal-title":"JAMA Netw. Open"},{"key":"1377_CR15","doi-asserted-by":"publisher","first-page":"1037","DOI":"10.1097\/MLR.0000000000001418","volume":"58","author":"HP Truong","year":"2020","unstructured":"Truong, H. P. et al. Utilization of social determinants of health ICD-10 Z-codes among hospitalized patients in the United States, 2016\u20132017. Med. Care 58, 1037 (2020).","journal-title":"Med. Care"},{"key":"1377_CR16","doi-asserted-by":"publisher","first-page":"188","DOI":"10.1093\/jamia\/ocad182","volume":"31","author":"A Swaminathan","year":"2024","unstructured":"Swaminathan, A. et al. Selective prediction for extracting unstructured clinical data. J. Am. Med. Inform. Assoc. 31, 188\u2013197 (2024).","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"1377_CR17","doi-asserted-by":"publisher","first-page":"S51","DOI":"10.1097\/01.XCS.0000893308.54894.46","volume":"235","author":"C Liu","year":"2022","unstructured":"Liu, C. et al. Predictive value of clinical complete response after chemoradiation for rectal cancer. J. Am. Coll. Surg. 235, S51 (2022).","journal-title":"J. Am. Coll. Surg."},{"key":"1377_CR18","doi-asserted-by":"publisher","first-page":"h1885","DOI":"10.1136\/bmj.h1885","volume":"350","author":"KP Liao","year":"2015","unstructured":"Liao, K. P. et al. Development of phenotype algorithms using electronic medical records and incorporating natural language processing. BMJ 350, h1885 (2015).","journal-title":"BMJ"},{"key":"1377_CR19","unstructured":"Huang, Z., Xu, W. & Yu, K. Bidirectional LSTM-CRF models for sequence tagging. Preprint at http:\/\/arxiv.org\/abs\/1508.01991 (2015)."},{"key":"1377_CR20","doi-asserted-by":"publisher","first-page":"1892","DOI":"10.1093\/jamia\/ocab090","volume":"28","author":"Y Zhang","year":"2021","unstructured":"Zhang, Y., Zhang, Y., Qi, P., Manning, C. D. & Langlotz, C. P. Biomedical and clinical English model packages for the Stanza Python NLP library. J. Am. Med. Inform. Assoc. JAMIA 28, 1892\u20131899 (2021).","journal-title":"J. Am. Med. Inform. Assoc. JAMIA"},{"key":"1377_CR21","doi-asserted-by":"publisher","unstructured":"Qi, P., Zhang, Y., Zhang, Y., Bolton, J. & Manning, C. D. Stanza: a Python natural language processing toolkit for many human languages. in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations (eds. Celikyilmaz, A. & Wen, T.-H.) 101\u2013108 (Association for Computational Linguistics, Online). https:\/\/doi.org\/10.18653\/v1\/2020.acl-demos.14 (2020).","DOI":"10.18653\/v1\/2020.acl-demos.14"},{"key":"1377_CR22","doi-asserted-by":"publisher","first-page":"552","DOI":"10.1136\/amiajnl-2011-000203","volume":"18","author":"\u00d6 Uzuner","year":"2011","unstructured":"Uzuner, \u00d6., South, B. R., Shen, S. & DuVall, S. L. 2010 i2b2\/VA challenge on concepts, assertions, and relations in clinical text. J. Am. Med. Inform. Assoc. JAMIA 18, 552\u2013556 (2011).","journal-title":"J. Am. Med. Inform. Assoc. JAMIA"},{"key":"1377_CR23","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-22328-4","volume":"12","author":"JA Fries","year":"2021","unstructured":"Fries, J. A. et al. Ontology-driven weak supervision for clinical entity classification in electronic health records. Nat. Commun. 12, 2017 (2021).","journal-title":"Nat. Commun."},{"key":"1377_CR24","doi-asserted-by":"publisher","first-page":"e28","DOI":"10.1055\/s-0042-1742388","volume":"61","author":"A Jaber","year":"2022","unstructured":"Jaber, A. & Mart\u00ednez, P. Disambiguating clinical abbreviations using a one-fits-all classifier based on deep learning techniques. Methods Inf. Med. 61, e28\u2013e34 (2022).","journal-title":"Methods Inf. Med."},{"key":"1377_CR25","doi-asserted-by":"crossref","unstructured":"Gu, Y. et al. Domain-specific language model pretraining for biomedical natural language processing. ACM Trans Comput. Healthc. 3, 1\u201323 (2022).","DOI":"10.1145\/3458754"},{"key":"1377_CR26","unstructured":"Zambrano Chaves, J. et al. RaLEs: a Benchmark for Radiology Language Evaluations. in Advances in Neural Information Processing Systems (eds. Oh, A. et al.) vol. 36 74429\u201374454 (Curran Associates, Inc., 2023)."},{"key":"1377_CR27","doi-asserted-by":"publisher","first-page":"e210258","DOI":"10.1148\/ryai.210258","volume":"4","author":"A Yan","year":"2022","unstructured":"Yan, A. et al. RadBERT: adapting transformer-based language models to radiology. Radiol. Artif. Intell. 4, e210258 (2022).","journal-title":"Radiol. Artif. Intell."},{"key":"1377_CR28","unstructured":"Sushil, M., Ludwig, D., Butte, A. J. & Rudrapatna, V. A. Developing a general-purpose clinical language inference model from a large corpus of clinical notes. arXiv.org https:\/\/arxiv.org\/abs\/2210.06566v1 (2022)."},{"key":"1377_CR29","doi-asserted-by":"publisher","unstructured":"Lin, B. Y. et al. Differentiable Open-Ended Commonsense Reasoning. in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (eds. Toutanova, K. et al.) 4611\u20134625. https:\/\/doi.org\/10.18653\/v1\/2021.naacl-main.366 (Association for Computational Linguistics, Online, 2021).","DOI":"10.18653\/v1\/2021.naacl-main.366"},{"key":"1377_CR30","unstructured":"Klein, T. & Nabi, M. Attention is (not) all you need for commonsense reasoning. https:\/\/arxiv.org\/abs\/1905.13497v1 (2019)."},{"key":"1377_CR31","doi-asserted-by":"publisher","unstructured":"Li, L., Xin, X. & Guo, P. The exploration of the reasoning capability of BERT in relation extraction. in 2020 10th International Conference on Information Science and Technology (ICIST) 219\u2013228. https:\/\/doi.org\/10.1109\/ICIST49303.2020.9202183 (2020).","DOI":"10.1109\/ICIST49303.2020.9202183"},{"key":"1377_CR32","unstructured":"Amirizaniani, M., Martin, E., Sivachenko, M., Mashhadi, A. & Shah, C. Do LLMs exhibit human-like reasoning? Evaluating theory of mind in LLMs for open-ended responses. https:\/\/arxiv.org\/abs\/2406.05659v1 (2024)."},{"key":"1377_CR33","doi-asserted-by":"publisher","DOI":"10.1056\/AIdbp2300110","volume":"1","author":"M Sushil","year":"2024","unstructured":"Sushil, M. et al. CORAL: Expert-curated oncology reports to advance language model inference. NEJM AI 1, AIdbp2300110 (2024).","journal-title":"NEJM AI"},{"key":"1377_CR34","doi-asserted-by":"publisher","unstructured":"Van Veen, D. et al. Adapted large language models can outperform medical experts in clinical text summarization. Nat. Med. 1\u20139 https:\/\/doi.org\/10.1038\/s41591-024-02855-5 (2024).","DOI":"10.1038\/s41591-024-02855-5"},{"key":"1377_CR35","doi-asserted-by":"publisher","unstructured":"Tu, T. et al. Towards conversational diagnostic AI. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2401.05654 (2024).","DOI":"10.48550\/arXiv.2401.05654"},{"key":"1377_CR36","unstructured":"Brown, T. et al. Language Models are Few-Shot Learners. in Advances in Neural Information Processing Systems vol. 33 1877\u20131901 (Curran Associates, Inc., 2020)."},{"key":"1377_CR37","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1038\/s41586-023-05881-4","volume":"616","author":"M Moor","year":"2023","unstructured":"Moor, M. et al. Foundation models for generalist medical artificial intelligence. Nature 616, 259\u2013265 (2023).","journal-title":"Nature"},{"key":"1377_CR38","doi-asserted-by":"publisher","unstructured":"Agrawal, M., Hegselmann, S., Lang, H., Kim, Y. & Sontag, D. Large language models are few-shot clinical information extractors. in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (eds. Goldberg, Y., Kozareva, Z. & Zhang, Y.) 1998\u20132022. https:\/\/doi.org\/10.18653\/v1\/2022.emnlp-main.130 (Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 2022).","DOI":"10.18653\/v1\/2022.emnlp-main.130"},{"key":"1377_CR39","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-023-00970-0","volume":"7","author":"M Guevara","year":"2024","unstructured":"Guevara, M. et al. Large language models to identify social determinants of health in electronic health records. Npj Digit. Med. 7, 1\u201314 (2024).","journal-title":"Npj Digit. Med."},{"key":"1377_CR40","unstructured":"Goel, A. et al. LLMs Accelerate Annotation for Medical Information Extraction. in Proceedings of the 3rd Machine Learning for Health Symposium 82\u2013100 (PMLR, 2023)."},{"key":"1377_CR41","unstructured":"Mahbub, M. et al. Leveraging large language models to extract information on substance use disorder severity from clinical notes: a zero-shot learning approach. Preprint at http:\/\/arxiv.org\/abs\/2403.12297 (2024)."},{"key":"1377_CR42","doi-asserted-by":"publisher","unstructured":"Levy, M., Jacoby, A. & Goldberg, Y. Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models. in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (eds. Ku, L.-W., Martins, A. & Srikumar, V.) 15339\u201315353. https:\/\/doi.org\/10.18653\/v1\/2024.acl-long.818 (Association for Computational Linguistics, Bangkok, Thailand, 2024).","DOI":"10.18653\/v1\/2024.acl-long.818"},{"key":"1377_CR43","doi-asserted-by":"crossref","unstructured":"Shaham, U., Ivgi, M., Efrat, A., Berant, J. & Levy, O. ZeroSCROLLS: a zero-shot benchmark for long text understanding. https:\/\/arxiv.org\/abs\/2305.14196v3 (2023).","DOI":"10.18653\/v1\/2023.findings-emnlp.536"},{"key":"1377_CR44","doi-asserted-by":"crossref","unstructured":"Bai, Y. et al. LongBench: A bilingual, multitask benchmark for long context understanding. https:\/\/arxiv.org\/abs\/2308.14508v1 (2023).","DOI":"10.18653\/v1\/2024.acl-long.172"},{"key":"1377_CR45","doi-asserted-by":"crossref","unstructured":"Li, J., Wang, M., Zheng, Z. & Zhang, M. LooGLE: Can long-context language models understand long contexts? https:\/\/arxiv.org\/abs\/2311.04939v1 (2023).","DOI":"10.18653\/v1\/2024.acl-long.859"},{"key":"1377_CR46","doi-asserted-by":"publisher","unstructured":"Gao, Y. et al. Retrieval-augmented generation for large language models: a survey. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2312.10997 (2024).","DOI":"10.48550\/arXiv.2312.10997"},{"key":"1377_CR47","doi-asserted-by":"publisher","unstructured":"Agrawal, G., Kumarage, T., Alghamdi, Z. & Liu, H. Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey. in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (eds. Duh, K., Gomez, H. & Bethard, S.) 3947\u20133960. https:\/\/doi.org\/10.18653\/v1\/2024.naacl-long.219. (Association for Computational Linguistics, Mexico City, Mexico, 2024).","DOI":"10.18653\/v1\/2024.naacl-long.219"},{"key":"1377_CR48","unstructured":"Wu, Y. et al. Retrieve-rewrite-answer: a KG-to-text enhanced LLMs framework for knowledge graph question answering. Preprint at http:\/\/arxiv.org\/abs\/2309.11206 (2023)."},{"key":"1377_CR49","doi-asserted-by":"publisher","unstructured":"Baek, J., Aji, A. F. & Saffari, A. Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering. in Proceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE) (eds. Dalvi Mishra, B., Durrett, G., Jansen, P., Neves Ribeiro, D. & Wei, J.) 78\u2013106. https:\/\/doi.org\/10.18653\/v1\/2023.nlrse-1.7. (Association for Computational Linguistics, Toronto, Canada, 2023).","DOI":"10.18653\/v1\/2023.nlrse-1.7"},{"key":"1377_CR50","doi-asserted-by":"publisher","unstructured":"Li, Y., Dong, B., Guerin, F. & Lin, C. Compressing Context to Enhance Inference Efficiency of Large Language Models. in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (eds. Bouamor, H., Pino, J. & Bali, K.) 6342\u20136353. https:\/\/doi.org\/10.18653\/v1\/2023.emnlp-main.391 (Association for Computational Linguistics, Singapore, 2023).","DOI":"10.18653\/v1\/2023.emnlp-main.391"},{"key":"1377_CR51","unstructured":"Mialon, G. et al. Augmented language models: a survey. Preprint at http:\/\/arxiv.org\/abs\/2302.07842 (2023)."},{"key":"1377_CR52","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1162\/tacl_a_00638","volume":"12","author":"NF Liu","year":"2024","unstructured":"Liu, N. F. et al. Lost in the Middle: How Language Models Use Long Contexts. Trans. Assoc. Comput. Linguist. 12, 157\u2013173 (2024).","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"1377_CR53","unstructured":"Zhao, P. et al. Retrieval-augmented generation for AI-generated content: a survey. Preprint at http:\/\/arxiv.org\/abs\/2402.19473 (2024)."},{"key":"1377_CR54","doi-asserted-by":"publisher","unstructured":"Das, R. et al. Case-based Reasoning for Natural Language Queries over Knowledge Bases. in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (eds. Moens, M.-F., Huang, X., Specia, L. & Yih, S. W.) 9594\u20139611. https:\/\/doi.org\/10.18653\/v1\/2021.emnlp-main.755. (Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 2021).","DOI":"10.18653\/v1\/2021.emnlp-main.755"},{"key":"1377_CR55","unstructured":"Hu, X., Wu, X., Shu, Y. & Qu, Y. Logical Form Generation via Multi-task Learning for Complex Question Answering over Knowledge Bases. in Proceedings of the 29th International Conference on Computational Linguistics (eds. Calzolari, N. et al.) 1687\u20131696 (International Committee on Computational Linguistics, Gyeongju, Republic of Korea, 2022)."},{"key":"1377_CR56","doi-asserted-by":"publisher","unstructured":"Zhang, L. et al. FC-KBQA: A fine-to-coarse composition framework for knowledge base question answering. in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (eds. Rogers, A., Boyd-Graber, J. & Okazaki, N.) 1002\u20131017 (Association for Computational Linguistics, Toronto, Canada, 2023). https:\/\/doi.org\/10.18653\/v1\/2023.acl-long.57 (2023).","DOI":"10.18653\/v1\/2023.acl-long.57"},{"key":"1377_CR57","doi-asserted-by":"publisher","unstructured":"Jiang, J. et al. StructGPT: A General Framework for Large Language Model to Reason over Structured Data. in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (eds. Bouamor, H., Pino, J. & Bali, K.) 9237\u20139251 (Association for Computational Linguistics, Singapore, 2023). https:\/\/doi.org\/10.18653\/v1\/2023.emnlp-main.574 (2023).","DOI":"10.18653\/v1\/2023.emnlp-main.574"},{"key":"1377_CR58","doi-asserted-by":"publisher","unstructured":"Xu, Z. et al. Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering. in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval 2905\u20132909. https:\/\/doi.org\/10.1145\/3626772.3661370. (Association for Computing Machinery, New York, NY, USA, 2024).","DOI":"10.1145\/3626772.3661370"},{"key":"1377_CR59","doi-asserted-by":"publisher","first-page":"7063","DOI":"10.1007\/s40747-024-01527-8","volume":"10","author":"T Guo","year":"2024","unstructured":"Guo, T. et al. KnowledgeNavigator: leveraging large language models for enhanced reasoning over knowledge graph. Complex Intell. Syst. 10, 7063\u20137076 (2024).","journal-title":"Complex Intell. Syst."},{"key":"1377_CR60","doi-asserted-by":"publisher","unstructured":"Ji, Z. et al. RHO: reducing hallucination in open-domain dialogues with knowledge grounding. in Findings of the Association for Computational Linguistics: ACL 2023 (eds. Rogers, A., Boyd-Graber, J. & Okazaki, N.) 4504\u20134522 (Association for Computational Linguistics, Toronto, Canada, 2023). https:\/\/doi.org\/10.18653\/v1\/2023.findings-acl.275 (2023).","DOI":"10.18653\/v1\/2023.findings-acl.275"},{"key":"1377_CR61","doi-asserted-by":"publisher","first-page":"32862","DOI":"10.1109\/ACCESS.2020.2973928","volume":"8","author":"T Al-Moslmi","year":"2020","unstructured":"Al-Moslmi, T., Gallofr\u00e9 Oca\u00f1a, M., Opdahl, A. L. & Veres, C. Named entity extraction for knowledge graphs: a literature overview. IEEE Access 8, 32862\u201332881 (2020).","journal-title":"IEEE Access"},{"key":"1377_CR62","first-page":"22021","volume":"38","author":"SL Fleming","year":"2024","unstructured":"Fleming, S. L. et al. MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records. Proc. AAAI Conf. Artif. Intell. 38, 22021\u201322030 (2024).","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"1377_CR63","doi-asserted-by":"publisher","first-page":"817","DOI":"10.1001\/jamainternmed.2020.1562","volume":"180","author":"S Galea","year":"2020","unstructured":"Galea, S., Merchant, R. M. & Lurie, N. The mental health consequences of COVID-19 and physical distancing: the need for prevention and early intervention. JAMA Intern. Med. 180, 817\u2013818 (2020).","journal-title":"JAMA Intern. Med."},{"key":"1377_CR64","doi-asserted-by":"publisher","first-page":"1792","DOI":"10.1111\/add.16587","volume":"119","author":"F Nateghi Haredasht","year":"2024","unstructured":"Nateghi Haredasht, F. et al. Predictability of buprenorphine-naloxone treatment retention: A multi-site analysis combining electronic health records and machine learning. Addiction 119, 1792\u20131802 (2024).","journal-title":"Addiction"},{"key":"1377_CR65","doi-asserted-by":"publisher","unstructured":"Datta, S. et al. A new paradigm for accelerating clinical data science at Stanford Medicine. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2003.10534 (2020).","DOI":"10.48550\/arXiv.2003.10534"},{"key":"1377_CR66","first-page":"590","volume":"33","author":"J Irvin","year":"2019","unstructured":"Irvin, J. et al. CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. Proc. AAAI Conf. Artif. Intell. 33, 590\u2013597 (2019).","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"1377_CR67","first-page":"1","volume":"25","author":"HW Chung","year":"2024","unstructured":"Chung, H. W. et al. Scaling Instruction-Finetuned Language Models. J. Mach. Learn. Res. 25, 1\u201353 (2024).","journal-title":"J. Mach. Learn. Res."},{"key":"1377_CR68","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jbi.2013.12.006","volume":"47","author":"RI Do\u011fan","year":"2014","unstructured":"Do\u011fan, R. I., Leaman, R. & Lu, Z. NCBI disease corpus: a resource for disease name recognition and concept normalization. J. Biomed. Inform. 47, 1\u201310 (2014).","journal-title":"J. Biomed. Inform."},{"key":"1377_CR69","doi-asserted-by":"publisher","unstructured":"Alsentzer, E. et al. Publicly Available Clinical BERT Embeddings. in Proceedings of the 2nd Clinical Natural Language Processing Workshop (eds. Rumshisky, A., Roberts, K., Bethard, S. & Naumann, T.) 72\u201378. https:\/\/doi.org\/10.18653\/v1\/W19-1909. (Association for Computational Linguistics, Minneapolis, Minnesota, USA, 2019).","DOI":"10.18653\/v1\/W19-1909"},{"key":"1377_CR70","doi-asserted-by":"publisher","first-page":"D267","DOI":"10.1093\/nar\/gkh061","volume":"32","author":"O Bodenreider","year":"2004","unstructured":"Bodenreider, O. The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res 32, D267\u2013D270 (2004).","journal-title":"Nucleic Acids Res"},{"key":"1377_CR71","unstructured":"Stearns, M. Q., Price, C., Spackman, K. A. & Wang, A. Y. SNOMED clinical terms: overview of the development process and project status. Proc. AMIA Symp. 2001, 662\u2013666 (2001)."},{"key":"1377_CR72","first-page":"438","volume":"2021","author":"H Eyre","year":"2022","unstructured":"Eyre, H. et al. Launching into clinical space with medspaCy: a new clinical text processing toolkit in Python. AMIA. Annu. Symp. Proc. 2021, 438\u2013447 (2022).","journal-title":"AMIA. Annu. Symp. Proc."},{"key":"1377_CR73","doi-asserted-by":"publisher","unstructured":"Christophe, C. et al. Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approaches. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2404.14779 (2024).","DOI":"10.48550\/arXiv.2404.14779"},{"key":"1377_CR74","unstructured":"AI, M. Mixtral of Experts. https:\/\/mistral.ai\/news\/mixtral-of-experts\/ (2023)."},{"key":"1377_CR75","doi-asserted-by":"publisher","unstructured":"Grattafiori, A. et al. The Llama 3 Herd of Models. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2407.21783 (2024).","DOI":"10.48550\/arXiv.2407.21783"},{"key":"1377_CR76","unstructured":"Tay, Y. et al. Unifying Language Learning Paradigms. https:\/\/arxiv.org\/abs\/2205.05131 (2023)."},{"key":"1377_CR77","unstructured":"OpenAI et al. GPT-4 Technical Report. https:\/\/arxiv.org\/abs\/2303.08774v6 (2023)."},{"key":"1377_CR78","unstructured":"Dong, Q. et al. A survey on in-context learning. https:\/\/arxiv.org\/abs\/2301.00234v3 (2022)."},{"key":"1377_CR79","doi-asserted-by":"publisher","first-page":"269","DOI":"10.14778\/3157794.3157797","volume":"11","author":"A Ratner","year":"2017","unstructured":"Ratner, A. et al. Snorkel: rapid training data creation with weak supervision. Proc. VLDB Endow. 11, 269\u2013282 (2017).","journal-title":"Proc. VLDB Endow."},{"key":"1377_CR80","unstructured":"OpenAI is an AI research and deployment company. Terms of Use. https:\/\/openai.com\/policies\/terms-of-use\/ (2023)."},{"key":"1377_CR81","doi-asserted-by":"publisher","unstructured":"Xiao, S., Liu, Z., Zhang, P. & Muennighoff, N. C-Pack: packaged resources to advance general Chinese embedding. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2309.07597 (2023).","DOI":"10.48550\/arXiv.2309.07597"},{"key":"1377_CR82","doi-asserted-by":"publisher","first-page":"80","DOI":"10.2307\/3001968","volume":"1","author":"F Wilcoxon","year":"1945","unstructured":"Wilcoxon, F. Individual comparisons by ranking methods. Biom. Bull. 1, 80\u201383 (1945).","journal-title":"Biom. Bull."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01377-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01377-1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01377-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,19]],"date-time":"2025-01-19T05:21:12Z","timestamp":1737264072000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-024-01377-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,19]]},"references-count":82,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["1377"],"URL":"https:\/\/doi.org\/10.1038\/s41746-024-01377-1","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,19]]},"assertion":[{"value":"1 July 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 December 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 January 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A.S. owns stock in Roche (RHHVF) and Cerebral Inc. and is an adviser to Daybreak Health and Cerebral Inc. S.N. owns stock in Meta, works at Insitro (an ML for drug discovery company), and owns stock options for Insitro. N.H.S. reported being a co-founder of Prealize Health (a predictive analytics company) and Atropos Health (an on-demand evidence generation company); receiving funding from the Gordon and Betty Moore Foundation for developing virtual model deployments; and serving on the Board of the Coalition for Healthcare AI (CHAI), a consensus-building organization providing guidelines for the responsible use of artificial intelligence in healthcare. J.H.C. reported being a co-founder of Reaction Explorer LLC, develops and licenses organic chemistry education software, paid consulting fees from Sutton Pierce, Younker Hyde MacFarlane, and Sykes McAllister as a medical expert witness, and paid consulting fees from ISHI Health. The remaining authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"45"}}