{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T14:37:17Z","timestamp":1784644637994,"version":"3.55.0"},"reference-count":65,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,3,7]],"date-time":"2025-03-07T00:00:00Z","timestamp":1741305600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,3,7]],"date-time":"2025-03-07T00:00:00Z","timestamp":1741305600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100020884","name":"Agencia Nacional de Investigaci\u00f3n y Desarrollo","doi-asserted-by":"publisher","award":["ICN17_002"],"award-info":[{"award-number":["ICN17_002"]}],"id":[{"id":"10.13039\/501100020884","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100020884","name":"Agencia Nacional de Investigaci\u00f3n y Desarrollo","doi-asserted-by":"publisher","award":["ICN17_002"],"award-info":[{"award-number":["ICN17_002"]}],"id":[{"id":"10.13039\/501100020884","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100020884","name":"Agencia Nacional de Investigaci\u00f3n y Desarrollo","doi-asserted-by":"publisher","award":["ICN17_002"],"award-info":[{"award-number":["ICN17_002"]}],"id":[{"id":"10.13039\/501100020884","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100021778","name":"Agencia Nacional de Promoci\u00f3n de la Investigaci\u00f3n, el Desarrollo Tecnol\u00f3gico y la Innovaci\u00f3n","doi-asserted-by":"publisher","award":["1241825"],"award-info":[{"award-number":["1241825"]}],"id":[{"id":"10.13039\/501100021778","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Background<\/jats:title>\n            <jats:p>Clinical decision-making in healthcare often relies on unstructured text data, which can be challenging to analyze using traditional methods. Natural Language Processing (NLP) has emerged as a promising solution, but its application in clinical settings is hindered by restricted data availability and the need for domain-specific knowledge.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>We conducted an experimental analysis to evaluate the performance of various NLP modeling paradigms on multiple clinical NLP tasks in Spanish. These tasks included referral prioritization and referral specialty classification. We simulated three clinical settings with varying levels of data availability and evaluated the performance of four foundation models.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>Clinical-specific pre-trained language models (PLMs) achieved the highest performance across tasks. For referral prioritization, Clinical PLMs attained an 88.85 % macro F1 score when fine-tuned. In referral specialty classification, the same models achieved a 53.79 % macro F1 score, surpassing domain-agnostic models. Continuing pre-training with environment-specific data improved model performance, but the gains were marginal compared to the computational resources required. Few-shot learning with large language models (LLMs) demonstrated lower performance but showed potential in data-scarce scenarios.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusions<\/jats:title>\n            <jats:p>Our study provides evidence-based recommendations for clinical NLP practitioners on selecting modeling paradigms based on data availability. We highlight the importance of considering data availability, task complexity, and institutional maturity when designing and training clinical NLP models. Our findings can inform the development of effective clinical NLP solutions in real-world settings.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1186\/s12911-025-02948-2","type":"journal-article","created":{"date-parts":[[2025,3,7]],"date-time":"2025-03-07T13:21:29Z","timestamp":1741353689000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["NLP modeling recommendations for restricted data availability in clinical settings"],"prefix":"10.1186","volume":"25","author":[{"given":"Fabi\u00e1n","family":"Villena","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felipe","family":"Bravo-Marquez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jocelyn","family":"Dunstan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,3,7]]},"reference":[{"key":"2948_CR1","unstructured":"Jurafsky D, Martin JH. Speech and Language Processing. 2021. https:\/\/web.stanford.edu\/~jurafsky\/slp3\/."},{"key":"2948_CR2","doi-asserted-by":"crossref","unstructured":"Dalianis H. Clinical text mining: Secondary use of electronic patient records. Springer Nature; 2018.","DOI":"10.1007\/978-3-319-78503-5"},{"issue":"3","key":"2948_CR3","doi-asserted-by":"publisher","first-page":"e17984","DOI":"10.2196\/17984","volume":"8","author":"I Spasic","year":"2020","unstructured":"Spasic I, Nenadic G. Clinical Text Data in Machine Learning: Systematic Review. JMIR Med Inform. 2020;8(3):e17984. https:\/\/doi.org\/10.2196\/17984.","journal-title":"JMIR Med Inform."},{"issue":"1","key":"2948_CR4","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1007\/s40171-024-00373-4","volume":"25","author":"G Kumar","year":"2024","unstructured":"Kumar G, Singh RK, Arya V, Mishra SK. Analyzing Barriers in Adoption of Artificial Intelligence for Resilient Health Care Services to Society. Glob J Flex Syst Manag. 2024;25(1):179\u201397. https:\/\/doi.org\/10.1007\/s40171-024-00373-4.","journal-title":"Glob J Flex Syst Manag."},{"key":"2948_CR5","doi-asserted-by":"crossref","unstructured":"Ranathunga S, de\u00a0Silva N. Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP World. 2022. arXiv:2210.08523.","DOI":"10.18653\/v1\/2022.aacl-main.62"},{"key":"2948_CR6","doi-asserted-by":"publisher","unstructured":"Wu H, Wang M, Wu J, Francis F, Chang YH, Shavick A, et\u00a0al. A survey on clinical natural language processing in the United Kingdom from 2007 to 2022. NPJ Digit Med. 2022;5(1). https:\/\/doi.org\/10.1038\/s41746-022-00730-6.","DOI":"10.1038\/s41746-022-00730-6"},{"key":"2948_CR7","unstructured":"Zesch T, Bewersdorff J. German Medical Natural Language Processing\u2013A Data-centric Survey. Appl Med Manuf. 2022:137."},{"key":"2948_CR8","unstructured":"Chung J, Gulcehre C, Cho K, Bengio Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. In: NIPS 2014 Workshop on Deep Learning, December 2014. 2014."},{"key":"2948_CR9","doi-asserted-by":"publisher","unstructured":"Hochreiter S, Schmidhuber J. Long Short-Term Memory. Neural Comput. 1997 11;9(8):1735\u20131780. https:\/\/doi.org\/10.1162\/neco.1997.9.8.1735.","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"2948_CR10","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et\u00a0al. Attention is All you Need. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, et\u00a0al., editors. Advances in Neural Information Processing Systems, vol.\u00a030. Curran Associates, Inc.; 2017. https:\/\/papers.nips.cc\/paper_files\/paper\/2017\/hash\/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html."},{"key":"2948_CR11","unstructured":"Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov R, Le QV. In: XLNet: Generalized Autoregressive Pretraining for Language Understanding. Red Hook: Curran Associates Inc.; 2019."},{"key":"2948_CR12","doi-asserted-by":"publisher","unstructured":"Mrini K, Dernoncourt F, Tran QH, Bui T, Chang W, Nakashole N. Rethinking Self-Attention: Towards Interpretability in Neural Parsing. In: Findings of the Association for Computational Linguistics: EMNLP 2020. Online: Association for Computational Linguistics; 2020. pp. 731\u2013742. https:\/\/doi.org\/10.18653\/v1\/2020.findings-emnlp.65.","DOI":"10.18653\/v1\/2020.findings-emnlp.65"},{"key":"2948_CR13","doi-asserted-by":"publisher","unstructured":"Edunov S, Ott M, Auli M, Grangier D. Understanding Back-Translation at Scale. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels: Association for Computational Linguistics; 2018. pp. 489\u2013500. https:\/\/doi.org\/10.18653\/v1\/D18-1045.","DOI":"10.18653\/v1\/D18-1045"},{"key":"2948_CR14","doi-asserted-by":"publisher","unstructured":"Wang X, Jiang Y, Bach N, Wang T, Huang Z, Huang F, et\u00a0al. Automated Concatenation of Embeddings for Structured Prediction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Online: Association for Computational Linguistics; 2021. pp. 2643\u20132660. https:\/\/doi.org\/10.18653\/v1\/2021.acl-long.206.","DOI":"10.18653\/v1\/2021.acl-long.206"},{"key":"2948_CR15","doi-asserted-by":"publisher","DOI":"10.1016\/j.eng.2022.04.024","author":"H Wang","year":"2022","unstructured":"Wang H, Li J, Wu H, Hovy E, Sun Y. Pre-Trained Language Models and Their Applications. Engineering. 2022. https:\/\/doi.org\/10.1016\/j.eng.2022.04.024.","journal-title":"Engineering."},{"key":"2948_CR16","unstructured":"Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed Representations of Words and Phrases and their Compositionality. In: Burges CJ, Bottou L, Welling M, Ghahramani Z, Weinberger KQ, editors. Advances in Neural Information Processing Systems, vol.\u00a026. Curran Associates, Inc.; 2013. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2013\/file\/9aa42b31882ec039965f3c4923ce901b-Paper.pdf."},{"key":"2948_CR17","doi-asserted-by":"publisher","unstructured":"Pennington J, Socher R, Manning C. GloVe: Global Vectors for Word Representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha: Association for Computational Linguistics; 2014. pp. 1532\u20131543. https:\/\/doi.org\/10.3115\/v1\/D14-1162.","DOI":"10.3115\/v1\/D14-1162"},{"key":"2948_CR18","unstructured":"Dai AM, Le QV. Semi-supervised Sequence Learning. In: Cortes C, Lawrence N, Lee D, Sugiyama M, Garnett R, editors. Advances in Neural Information Processing Systems, vol.\u00a028. Curran Associates, Inc.; 2015. https:\/\/static.googleusercontent.com\/media\/research.google.com\/es\/\/pubs\/archive\/44267.pdf."},{"key":"2948_CR19","doi-asserted-by":"publisher","unstructured":"Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, et\u00a0al. Deep Contextualized Word Representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). New Orleans: Association for Computational Linguistics; 2018. pp. 2227\u20132237. https:\/\/doi.org\/10.18653\/v1\/N18-1202.","DOI":"10.18653\/v1\/N18-1202"},{"key":"2948_CR20","doi-asserted-by":"publisher","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis: Association for Computational Linguistics; 2019. pp. 4171\u20134186. https:\/\/doi.org\/10.18653\/v1\/N19-1423.","DOI":"10.18653\/v1\/N19-1423"},{"key":"2948_CR21","unstructured":"He P, Liu X, Gao J, Chen W. Deberta: decoding-enhanced bert with disentangled attention. In: International Conference on Learning Representations. 2021. https:\/\/openreview.net\/forum?id=XPZIaotutsD."},{"key":"2948_CR22","doi-asserted-by":"publisher","unstructured":"Gu Y, Tinn R, Cheng H, Lucas M, Usuyama N, Liu X, et\u00a0al. Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing. ACM Trans Comput Healthcare. 2021;3(1). https:\/\/doi.org\/10.1145\/3458754.","DOI":"10.1145\/3458754"},{"key":"2948_CR23","unstructured":"Carrino CP, Armengol-Estap\u00e9 J, Guti\u00e9rrez-Fandi\u00f1o A, Llop-Palao J, P\u00e0mies M, Gonzalez-Agirre A, et\u00a0al. Biomedical and Clinical Language Models for Spanish: On the Benefits of Domain-Specific Pretraining in a Mid-Resource Scenario. 2021. https:\/\/arxiv.org\/abs\/2109.03570."},{"key":"2948_CR24","unstructured":"Zhao WX, Zhou K, Li J, Tang T, Wang X, Hou Y, et\u00a0al. A Survey of Large Language Models. 2023. arXiv:2303.18223."},{"key":"2948_CR25","unstructured":"Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, et\u00a0al. Language Models are Few-Shot Learners. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H, editors. Advances in Neural Information Processing Systems, vol.\u00a033. Curran Associates, Inc.; 2020. p. 1877\u2013901. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2020\/file\/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf."},{"key":"2948_CR26","unstructured":"Kaplan J, McCandlish S, Henighan T, Brown TB, Chess B, Child R, et\u00a0al. Scaling Laws for Neural Language Models. CoRR. 2020. arXiv:2001.08361."},{"key":"2948_CR27","unstructured":"Wei J, Tay Y, Bommasani R, Raffel C, Zoph B, Borgeaud S, et\u00a0al. Emergent Abilities of Large Language Models. Trans Mach Learn Res. 2022. Survey Certification. https:\/\/openreview.net\/forum?id=yzkSU5zdwD."},{"key":"2948_CR28","unstructured":"Taylor R, Kardas M, Cucurull G, Scialom T, Hartshorn A, Saravia E, et\u00a0al. Galactica: A Large Language Model for Science. 2022. https:\/\/arxiv.org\/abs\/2211.09085."},{"key":"2948_CR29","unstructured":"META. Llama 3.1. 2024. https:\/\/www.llama.com\/. Accessed 23 Sep 2024."},{"key":"2948_CR30","unstructured":"Ouyang L, Wu J, Jiang X, Almeida D, Wainwright CL, Mishkin P, et\u00a0al. Training language models to follow instructions with human feedback. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. NIPS \u201922. Red Hook: Curran Associates Inc.; 2022."},{"issue":"10","key":"2948_CR31","doi-asserted-by":"publisher","first-page":"1872","DOI":"10.1007\/s11431-020-1647-3","volume":"63","author":"X Qiu","year":"2020","unstructured":"Qiu X, Sun T, Xu Y, Shao Y, Dai N, Huang X. Pre-trained models for natural language processing: A survey. Sci China Technol Sci. 2020;63(10):1872\u201397. https:\/\/doi.org\/10.1007\/s11431-020-1647-3.","journal-title":"Sci China Technol Sci."},{"key":"2948_CR32","doi-asserted-by":"publisher","unstructured":"Howard J, Ruder S. Universal Language Model Fine-tuning for Text Classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Melbourne: Association for Computational Linguistics; 2018. pp. 328\u2013339. https:\/\/doi.org\/10.18653\/v1\/P18-1031.","DOI":"10.18653\/v1\/P18-1031"},{"issue":"3","key":"2948_CR33","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1038\/s42256-023-00626-4","volume":"5","author":"N Ding","year":"2023","unstructured":"Ding N, Qin Y, Yang G, Wei F, Yang Z, Su Y, et al. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nat Mach Intel. 2023;5(3):220\u201335. https:\/\/doi.org\/10.1038\/s42256-023-00626-4.","journal-title":"Nat Mach Intel."},{"key":"2948_CR34","unstructured":"Liu H, Tam D, Mohammed M, Mohta J, Huang T, Bansal M, et\u00a0al. Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning. In: Oh AH, Agarwal A, Belgrave D, Cho K, editors. Advances in Neural Information Processing Systems. 2022. https:\/\/openreview.net\/forum?id=rBCvMG-JsPd."},{"issue":"9","key":"2948_CR35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3560815","volume":"55","author":"P Liu","year":"2023","unstructured":"Liu P, Yuan W, Fu J, Jiang Z, Hayashi H, Neubig G. Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Comput Surv. 2023;55(9):1\u201335. https:\/\/doi.org\/10.1145\/3560815.","journal-title":"ACM Comput Surv."},{"key":"2948_CR36","doi-asserted-by":"crossref","unstructured":"Jimenez\u00a0Gutierrez B, McNeal N, Washington C, Chen Y, Li L, Sun H, et\u00a0al. Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again. In: Findings of the Association for Computational Linguistics: EMNLP 2022. Abu Dhabi: Association for Computational Linguistics; 2022. p. 4497\u2013512. https:\/\/aclanthology.org\/2022.findings-emnlp.329.","DOI":"10.18653\/v1\/2022.findings-emnlp.329"},{"key":"2948_CR37","doi-asserted-by":"publisher","unstructured":"Mosbach M, Pimentel T, Ravfogel S, Klakow D, Elazar Y. Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation. In:\u00a0Rogers A,\u00a0Boyd-Graber J,\u00a0Okazaki N, editors.\u00a0Findings of the Association for Computational Linguistics: ACL 2023.\u00a0Toronto:\u00a0Association for Computational Linguistics; 2023. p.\u00a012284\u2013314. https:\/\/doi.org\/10.18653\/v1\/2023.findings-acl.779.","DOI":"10.18653\/v1\/2023.findings-acl.779"},{"key":"2948_CR38","doi-asserted-by":"publisher","unstructured":"Wang C, Yang Y, Gao C, Peng Y, Zhang H, Lyu MR. No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence. In: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ESEC\/FSE 2022. New York: Association for Computing Machinery; 2022. pp. 382\u2013394. https:\/\/doi.org\/10.1145\/3540250.3549113.","DOI":"10.1145\/3540250.3549113"},{"key":"2948_CR39","doi-asserted-by":"publisher","first-page":"103733","DOI":"10.1016\/j.compind.2022.103733","volume":"142","author":"Z Zheng","year":"2022","unstructured":"Zheng Z, Lu XZ, Chen KY, Zhou YC, Lin JR. Pretrained domain-specific language model for natural language processing tasks in the AEC domain. Comput Ind. 2022;142:103733. https:\/\/doi.org\/10.1016\/j.compind.2022.103733.","journal-title":"Comput Ind."},{"key":"2948_CR40","doi-asserted-by":"publisher","unstructured":"B\u00e1ez P, Villena F, Rojas M, Dur\u00e1n M, Dunstan J. The Chilean Waiting List Corpus: a new resource for clinical Named Entity Recognition in Spanish. In: Proceedings of the 3rd Clinical Natural Language Processing Workshop. Online: Association for Computational Linguistics; 2020. pp. 291\u2013300. https:\/\/doi.org\/10.18653\/v1\/2020.clinicalnlp-1.32.","DOI":"10.18653\/v1\/2020.clinicalnlp-1.32"},{"key":"2948_CR41","doi-asserted-by":"publisher","unstructured":"Villena F, P\u00e9rez J, Lagos R, Dunstan J. Supporting the classification of patients in public hospitals in Chile by designing, deploying and validating a system based on natural language processing. BMC Med Inform Decis Mak. 2021;21(1). https:\/\/doi.org\/10.1186\/s12911-021-01565-z.","DOI":"10.1186\/s12911-021-01565-z"},{"issue":"3","key":"2948_CR42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3498324","volume":"3","author":"P B\u00e1ez","year":"2022","unstructured":"B\u00e1ez P, Bravo-Marquez F, Dunstan J, Rojas M, Villena F. Automatic Extraction of Nested Entities in Clinical Referrals in Spanish. ACM Trans Comput Healthc. 2022;3(3):1\u201322. https:\/\/doi.org\/10.1145\/3498324.","journal-title":"ACM Trans Comput Healthc."},{"key":"2948_CR43","doi-asserted-by":"crossref","unstructured":"Conneau A, Khandelwal K, Goyal N, Chaudhary V, Wenzek G, Guzm\u00e1n F, et\u00a0al. Unsupervised Cross-lingual Representation Learning at Scale. CoRR. 2019. arXiv:1911.02116.","DOI":"10.18653\/v1\/2020.acl-main.747"},{"key":"2948_CR44","doi-asserted-by":"publisher","unstructured":"Fandi\u00f1o AG, Estap\u00e9 JA, P\u00e0mies M, Palao JL, Ocampo JS, Carrino CP, et\u00a0al. MarIA: Spanish Language Models. Procesamiento Lenguaje Nat. 2022;68. https:\/\/doi.org\/10.26342\/2022-68-3.","DOI":"10.26342\/2022-68-3"},{"key":"2948_CR45","unstructured":"Touvron H, Martin L, Stone K, Albert P, Almahairi A, Babaei Y, et\u00a0al. Llama 2: Open Foundation and Fine-Tuned Chat Models. 2023. https:\/\/arxiv.org\/abs\/2307.09288."},{"key":"2948_CR46","unstructured":"AI@Meta. Llama 3 Model Card. 2024. https:\/\/github.com\/meta-llama\/llama3\/blob\/main\/MODEL_CARD.md."},{"key":"2948_CR47","doi-asserted-by":"publisher","unstructured":"Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, et\u00a0al. Transformers: State-of-the-art Natural Language Processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations.\u00a0Association for Computational Linguistics; 2020. p.\u00a038\u201345. https:\/\/doi.org\/10.18653\/v1\/2020.emnlp-demos.6.","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"2948_CR48","doi-asserted-by":"publisher","unstructured":"B\u00e1ez P, Bravo-Marquez F, Dunstan J, Rojas M, Villena F. Automatic Extraction of Nested Entities in Clinical Referrals in Spanish. ACM Trans Comput Healthcare. 2022;3(3). https:\/\/doi.org\/10.1145\/3498324.","DOI":"10.1145\/3498324"},{"key":"2948_CR49","doi-asserted-by":"publisher","unstructured":"Aracena C, Miranda L, Vakili T, Villena F, Quiroga T, N\u00fa\u00f1ez-Torres F, et\u00a0al. A Privacy-Preserving Corpus for Occupational Health in Spanish: Evaluation for NER and Classification Tasks. In: Naumann T, Ben\u00a0Abacha A, Bethard S, Roberts K, Bitterman D, editors. Proceedings of the 6th Clinical Natural Language Processing Workshop. Mexico City, Mexico: Association for Computational Linguistics. 2024. pp. 111\u2013121. https:\/\/doi.org\/10.18653\/v1\/2024.clinicalnlp-1.11.","DOI":"10.18653\/v1\/2024.clinicalnlp-1.11"},{"key":"2948_CR50","doi-asserted-by":"publisher","unstructured":"Richter-Pechanski P, Wiesenbach P, Schwab DM, Kiriakou C, Geis N, Dieterich C, et\u00a0al. Clinical information extraction for lower-resource languages and domains with few-shot learning using pretrained language models and prompting. Nat Lang Process. 2024:1\u201324. https:\/\/doi.org\/10.1017\/nlp.2024.52.","DOI":"10.1017\/nlp.2024.52"},{"issue":"12","key":"2948_CR51","doi-asserted-by":"publisher","first-page":"3267","DOI":"10.1093\/bioinformatics\/btac297","volume":"38","author":"L Lange","year":"2022","unstructured":"Lange L, Adel H, Str\u00f6tgen J, Klakow D. CLIN-X: pre-trained language models and a study on cross-task transfer for concept extraction in the clinical domain. Bioinformatics. 2022;38(12):3267\u201374. https:\/\/doi.org\/10.1093\/bioinformatics\/btac297.","journal-title":"Bioinformatics."},{"key":"2948_CR52","doi-asserted-by":"publisher","unstructured":"Liang S, Hartmann M, Sonntag D. Cross-domain German Medical Named Entity Recognition using a Pre-Trained Language Model and Unified Medical Semantic Types. In: Naumann T, Ben\u00a0Abacha A, Bethard S, Roberts K, Rumshisky A, editors. Proceedings of the 5th Clinical Natural Language Processing Workshop. Toronto: Association for Computational Linguistics; 2023. pp. 259\u2013271. https:\/\/doi.org\/10.18653\/v1\/2023.clinicalnlp-1.31.","DOI":"10.18653\/v1\/2023.clinicalnlp-1.31"},{"key":"2948_CR53","doi-asserted-by":"publisher","unstructured":"Schneider ETR, de\u00a0Souza JVA, Knafou J, Oliveira LESe, Copara J, Gumiel YB, et\u00a0al. BioBERTpt - A Portuguese Neural Language Model for Clinical Named Entity Recognition. In: Rumshisky A, Roberts K, Bethard S, Naumann T, editors. Proceedings of the 3rd Clinical Natural Language Processing Workshop. Online: Association for Computational Linguistics. 2020. pp. 65\u201372. https:\/\/doi.org\/10.18653\/v1\/2020.clinicalnlp-1.7.","DOI":"10.18653\/v1\/2020.clinicalnlp-1.7"},{"key":"2948_CR54","doi-asserted-by":"publisher","unstructured":"Gaschi F, Fontaine X, Rastin P, Toussaint Y. Multilingual Clinical NER: Translation or Cross-lingual Transfer? In: Naumann T, Ben\u00a0Abacha A, Bethard S, Roberts K, Rumshisky A, editors. Proceedings of the 5th Clinical Natural Language Processing Workshop. Toronto: Association for Computational Linguistics; 2023. pp. 289\u2013311. https:\/\/doi.org\/10.18653\/v1\/2023.clinicalnlp-1.34.","DOI":"10.18653\/v1\/2023.clinicalnlp-1.34"},{"key":"2948_CR55","unstructured":"Chen Z, Cano AH, Romanou A, Bonnet A, Matoba K, Salvi F, et\u00a0al. MEDITRON-70B: Scaling Medical Pretraining for Large Language Models. 2023."},{"issue":"1","key":"2948_CR56","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1093\/jamia\/ocad190","volume":"31","author":"W Zhou","year":"2023","unstructured":"Zhou W, Yetisgen M, Afshar M, Gao Y, Savova G, Miller TA. Improving model transferability for clinical note section classification models using continued pretraining. J Am Med Inform Assoc. 2023;31(1):89\u201397. https:\/\/doi.org\/10.1093\/jamia\/ocad190.","journal-title":"J Am Med Inform Assoc."},{"key":"2948_CR57","doi-asserted-by":"publisher","unstructured":"Lamproudis A, Henriksson A, Dalianis H. Developing a Clinical Language Model for Swedish: Continued Pretraining of Generic BERT with In-Domain Data. In: Proceedings of the Conference Recent Advances in Natural Language Processing - Deep Learning for Natural Language Processing Methods and Applications. RANLP 2021. INCOMA Ltd. Shoumen; 2021. pp. 790\u2013797. https:\/\/doi.org\/10.26615\/978-954-452-072-4_090.","DOI":"10.26615\/978-954-452-072-4_090"},{"key":"2948_CR58","unstructured":"Pal A, Minervini P, Motzfeldt AG, Pradipta\u00a0Gema A, Alex B. Open Medical LLM Leaderboard. Hugging Face; 2024. https:\/\/huggingface.co\/spaces\/openlifescienceai\/open_medical_llm_leaderboard."},{"key":"2948_CR59","doi-asserted-by":"publisher","first-page":"104458","DOI":"10.1016\/j.jbi.2023.104458","volume":"144","author":"Y Ge","year":"2023","unstructured":"Ge Y, Guo Y, Das S, Al-Garadi MA, Sarker A. Few-shot learning for medical text: A review of advances, trends, and opportunities. J Biomed Inform. 2023;144:104458. https:\/\/doi.org\/10.1016\/j.jbi.2023.104458.","journal-title":"J Biomed Inform."},{"key":"2948_CR60","unstructured":"Moradi M, Blagec K, Haberl F, Samwald M. GPT-3 Models are Poor Few-Shot Learners in the Biomedical Domain. 2022. arXiv:2109.02555."},{"key":"2948_CR61","doi-asserted-by":"crossref","unstructured":"Labrak Y, Bazoge A, Morin E, Gourraud PA, Rouvier M, Dufour R. BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains. 2024. arXiv:2402.10373.","DOI":"10.18653\/v1\/2024.findings-acl.348"},{"key":"2948_CR62","unstructured":"Li R, Wang X, Yu H. LlamaCare: an instruction fine-tuned large language model for clinical NLP. In: Calzolari N, Kan MY, Hoste V, Lenci A, Sakti S, Xue N, editors. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). Torino: ELRA and ICCL; 2024. p. 10632\u201341. https:\/\/aclanthology.org\/2024.lrec-main.930\/."},{"key":"2948_CR63","unstructured":"Xi Z, Chen W, Guo X, He W, Ding Y, Hong B, et\u00a0al. The Rise and Potential of Large Language Model Based Agents: A Survey. 2023. arXiv:2309.07864."},{"key":"2948_CR64","unstructured":"Li R, Wang X, Yu H. Exploring LLM Multi-Agents for ICD Coding. 2024. arXiv:2406.15363."},{"key":"2948_CR65","unstructured":"Pan American Health Organization. Maturity Assessment Levels: Information Systems for Health. Washington: PAHO; 2021. Digital Transformation Toolkit. https:\/\/iris.paho.org\/handle\/10665.2\/54962."}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-025-02948-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-025-02948-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-025-02948-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,7]],"date-time":"2025-03-07T13:21:36Z","timestamp":1741353696000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-025-02948-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,7]]},"references-count":65,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["2948"],"URL":"https:\/\/doi.org\/10.1186\/s12911-025-02948-2","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,7]]},"assertion":[{"value":"5 November 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 March 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The ethics committee at the South East Metropolitan Health Service has waived the need for approval. The Waiting List Dataset was collected through the Chilean Transparency Law ().","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"116"}}