{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T21:36:13Z","timestamp":1775338573909,"version":"3.50.1"},"reference-count":55,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T00:00:00Z","timestamp":1736899200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000049","name":"National Institute on Aging","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000049","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01AG078281"],"award-info":[{"award-number":["R01AG078281"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>This study aimed to explore the utilization of a fine-tuned language model to extract expressions related to the Age-Friendly Health Systems 4M Framework (What Matters, Medication, Mentation, and Mobility) from nursing home worker text messages, deploy automated mapping of these expressions to a taxonomy, and explore the created expressions and relationships.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>The dataset included 21\u00a0357 text messages from healthcare workers in 12 Missouri nursing homes. A sample of 860 messages was annotated by clinical experts to form a \u201cGold Standard\u201d dataset. Model performance was evaluated using classification metrics including Cohen\u2019s Kappa (\u03ba), with \u03ba \u2265 0.60 as the performance threshold. The selected model was fine-tuned. Extractions were clustered, labeled, and arranged into a structured taxonomy for exploration.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The fine-tuned model demonstrated improved extraction of 4M content (\u03ba\u2009=\u20090.73). Extractions were clustered and labeled, revealing large groups of expressions related to care preferences, medication adjustments, cognitive changes, and mobility issues.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>The preliminary development of the 4M model and 4M taxonomy enables knowledge extraction from clinical text messages and aids future development of a 4M ontology. Results compliment themes and findings in other 4M research.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>This research underscores the need for consensus building in ontology creation and the role of language models in developing ontologies, while acknowledging their limitations in logical reasoning and ontological commitments. Further development and context expansion with expert involvement of a 4M ontology are necessary.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocaf006","type":"journal-article","created":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T16:08:01Z","timestamp":1736957281000},"page":"535-544","source":"Crossref","is-referenced-by-count":3,"title":["Development and evaluation of a 4M taxonomy from nursing home staff text messages using a fine-tuned generative language model"],"prefix":"10.1093","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0989-2968","authenticated-orcid":false,"given":"Matthew Steven","family":"Farmer","sequence":"first","affiliation":[{"name":"Sinclair School of Nursing, University of Missouri , Columbia, MO 65211,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mihail","family":"Popescu","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Biostatistics, and Medical Epidemiology, School of Medicine, University of Missouri , Columbia, MO 65211,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kimberly","family":"Powell","sequence":"additional","affiliation":[{"name":"Sinclair School of Nursing, University of Missouri , Columbia, MO 65211,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2025,1,15]]},"reference":[{"key":"2025021811392712600_ocaf006-B1","doi-asserted-by":"publisher","first-page":"1060","DOI":"10.3233\/SHTI220267","volume-title":"MEDINFO 2021: One World, One Health\u2014Global Partnership for Digital Innovation","author":"Despotou","year":"2022"},{"key":"2025021811392712600_ocaf006-B2","first-page":"1441","article-title":"Normalizing clinical document titles to LOINC document ontology: an initial study","volume":"2020","author":"Zuo","year":"2020","journal-title":"AMIA Annu Symp Proc"},{"key":"2025021811392712600_ocaf006-B3","doi-asserted-by":"publisher","first-page":"e50231","DOI":"10.2196\/50231","article-title":"Examining the use of text messages among multidisciplinary care teams to reduce avoidable hospitalization of nursing home residents with dementia: protocol for a secondary analysis","volume":"12","author":"Powell","year":"2023","journal-title":"JMIR Res Protoc"},{"key":"2025021811392712600_ocaf006-B4","doi-asserted-by":"publisher","first-page":"789","DOI":"10.1111\/jgs.17571","article-title":"Clinician knowledge and behaviors related to the 4Ms framework of Age-Friendly Health Systems","volume":"70","author":"Lesser","year":"2022","journal-title":"J Am Geriatr Soc"},{"key":"2025021811392712600_ocaf006-B5","doi-asserted-by":"publisher","first-page":"469","DOI":"10.1177\/0898264321991658","article-title":"Evidence for the 4Ms: interactions and outcomes across the care continuum","volume":"33","author":"Mate","year":"2021","journal-title":"J Aging Health"},{"key":"2025021811392712600_ocaf006-B6","doi-asserted-by":"publisher","first-page":"882","DOI":"10.1111\/jgs.18722","article-title":"Developing electronic health record-based measures of the 4Ms to support implementation and evidence generation for age-friendly health systems","volume":"72","author":"Thombley","year":"2024","journal-title":"J Am Geriatr Soc"},{"key":"2025021811392712600_ocaf006-B7","first-page":"185","article-title":"The ontology-epistemology divide: a case study in medical terminology","volume":"2004","author":"Bodeneider","year":"2004","journal-title":"Form Ontol Inf Syst Proc Int Conf FOIS Conf"},{"key":"2025021811392712600_ocaf006-B8","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1002\/bul2.2017.1720430213","article-title":"Increasing patient findability of medical research: annotating clinical trials using standard vocabularies","volume":"43","author":"Panzer","year":"2017","journal-title":"Bull Assoc Inf Sci"},{"key":"2025021811392712600_ocaf006-B9","author":"Sahay","year":"2013"},{"key":"2025021811392712600_ocaf006-B10","doi-asserted-by":"publisher","first-page":"104368","DOI":"10.1016\/j.jbi.2023.104368","article-title":"Causal feature selection using a knowledge graph combining structured knowledge from the biomedical literature and ontologies: a use case studying depression as a risk factor for Alzheimer\u2019s disease","volume":"142","author":"Malec","year":"2023","journal-title":"J Biomed Inform"},{"key":"2025021811392712600_ocaf006-B11","doi-asserted-by":"publisher","first-page":"272","DOI":"10.30534\/ijatcse\/2019\/4181.62019","article-title":"Integrated ontology development for clinical decision support system in the case study of methadone maintenance therapy","volume":"8","author":"Rahim","year":"2019","journal-title":"IJATCSE"},{"key":"2025021811392712600_ocaf006-B12","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1145\/2944165.2944172","author":"El-Atawy","year":"2016"},{"key":"2025021811392712600_ocaf006-B13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/ICSC.2014.67","author":"Hsu","year":"2014"},{"key":"2025021811392712600_ocaf006-B14","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.16648","article-title":"LLMs4OL: large language models for ontology learning","volume-title":"arXiv","author":"Giglou","year":"2023"},{"key":"2025021811392712600_ocaf006-B15","doi-asserted-by":"publisher","first-page":"btae163","DOI":"10.1093\/bioinformatics\/btae163","article-title":"Advancing entity recognition in biomedicine via instruction tuning of large language models","volume":"40","author":"Keloth","year":"2024","journal-title":"Bioinformatics"},{"key":"2025021811392712600_ocaf006-B16","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2306.10062","article-title":"Revealing the structure of language model capabilities","volume-title":"arXiv","author":"Burnell","year":"2023"},{"key":"2025021811392712600_ocaf006-B17","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2308.01727","article-title":"Local large language models for complex structured medical tasks","volume-title":"arXiv","author":"Bumgardner","year":"2023"},{"key":"2025021811392712600_ocaf006-B18","doi-asserted-by":"publisher","author":"Rula","year":"2023","DOI":"10.48550\/arXiv.2310.03376"},{"key":"2025021811392712600_ocaf006-B19","doi-asserted-by":"publisher","first-page":"04031","DOI":"10.1051\/e3sconf\/202339904031","volume-title":"E3S Web Conf","author":"John Basha","year":"2023"},{"key":"2025021811392712600_ocaf006-B20","doi-asserted-by":"publisher","first-page":"15","DOI":"10.5121\/cseij.2024.14302","article-title":"Revolutionising translation technology: a comparative study of variant transformer models\u2014BERT, GPT, and T5","volume":"14","author":"Zayyanu","year":"2024","journal-title":"CSEIJ"},{"key":"2025021811392712600_ocaf006-B21","doi-asserted-by":"publisher","first-page":"18","DOI":"10.18231\/j.jmra.2023.004","article-title":"Exploring the capabilities and limitations of GPT and Chat GPT in natural language processing","volume":"10","author":"Thakkar","year":"2023","journal-title":"JMRA"},{"key":"2025021811392712600_ocaf006-B22","doi-asserted-by":"publisher","first-page":"101","DOI":"10.3390\/bdcc7020101","article-title":"An ontology development methodology based on ontology-driven conceptual modeling and natural language processing: tourism case study","volume":"7","author":"Haridy","year":"2023","journal-title":"BDCC"},{"key":"2025021811392712600_ocaf006-B23","author":"Smith","year":"2004"},{"key":"2025021811392712600_ocaf006-B24","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1007\/s40266-023-01016-6","article-title":"Optimizing medications with the geriatrics 5Ms: an age-friendly approach","volume":"40","author":"Monette","year":"2023","journal-title":"Drugs Aging"},{"key":"2025021811392712600_ocaf006-B25","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1007\/s10329-021-00891-0","article-title":"Human language evolution: a view from theoretical linguistics on how syntax and the lexicon first came into being","volume":"63","author":"Fujita","year":"2022","journal-title":"Primates"},{"key":"2025021811392712600_ocaf006-B26","doi-asserted-by":"publisher","first-page":"846","DOI":"10.1097\/ACM.0000000000003109","article-title":"Revisioning academic medicine through a constructionist lens","volume":"95","author":"Rees","year":"2020","journal-title":"Acad Med"},{"key":"2025021811392712600_ocaf006-B27","doi-asserted-by":"publisher","first-page":"399","DOI":"10.3233\/AO-230072","article-title":"Ontologies in the era of large language models\u2014a perspective","volume":"18","author":"Neuhaus","year":"2023","journal-title":"AO"},{"key":"2025021811392712600_ocaf006-B28","doi-asserted-by":"publisher","first-page":"495","DOI":"10.3233\/AO-220273","article-title":"Ontology development is consensus creation, not (merely) representation","volume":"17","author":"Neuhaus","year":"2022","journal-title":"AO"},{"key":"2025021811392712600_ocaf006-B29","doi-asserted-by":"publisher","author":"Mai","year":"2024","DOI":"10.48550\/ARXIV.2407.19998"},{"key":"2025021811392712600_ocaf006-B30","author":"IHI","year":"2024"},{"key":"2025021811392712600_ocaf006-B31","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: the kappa statistic","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochem Med"},{"key":"2025021811392712600_ocaf006-B32","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2312.12148","article-title":"Parameter-efficient fine-tuning methods for pretrained language models: a critical review and assessment","volume-title":"arXiv","author":"Xu","year":"2023"},{"key":"2025021811392712600_ocaf006-B33","doi-asserted-by":"publisher","author":"Hong","year":"2024","DOI":"10.48550\/arXiv.2403.07691"},{"key":"2025021811392712600_ocaf006-B34","doi-asserted-by":"publisher","author":"Reimers","year":"2019","DOI":"10.48550\/arXiv.1908.10084"},{"key":"2025021811392712600_ocaf006-B35","author":"AgglomerativeClustering","year":"2024"},{"key":"2025021811392712600_ocaf006-B36","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1017\/S0269888900007797","article-title":"Ontologies: principles, methods and applications","volume":"11","author":"Uschold","year":"1996","journal-title":"Knowl Eng Rev"},{"key":"2025021811392712600_ocaf006-B37","doi-asserted-by":"publisher","author":"Gr\u00fcninger","year":"1995","DOI":"10.1007\/978-0-387-34847-6_3"},{"key":"2025021811392712600_ocaf006-B38","author":"Google Deepmind","year":"2024"},{"key":"2025021811392712600_ocaf006-B39","author":"Dubey"},{"key":"2025021811392712600_ocaf006-B40","author":"Mistral AI Team","year":"2024"},{"key":"2025021811392712600_ocaf006-B41","doi-asserted-by":"publisher","first-page":"8038","DOI":"10.1002\/alz.14292","article-title":"Untangling the complex web of avoidable nursing home-to-hospital transfers of residents with dementia","volume":"20","author":"Powell","year":"2024","journal-title":"Alzheimers Dement"},{"key":"2025021811392712600_ocaf006-B42","doi-asserted-by":"publisher","first-page":"1321","DOI":"10.2147\/CIA.S409348","article-title":"Unmet needs and barriers in providing hospital care for older adults: a qualitative study using the age-friendly health system framework","volume":"18","author":"Yi","year":"2023","journal-title":"Clin Interv Aging"},{"key":"2025021811392712600_ocaf006-B43","doi-asserted-by":"publisher","first-page":"1500","DOI":"10.1016\/j.cjca.2024.04.024","article-title":"Enhancing the 4M framework: the critical role of malnutrition in immobility among hospitalized patients","volume":"40","author":"Chao","year":"2024","journal-title":"Can J Cardiol"},{"key":"2025021811392712600_ocaf006-B44","doi-asserted-by":"publisher","first-page":"e224","DOI":"10.1093\/geront\/gnab071","article-title":"Assessing what matters most in older adults with multicomplexity","volume":"62","author":"Moye","year":"2022","journal-title":"Gerontologist"},{"key":"2025021811392712600_ocaf006-B45","doi-asserted-by":"publisher","first-page":"2115","DOI":"10.1111\/jgs.14979","article-title":"The geriatrics 5M\u2019s: a new way of communicating what we do","volume":"65","author":"Tinetti","year":"2017","journal-title":"J Am Geriatr Soc"},{"key":"2025021811392712600_ocaf006-B46","doi-asserted-by":"publisher","first-page":"110385","DOI":"10.1016\/j.knosys.2023.110385","article-title":"Using terms and informal definitions to classify domain entities into top-level ontology concepts: an approach based on language models","volume":"265","author":"Lopes","year":"2023","journal-title":"Knowl-Based Syst"},{"key":"2025021811392712600_ocaf006-B47","doi-asserted-by":"publisher","author":"Mateiu","year":"2023","DOI":"10.48550\/arXiv.2307.16699"},{"key":"2025021811392712600_ocaf006-B48","doi-asserted-by":"publisher","first-page":"3580","DOI":"10.1109\/TKDE.2024.3352100","article-title":"Unifying large language models and knowledge graphs: a roadmap","volume":"36","author":"Pan","year":"2024","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"2025021811392712600_ocaf006-B49","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2405.01581","article-title":"The mercurial top-level ontology of large language models","volume-title":"arXiv","author":"K\u00f6hler","year":"2024"},{"key":"2025021811392712600_ocaf006-B50","doi-asserted-by":"publisher","first-page":"1452","DOI":"10.1056\/NEJMra1615014","article-title":"Classification, ontology, and precision medicine","volume":"379","author":"Haendel","year":"2018","journal-title":"N Engl J Med"},{"key":"2025021811392712600_ocaf006-B51","doi-asserted-by":"publisher","author":"Hart","year":"2024","DOI":"10.1101\/2024.06.26.24308798"},{"key":"2025021811392712600_ocaf006-B52","doi-asserted-by":"publisher","first-page":"177","DOI":"10.3233\/SHTI200718","article-title":"Creating synthetic patients to address interoperability issues: a case study with the management of breast cancer patients","volume":"275","author":"Redjdal","year":"2020","journal-title":"Stud Health Technol Inform"},{"key":"2025021811392712600_ocaf006-B53","doi-asserted-by":"publisher","first-page":"101","DOI":"10.30699\/fhi.v11i1.339","article-title":"eCOVID19\u2014development of ontology-based clinical decision support system for COVID-19","volume":"11","author":"Sherimon","year":"2022","journal-title":"Front Health Inform"},{"key":"2025021811392712600_ocaf006-B54","doi-asserted-by":"publisher","author":"Gomez-Perez","year":"2020","DOI":"10.1007\/978-3-030-44830-1"},{"key":"2025021811392712600_ocaf006-B55","doi-asserted-by":"publisher","DOI":"10.21203\/rs.3.rs-3483777\/v1","article-title":"Clinical text summarization: adapting large language models can outperform human experts","author":"Van Veen","year":"2023","journal-title":"Res Sq [Preprint]"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/3\/535\/61442720\/ocaf006.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/3\/535\/61442720\/ocaf006.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T11:39:42Z","timestamp":1739878782000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/32\/3\/535\/7954486"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,15]]},"references-count":55,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,1,15]]},"published-print":{"date-parts":[[2025,3,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaf006","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,3]]},"published":{"date-parts":[[2025,1,15]]}}}