{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T09:03:38Z","timestamp":1791882218541,"version":"4.3.4"},"reference-count":24,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2025,1,9]],"date-time":"2025-01-09T00:00:00Z","timestamp":1736380800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>This study evaluates the utility of word embeddings, generated by large language models (LLMs), for medical diagnosis by comparing the semantic proximity of symptoms to their eponymic disease embedding (\u201ceponymic condition\u201d) and the mean of all symptom embeddings associated with a disease (\u201censemble mean\u201d).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>Symptom data for 5 diagnostically challenging pediatric diseases\u2014CHARGE syndrome, Cowden disease, POEMS syndrome, Rheumatic fever, and Tuberous sclerosis\u2014were collected from PubMed. Using the Ada-002 embedding model, disease names and symptoms were translated into vector representations in a high-dimensional space. Euclidean and Chebyshev distance metrics were used to classify symptoms based on their proximity to both the eponymic condition and the ensemble mean of the condition's symptoms.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The ensemble mean approach showed significantly higher classification accuracy, correctly classifying between 80% (Cowden disease) to 100% (Tuberous sclerosis) of the sample disease symptoms using the Euclidean distance metric. In contrast, the eponymic condition approach using Euclidian distance metric and Chebyshev distances, in general, showed poor symptom classification performance, with erratic results (0%-100% accuracy), largely ranging between 0% and 3% accuracy.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>The ensemble mean captures a disease's collective symptom profile, providing a more nuanced representation than the disease name alone. However, some misclassifications were due to superficial semantic similarities, highlighting the need for LLM models trained on medical corpora.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>The ensemble mean of symptom embeddings improves classification accuracy over the eponymic condition approach. Future efforts should focus on medical-specific training of LLMs to enhance their diagnostic accuracy and clinical utility.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocae314","type":"journal-article","created":{"date-parts":[[2025,1,9]],"date-time":"2025-01-09T16:48:24Z","timestamp":1736441304000},"page":"526-534","source":"Crossref","is-referenced-by-count":6,"title":["Utility of word embeddings from large language models in medical diagnosis"],"prefix":"10.1093","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1138-1756","authenticated-orcid":false,"given":"Shahram","family":"Yazdani","sequence":"first","affiliation":[{"name":"Department of Pediatrics, David Geffen School of Medicine, University of California Los Angeles , Los Angeles, CA 90095,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronald Claude","family":"Henry","sequence":"additional","affiliation":[{"name":"Department of Civil Engineering, University of Southern California , Los Angeles, CA 90089,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Avery","family":"Byrne","sequence":"additional","affiliation":[{"name":"San Francisco , CA 94129,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Isaac Claude","family":"Henry","sequence":"additional","affiliation":[{"name":"Kennewick , WA 99338,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,1,9]]},"reference":[{"key":"2025021811401035000_ocae314-B1","doi-asserted-by":"crossref","first-page":"1589","DOI":"10.1109\/JBHI.2017.2767063","article-title":"Deep EHR: a survey of recent advances on deep learning techniques for electronic health record (EHR) analysis","volume":"22","author":"Shickel","year":"2018","journal-title":"IEEE J Biomed Health Inform"},{"key":"2025021811401035000_ocae314-B2","doi-asserted-by":"crossref","first-page":"1317","DOI":"10.1001\/jama.2017.18391","article-title":"Big data and machine learning in health care","volume":"319","author":"Beam","year":"2018","journal-title":"JAMA"},{"key":"2025021811401035000_ocae314-B3","author":"Zhang"},{"key":"2025021811401035000_ocae314-B4","doi-asserted-by":"crossref","first-page":"1297","DOI":"10.1093\/jamia\/ocz096","article-title":"Enhancing clinical concept extraction with contextual embeddings","volume":"26","author":"Si","year":"2019","journal-title":"J Am Med Inform Assoc"},{"key":"2025021811401035000_ocae314-B5","author":"Mikolov"},{"key":"2025021811401035000_ocae314-B6","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3115\/v1\/D14-1162","article-title":"GloVe: global vectors for word representation","author":"Pennington","year":"2014","journal-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)"},{"key":"2025021811401035000_ocae314-B7","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1162\/tacl_a_00051","article-title":"Enriching word vectors with subword information","volume":"5","author":"Bojanowski","year":"2017","journal-title":"TACL"},{"key":"2025021811401035000_ocae314-B8","author":"Tekir"},{"key":"2025021811401035000_ocae314-B9","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics."},{"key":"2025021811401035000_ocae314-B10","author":"Alsentzer"},{"key":"2025021811401035000_ocae314-B11","author":"Beltagy"},{"key":"2025021811401035000_ocae314-B12","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: the convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat Med."},{"key":"2025021811401035000_ocae314-B13","doi-asserted-by":"crossref","first-page":"59","DOI":"10.12793\/tcp.2019.27.2.59","article-title":"A review of computational drug repurposing","volume":"27","author":"Park","year":"2019","journal-title":"Transl Clin Pharmacol."},{"key":"2025021811401035000_ocae314-B14","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.jbi.2018.09.008","article-title":"A comparison of word embeddings for the biomedical natural language processing","volume":"87","author":"Wang","year":"2018","journal-title":"J Biomed Inform."},{"key":"2025021811401035000_ocae314-B15","doi-asserted-by":"crossref","first-page":"e0192360","DOI":"10.1371\/journal.pone.0192360","article-title":"Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives","volume":"13","author":"Gehrmann","year":"2018","journal-title":"PLoS One."},{"key":"2025021811401035000_ocae314-B16","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1177\/000992289803700302","article-title":"CHARGE association: an update and review for the primary pediatrician","volume":"37","author":"Blake","year":"1998","journal-title":"Clin Pediatr (Phila)"},{"key":"2025021811401035000_ocae314-B17","doi-asserted-by":"crossref","first-page":"1607","DOI":"10.1093\/jnci\/djt277","article-title":"Cowden syndrome and the PTEN hamartoma tumor syndrome: systematic review and revised diagnostic criteria","volume":"105","author":"Pilarski","year":"2013","journal-title":"J Natl Cancer Inst."},{"key":"2025021811401035000_ocae314-B18","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1016\/j.blre.2007.07.004","article-title":"POEMS syndrome","volume":"21","author":"Dispenzieri","year":"2007","journal-title":"Blood Rev."},{"key":"2025021811401035000_ocae314-B19","doi-asserted-by":"crossref","first-page":"1806","DOI":"10.1161\/CIR.0000000000000205","article-title":"Revision of the Jones Criteria for the diagnosis of acute rheumatic fever in the era of Doppler echocardiography: a scientific statement from the American Heart Association","volume":"131","author":"Gewitz","year":"2015","journal-title":"Circulation"},{"key":"2025021811401035000_ocae314-B20","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.pediatrneurol.2021.07.011","article-title":"Updated international tuberous sclerosis complex diagnostic criteria and surveillance and management recommendations","volume":"123","author":"Northrup","year":"2021","journal-title":"Pediatr Neurol"},{"key":"2025021811401035000_ocae314-B21","author":"OpenAI"},{"key":"2025021811401035000_ocae314-B22","volume-title":"Directional Statistics","author":"Mardia","year":"2009","edition":"2nd ed."},{"key":"2025021811401035000_ocae314-B23","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-17146-8","volume-title":"Nonparametric and Semiparametric Methods","author":"H\u00e4rdle","year":"2004"},{"key":"2025021811401035000_ocae314-B24","doi-asserted-by":"crossref","first-page":"e36678","DOI":"10.1097\/MD.0000000000036678","article-title":"POEMS syndrome characterized by bone lesions: a case report","volume":"102","author":"Wang","year":"2023","journal-title":"Medicine (Baltimore)"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/3\/526\/61396806\/ocae314.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/3\/526\/61396806\/ocae314.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T11:40:30Z","timestamp":1739878830000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/32\/3\/526\/7950666"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,9]]},"references-count":24,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,1,9]]},"published-print":{"date-parts":[[2025,3,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae314","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,3]]},"published":{"date-parts":[[2025,1,9]]}}}