{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T17:34:33Z","timestamp":1776447273284,"version":"3.51.2"},"reference-count":48,"publisher":"Oxford University Press (OUP)","issue":"10","license":[{"start":{"date-parts":[[2024,7,13]],"date-time":"2024-07-13T00:00:00Z","timestamp":1720828800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"name":"University of Pittsburgh Momentum Funds"},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["UL1TR001857"],"award-info":[{"award-number":["UL1TR001857"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["U24TR004111"],"award-info":[{"award-number":["U24TR004111"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01LM014306"],"award-info":[{"award-number":["R01LM014306"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>Alzheimer\u2019s disease (AD) is the most common form of dementia in the United States. Sleep is one of the lifestyle-related factors that has been shown critical for optimal cognitive function in old age. However, there is a lack of research studying the association between sleep and AD incidence. A major bottleneck for conducting such research is that the traditional way to acquire sleep information is time-consuming, inefficient, non-scalable, and limited to patients\u2019 subjective experience. We aim to automate the extraction of specific sleep-related patterns, such as snoring, napping, poor sleep quality, daytime sleepiness, night wakings, other sleep problems, and sleep duration, from clinical notes of AD patients. These sleep patterns are hypothesized to play a role in the incidence of AD, providing insight into the relationship between sleep and AD onset and progression.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>A gold standard dataset is created from manual annotation of 570 randomly sampled clinical note documents from the adSLEEP, a corpus of 192\u00a0000 de-identified clinical notes of 7266 AD patients retrieved from the University of Pittsburgh Medical Center (UPMC). We developed a rule-based natural language processing (NLP) algorithm, machine learning models, and large language model (LLM)-based NLP algorithms to automate the extraction of sleep-related concepts, including snoring, napping, sleep problem, bad sleep quality, daytime sleepiness, night wakings, and sleep duration, from the gold standard dataset.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The annotated dataset of 482 patients comprised a predominantly White (89.2%), older adult population with an average age of 84.7 years, where females represented 64.1%, and a vast majority were non-Hispanic or Latino (94.6%). Rule-based NLP algorithm achieved the best performance of F1 across all sleep-related concepts. In terms of positive predictive value (PPV), the rule-based NLP algorithm achieved the highest PPV scores for daytime sleepiness (1.00) and sleep duration (1.00), while the machine learning models had the highest PPV for napping (0.95) and bad sleep quality (0.86), and LLAMA2 with finetuning had the highest PPV for night wakings (0.93) and sleep problem (0.89).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>Although sleep information is infrequently documented in the clinical notes, the proposed rule-based NLP algorithm and LLM-based NLP algorithms still achieved promising results. In comparison, the machine learning-based approaches did not achieve good results, which is due to the small size of sleep information in the training data.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>The results show that the rule-based NLP algorithm consistently achieved the best performance for all sleep concepts. This study focused on the clinical notes of patients with AD but could be extended to general sleep information extraction for other diseases.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocae177","type":"journal-article","created":{"date-parts":[[2024,7,1]],"date-time":"2024-07-01T16:35:09Z","timestamp":1719851709000},"page":"2217-2227","source":"Crossref","is-referenced-by-count":19,"title":["Extraction of sleep information from clinical notes of Alzheimer\u2019s disease patients using natural language processing"],"prefix":"10.1093","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4173-1988","authenticated-orcid":false,"given":"Sonish","family":"Sivarajkumar","sequence":"first","affiliation":[{"name":"Intelligent Systems Program, University of Pittsburgh , Pittsburgh, PA 15260, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thomas Yu Chow","family":"Tam","sequence":"additional","affiliation":[{"name":"Department of Health Information Management, University of Pittsburgh , Pittsburgh, PA 15260, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haneef Ahamed","family":"Mohammad","sequence":"additional","affiliation":[{"name":"Department of Health Information Management, University of Pittsburgh , Pittsburgh, PA 15260, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Samuel","family":"Viggiano","sequence":"additional","affiliation":[{"name":"Department of Health Information Management, University of Pittsburgh , Pittsburgh, PA 15260, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Oniani","sequence":"additional","affiliation":[{"name":"Department of Health Information Management, University of Pittsburgh , Pittsburgh, PA 15260, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2079-8684","authenticated-orcid":false,"given":"Shyam","family":"Visweswaran","sequence":"additional","affiliation":[{"name":"Intelligent Systems Program, University of Pittsburgh , Pittsburgh, PA 15260, United States"},{"name":"Department of Biomedical Informatics, University of Pittsburgh , Pittsburgh, PA 15260, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanshan","family":"Wang","sequence":"additional","affiliation":[{"name":"Intelligent Systems Program, University of Pittsburgh , Pittsburgh, PA 15260, United States"},{"name":"Department of Health Information Management, University of Pittsburgh , Pittsburgh, PA 15260, United States"},{"name":"Department of Biomedical Informatics, University of Pittsburgh , Pittsburgh, PA 15260, United States"},{"name":"Clinical and Translational Science Institute, University of Pittsburgh , Pittsburgh, PA 15260, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2024,7,13]]},"reference":[{"issue":"3","key":"2024092007524254600_ocae177-B1","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/j.jalz.2018.02.001","article-title":"2018 Alzheimer\u2019s disease facts and figures","volume":"14","author":"Alzheimer\u2019s Association","year":"2018","journal-title":"Alzheimers Dementia"},{"issue":"3","key":"2024092007524254600_ocae177-B2","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/j.jalz.2019.01.010","article-title":"2019 Alzheimer\u2019s disease facts and figures","volume":"15","author":"Alzheimer\u2019s Association","year":"2019","journal-title":"Alzheimers Dementia"},{"issue":"4","key":"2024092007524254600_ocae177-B3","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1016\/j.jalz.2017.12.006","article-title":"The cost of Alzheimer\u2019s disease in China and re-estimation of costs worldwide","volume":"14","author":"Jia","year":"2018","journal-title":"Alzheimers Dement"},{"issue":"1","key":"2024092007524254600_ocae177-B4","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1007\/s00405-023-08282-5","article-title":"ChatGPT performance in laryngology and head and neck surgery: a clinical case-series","volume":"281","author":"Lechien","year":"2024","journal-title":"Eur Arch Otorhinolaryngol"},{"issue":"11","key":"2024092007524254600_ocae177-B5","doi-asserted-by":"crossref","first-page":"877","DOI":"10.1016\/S1474-4422(17)30299-5","article-title":"Global, regional, and national burden of neurological disorders during 1990\u20132015: a systematic analysis for the Global Burden of Disease Study 2015","volume":"16","author":"Feigin","year":"2017","journal-title":"Lancet Neurol"},{"issue":"3","key":"2024092007524254600_ocae177-B6","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1016\/j.jalz.2017.09.006","article-title":"Multidomain lifestyle intervention benefits a large elderly population at risk for cognitive decline and dementia regardless of baseline characteristics: the FINGER trial","volume":"14","author":"Rosenberg","year":"2018","journal-title":"Alzheimers Dement"},{"issue":"7","key":"2024092007524254600_ocae177-B7","doi-asserted-by":"crossref","first-page":"886","DOI":"10.1016\/j.sleep.2012.02.003","article-title":"What sleep characteristics predict cognitive decline in the elderly?","volume":"13","author":"Keage","year":"2012","journal-title":"Sleep Med"},{"issue":"9","key":"2024092007524254600_ocae177-B8","doi-asserted-by":"crossref","first-page":"1185","DOI":"10.1046\/j.1532-5415.2001.49235.x","article-title":"The impact of insomnia on cognitive functioning in older adults","volume":"49","author":"Cricco","year":"2001","journal-title":"J Am Geriatr Soc"},{"issue":"12","key":"2024092007524254600_ocae177-B9","first-page":"1628","article-title":"Daytime sleepiness is associated with 3-year incident dementia and cognitive decline in older Japanese-American men","volume":"49","author":"Foley","year":"2001","journal-title":"J Am Geriatr Soc"},{"issue":"9","key":"2024092007524254600_ocae177-B10","doi-asserted-by":"crossref","first-page":"820","DOI":"10.1136\/jech.2009.100503","article-title":"Sleep disturbance and daytime sleepiness predict vascular dementia","volume":"65","author":"Elwood","year":"2011","journal-title":"J Epidemiol Community Health"},{"issue":"9","key":"2024092007524254600_ocae177-B11","doi-asserted-by":"crossref","first-page":"1159","DOI":"10.1111\/j.1532-5415.1999.tb05252.x","article-title":"Snoring and risk of cognitive decline: a 4-year follow-up study in 1389 older individuals","volume":"47","author":"Quesnot","year":"1999","journal-title":"J Am Geriatr Soc"},{"issue":"1","key":"2024092007524254600_ocae177-B12","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1097\/01.wad.0000201850.52707.80","article-title":"The association of self-reported sleep duration, difficulty sleeping, and snoring with cognitive function in older women","volume":"20","author":"Tworoger","year":"2006","journal-title":"Alzheimer Dis Assoc Disord"},{"issue":"4","key":"2024092007524254600_ocae177-B13","doi-asserted-by":"crossref","first-page":"491","DOI":"10.5665\/sleep.1732","article-title":"Sleep quality and 1-year incident cognitive impairment in community-dwelling older adults","volume":"35","author":"Potvin","year":"2012","journal-title":"Sleep"},{"issue":"12","key":"2024092007524254600_ocae177-B14","doi-asserted-by":"crossref","first-page":"1577","DOI":"10.1080\/13607863.2017.1387760","article-title":"Psychosocial risk factors and Alzheimer\u2019s disease: the associative effect of depression, sleep disturbance, and anxiety","volume":"22","author":"Burke","year":"2018","journal-title":"Aging Ment Health"},{"issue":"1","key":"2024092007524254600_ocae177-B15","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1002\/gps.529","article-title":"Subjective sleep problems in later life as predictors of cognitive decline. Report from the Maastricht Ageing Study (MAAS)","volume":"17","author":"Jelicic","year":"2002","journal-title":"Int J Geriatr Psychiatry"},{"issue":"5","key":"2024092007524254600_ocae177-B16","doi-asserted-by":"crossref","first-page":"zsz040","DOI":"10.1093\/sleep\/zsz040","article-title":"Sleep and cognitive function in chronic stroke: a comparative cross-sectional study","volume":"42","author":"Falck","year":"2019","journal-title":"Sleep"},{"key":"2024092007524254600_ocae177-B17","doi-asserted-by":"crossref","first-page":"1037650","DOI":"10.3389\/fnagi.2022.1037650","article-title":"Longitudinal associations between sleep duration and cognitive impairment in Chinese elderly","volume":"14","author":"Chen","year":"2022","journal-title":"Front Aging Neurosci"},{"issue":"5","key":"2024092007524254600_ocae177-B18","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1056\/NEJMp0912825","article-title":"Launching hitech","volume":"362","author":"Blumenthal","year":"2010","journal-title":"N Engl J Med"},{"issue":"11","key":"2024092007524254600_ocae177-B19","doi-asserted-by":"crossref","first-page":"e073734","DOI":"10.1136\/bmjopen-2023-073734","article-title":"Study protocol for a longitudinal observational study of disparities in sleep and cognition in older adults: the DISCO study","volume":"13","author":"Knutson","year":"2023","journal-title":"BMJ Open"},{"issue":"6","key":"2024092007524254600_ocae177-B20","doi-asserted-by":"crossref","first-page":"fcac257","DOI":"10.1093\/braincomms\/fcac257","article-title":"Cross-sectional and longitudinal association of sleep and Alzheimer biomarkers in cognitively unimpaired adults","volume":"4","author":"Blackman","year":"2022","journal-title":"Brain Commun"},{"issue":"2","key":"2024092007524254600_ocae177-B21","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1016\/j.jalz.2017.06.2270","article-title":"Dementia prevalence and incidence in a federation of European Electronic Health Record databases: the European Medical Informatics Framework resource","volume":"14","author":"Perera","year":"2018","journal-title":"Alzheimers Dement"},{"issue":"1","key":"2024092007524254600_ocae177-B22","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1186\/1471-2318-14-76","article-title":"Health care resource utilisation in primary care prior to and after a diagnosis of Alzheimer\u2019s disease: a retrospective, matched case\u2013control study in the United Kingdom","volume":"14","author":"Chen","year":"2014","journal-title":"BMC Geriatr"},{"issue":"1","key":"2024092007524254600_ocae177-B23","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1186\/1471-244X-14-84","article-title":"Comorbidity of dementia: a cross-sectional study of primary care older patients","volume":"14","author":"Poblador-Plou","year":"2014","journal-title":"BMC Psychiatry"},{"issue":"3","key":"2024092007524254600_ocae177-B24","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/j.jalz.2015.12.007","article-title":"Inequalities in dementia incidence between six racial and ethnic groups over 14 years","volume":"12","author":"Mayeda","year":"2016","journal-title":"Alzheimers Dement"},{"key":"2024092007524254600_ocae177-B25","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.jbi.2017.11.011","article-title":"Clinical information extraction applications: a literature review","volume":"77","author":"Wang","year":"2018","journal-title":"J Biomed Inform"},{"issue":"3","key":"2024092007524254600_ocae177-B26","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1097\/AOG.0000000000002132","article-title":"Sleep disorder diagnosis during pregnancy and risk of preterm birth","volume":"130","author":"Felder","year":"2017","journal-title":"Obstet Gynecol"},{"issue":"4","key":"2024092007524254600_ocae177-B27","doi-asserted-by":"crossref","first-page":"581","DOI":"10.5665\/sleep.4574","article-title":"Sleep disorders and increased risk of autoimmune diseases in individuals without sleep apnea","volume":"38","author":"Hsiao","year":"2015","journal-title":"Sleep"},{"key":"2024092007524254600_ocae177-B28","volume-title":"Towards Validating the Effectiveness of Obstructive Sleep Apnea Classification from Electronic Health Records Using Machine Learning Healthcare","author":"Ramesh","year":"2021"},{"issue":"12","key":"2024092007524254600_ocae177-B29","doi-asserted-by":"crossref","first-page":"1443","DOI":"10.5664\/jcsm.5284","article-title":"Evidence supports no relationship between obstructive sleep apnea and premolar extraction: an electronic health records review","volume":"11","author":"Larsen","year":"2015","journal-title":"J Clin Sleep Med"},{"issue":"1","key":"2024092007524254600_ocae177-B30","first-page":"448","article-title":"Identifying cases of sleep disorders through international classification of diseases (ICD) codes in administrative data","volume":"3","author":"Jolley","year":"2018","journal-title":"Int J Popul Data Sci"},{"key":"2024092007524254600_ocae177-B31","first-page":"88","author":"Singer","year":"2021"},{"key":"2024092007524254600_ocae177-B32","first-page":"356","volume-title":"MEDINFO 2017: Precision Healthcare through Informatics","author":"Divita","year":"2017"},{"issue":"1","key":"2024092007524254600_ocae177-B33","doi-asserted-by":"crossref","first-page":"e012012","DOI":"10.1136\/bmjopen-2016-012012","article-title":"Natural language processing to extract symptoms of severe mental illness from clinical text: the Clinical Record Interactive Search Comprehensive Data Extraction (CRIS-CODE) project","volume":"7","author":"Jackson","year":"2017","journal-title":"BMJ Open"},{"key":"2024092007524254600_ocae177-B34","first-page":"629","volume-title":"MEDINFO 2015: eHealth-Enabled Health","author":"Zhou","year":"2015"},{"issue":"2","key":"2024092007524254600_ocae177-B35","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1093\/schbul\/sbaa126","article-title":"Using natural language processing on electronic health records to enhance detection and prediction of psychosis risk","volume":"47","author":"Irving","year":"2021","journal-title":"Schizophr Bull"},{"issue":"1","key":"2024092007524254600_ocae177-B36","doi-asserted-by":"crossref","first-page":"7862","DOI":"10.1038\/s41598-018-25312-z","article-title":"Development of an algorithm to identify patients with physician-documented insomnia","volume":"8","author":"Kartoun","year":"2018","journal-title":"Sci Rep"},{"key":"2024092007524254600_ocae177-B37","doi-asserted-by":"crossref","first-page":"1178222617713018","DOI":"10.1177\/1178222617713018","article-title":"Leveraging food and drug administration adverse event reports for the automated monitoring of electronic health records in a pediatric hospital","volume":"9","author":"Tang","year":"2017","journal-title":"Biomed Inform Insights"},{"issue":"3","key":"2024092007524254600_ocae177-B38","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1197\/jamia.M3028","article-title":"Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study","volume":"16","author":"Wang","year":"2009","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2024092007524254600_ocae177-B39","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1159\/000442418","article-title":"Sleep duration in relation to cognitive function among older adults: a systematic review of observational studies","volume":"46","author":"Devore","year":"2016","journal-title":"Neuroepidemiology"},{"key":"2024092007524254600_ocae177-B40","first-page":"149","article-title":"An information extraction framework for cohort identification using electronic health records","volume":"2013","author":"Liu","year":"2013","journal-title":"AMIA Jt Summits on Transl Sci Proc"},{"issue":"3-4","key":"2024092007524254600_ocae177-B41","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1017\/S1351324904003523","article-title":"UIMA: an architectural approach to unstructured information processing in the corporate research environment","volume":"10","author":"Ferrucci","year":"2004","journal-title":"Nat Lang Eng"},{"key":"2024092007524254600_ocae177-B42","author":"Mikolov","year":"2013"},{"key":"2024092007524254600_ocae177-B43","author":"Touvron","year":"2023"},{"key":"2024092007524254600_ocae177-B44","author":"Sivarajkumar"},{"key":"2024092007524254600_ocae177-B45","first-page":"972","article-title":"HealthPrompt: a zero-shot learning paradigm for clinical natural language processing","volume":"2022","author":"Sivarajkumar","year":"2022","journal-title":"AMIA Annu Symp Proc."},{"key":"2024092007524254600_ocae177-B46","volume-title":"The State of Health Equity in Pennsylvania","author":"Pennsylvania Department of Health","year":"2019"},{"issue":"1","key":"2024092007524254600_ocae177-B47","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12911-018-0723-6","article-title":"A clinical text classification paradigm using weak supervision and deep representation","volume":"19","author":"Wang","year":"2019","journal-title":"BMC Med Inform Decis"},{"key":"2024092007524254600_ocae177-B48","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1016\/j.jpsychires.2021.01.052","article-title":"Using weak supervision and deep learning to classify clinical notes for identification of current suicidal ideation","volume":"136","author":"Cusick","year":"2021","journal-title":"J Psychiatr Res"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/10\/2217\/59206318\/ocae177.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/10\/2217\/59206318\/ocae177.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T03:53:24Z","timestamp":1726804404000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/10\/2217\/7713266"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,13]]},"references-count":48,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2024,7,13]]},"published-print":{"date-parts":[[2024,10,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae177","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2022.03.29.22273078","asserted-by":"object"}]},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,10]]},"published":{"date-parts":[[2024,7,13]]}}}