{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T06:21:39Z","timestamp":1791872499800,"version":"4.3.4"},"reference-count":60,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,8,2]],"date-time":"2024-08-02T00:00:00Z","timestamp":1722556800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["NRF-2022R1A2C1007780"],"award-info":[{"award-number":["NRF-2022R1A2C1007780"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100016275","name":"Seoul National University Bundang Hospital","doi-asserted-by":"publisher","award":["18-2022-0006 14-2022-0017"],"award-info":[{"award-number":["18-2022-0006 14-2022-0017"]}],"id":[{"id":"10.13039\/100016275","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Bioeng. Biotechnol."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>This study aimed to identify differences in voice characteristics and changes between patients with dysphagia-aspiration and healthy individuals using a deep learning model, with a focus on under-researched areas of pre- and post-swallowing voice changes in patients with dysphagia. We hypothesized that these variations may be due to weakened muscles and blocked airways in patients with dysphagia.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>A prospective cohort study was conducted on 198 participants aged &amp;gt;40 years at the Seoul National University Bundang Hospital from October 2021 to February 2023. Pre- and post-swallowing voice data of the participants were converted to a 64-kbps mp3 format, and all voice data were trimmed to a length of 2\u00a0s. The data were divided for 10-fold cross-validation and stored in HDF5 format with anonymized IDs and labels for the normal and aspiration groups. During preprocessing, the data were converted to Mel spectrograms, and the EfficientAT model was modified using the final layer of MobileNetV3 to effectively detect voice changes and analyze pre- and post-swallowing voices. This enabled the model to probabilistically categorize new patient voices as normal or aspirated.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>In a study of the machine-learning model for aspiration detection, area under the receiver operating characteristic curve (AUC) values were analyzed across sexes under different configurations. The average AUC values for males ranged from 0.8117 to 0.8319, with the best performance achieved at a learning rate of 3.00e-5 and a batch size of 16. The average AUC values for females improved from 0.6975 to 0.7331, with the best performance observed at a learning rate of 5.00e-5 and a batch size of 32. As there were fewer female participants, a combined model was developed to maintain the sex balance. In the combined model, the average AUC values ranged from 0.7746 to 0.7997, and optimal performance was achieved at a learning rate of 3.00e-5 and a batch size of 16.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>This study evaluated a voice analysis-based program to detect pre- and post-swallowing changes in patients with dysphagia, potentially aiding in real-time monitoring. Such a system can provide healthcare professionals with daily insights into the conditions of patients, allowing for personalized interventions.<\/jats:p><\/jats:sec><jats:sec><jats:title>Clinical Trial Registration:<\/jats:title><jats:p><jats:ext-link>ClinicalTrials.gov<\/jats:ext-link>, identifier NCT05149976<\/jats:p><\/jats:sec>","DOI":"10.3389\/fbioe.2024.1433087","type":"journal-article","created":{"date-parts":[[2024,8,2]],"date-time":"2024-08-02T07:51:05Z","timestamp":1722585065000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["A deep learning approach to dysphagia-aspiration detecting algorithm through pre- and post-swallowing voice changes"],"prefix":"10.3389","volume":"12","author":[{"given":"Jung-Min","family":"Kim","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min-Seop","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sun-Young","family":"Choi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kyogu","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ju Seok","family":"Ryu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,8,2]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1138.e17","DOI":"10.1016\/j.amjmed.2015.04.026","article-title":"Approach to the patient with dysphagia","volume":"128","author":"Abdel Jalil","year":"2015","journal-title":"Am. J. Med."},{"key":"B2","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1007\/s00455-019-10064-3","article-title":"Use of the penetration-aspiration scale in dysphagia research: a systematic review","volume":"35","author":"Borders","year":"2020","journal-title":"Dysphagia"},{"key":"B3","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1016\/j.chest.2018.09.003","article-title":"The aging lung: is lung health good health for older adults?","volume":"155","author":"Bowdish","year":"2019","journal-title":"Chest"},{"key":"B4","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1016\/j.chest.2016.03.059","article-title":"Screening accuracy for aspiration using bedside water swallow tests: a systematic review and meta-analysis","volume":"150","author":"Brodsky","year":"2016","journal-title":"Chest"},{"key":"B5","doi-asserted-by":"publisher","first-page":"806","DOI":"10.1016\/j.clnu.2008.06.011","article-title":"Accuracy of the volume-viscosity swallow test for clinical screening of oropharyngeal dysphagia and aspiration","volume":"27","author":"Clave","year":"2008","journal-title":"Clin. Nutr."},{"key":"B6","doi-asserted-by":"publisher","first-page":"119","DOI":"10.4321\/s1130-01082004000200005","article-title":"Approaching oropharyngeal dysphagia","volume":"96","author":"Clave","year":"2004","journal-title":"Rev. Esp. Enferm. Dig."},{"key":"B7","doi-asserted-by":"publisher","first-page":"1212","DOI":"10.1007\/s00455-022-10546-x","article-title":"The role of aspiration amount on airway protective responses in people with neurogenic dysphagia","volume":"38","author":"Curtis","year":"2023","journal-title":"Dysphagia"},{"key":"B8","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1044\/1058-0360.0604.17","article-title":"Clinical assessment of swallowing and prediction of dysphagia severity","volume":"6","author":"Daniels","year":"1997","journal-title":"Am. J. speech-language pathology"},{"key":"B9","doi-asserted-by":"publisher","first-page":"736","DOI":"10.1007\/s00455-021-10319-y","article-title":"Using voice change as an indicator of dysphagia: a systematic review","volume":"37","author":"Dos Santos","year":"2022","journal-title":"Dysphagia"},{"key":"B10","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1378\/chest.15-0789","article-title":"Novel bedside phonetic evaluation to identify dysphagia and aspiration risk","volume":"149","author":"Festic","year":"2016","journal-title":"Chest"},{"key":"B12","first-page":"523","article-title":"The effects of mp3 compression on acoustic measurements of fundamental frequency and pitch range","author":"Fuchs","year":"2016"},{"key":"B13","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1007\/s00455-006-9046-3","article-title":"The impact of rheologically controlled materials on the identification of airway compromise on the clinical and videofluoroscopic swallowing examinations","volume":"21","author":"Groher","year":"2006","journal-title":"Dysphagia"},{"key":"B14","volume-title":"HDF5 - A New Generation of HDF The Hierarchical Data Format (Introduction to HDF5)","year":"2006"},{"key":"B15","doi-asserted-by":"publisher","first-page":"620","DOI":"10.1044\/1092-4388(2009\/08-0246)","article-title":"Perception of wet vocal quality in identifying penetration\/aspiration during swallowing","volume":"53","author":"Groves-Wright","year":"2010","journal-title":"J. Speech Lang. Hear Res."},{"key":"B16","doi-asserted-by":"publisher","first-page":"284","DOI":"10.1016\/j.radi.2022.12.007","article-title":"The use of videofluoroscopy (VFS) and fibreoptic endoscopic evaluation of swallowing (FEES) in the investigation of oropharyngeal dysphagia in stroke patients: a narrative review","volume":"29","author":"Helliwell","year":"2023","journal-title":"Radiogr. (Lond)"},{"key":"B17","first-page":"1314","article-title":"Searching for mobilenetv3","author":"Howard","year":"2019"},{"key":"B18","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1007\/s12149-012-0680-6","article-title":"Comparison between the radionuclide salivagram and videofluoroscopic swallowing study methods for evaluating patients with aspiration pneumonia","volume":"27","author":"Jang","year":"2013","journal-title":"Ann. Nucl. Med."},{"key":"B19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/JTEHM.2015.2500562","article-title":"Smartphone-based real-time assessment of swallowing ability from the swallowing sound","volume":"3","author":"Jayatilake","year":"2015","journal-title":"IEEE J. Transl. Eng. Health Med."},{"key":"B20","first-page":"656","article-title":"HDF5-based I\/O optimization for extragalactic HI data pipeline of FAST","author":"Ji","year":"2020"},{"key":"B21","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1016\/j.anl.2017.10.007","article-title":"Detection of voice changes due to aspiration via acoustic voice analysis","volume":"45","author":"Kang","year":"2018","journal-title":"Auris Nasus Larynx"},{"key":"B22","doi-asserted-by":"publisher","first-page":"105259","DOI":"10.1016\/j.bspc.2023.105259","article-title":"Non-invasive way to diagnose dysphagia by training deep learning model with voice spectrograms","volume":"86","author":"Kim","year":"2023","journal-title":"Biomed. Signal Process. Control"},{"key":"B23","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1186\/s12984-024-01329-6","article-title":"Prediction of dysphagia aspiration through machine learning-based analysis of patients\u2019 postprandial voices","volume":"21","author":"Kim","year":"2024","journal-title":"J. NeuroEngineering Rehabilitation"},{"key":"B24","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1007\/s00455-016-9775-x","article-title":"History of fiberoptic endoscopic evaluation of swallowing for evaluation and management of pharyngeal dysphagia: changes over the years","volume":"32","author":"Langmore","year":"2017","journal-title":"Dysphagia"},{"key":"B25","doi-asserted-by":"publisher","first-page":"553","DOI":"10.1016\/s0889-8553(03)00024-4","article-title":"Dysphagia: evaluation and treatment","volume":"32","author":"Lind","year":"2003","journal-title":"Gastroenterol. Clin. North Am."},{"key":"B26","doi-asserted-by":"publisher","first-page":"474","DOI":"10.1053\/hn.1999.v120.a91765","article-title":"Aspiration: cause and implications","volume":"120","author":"Lundy","year":"1999","journal-title":"Otolaryngol. Head. Neck Surg."},{"key":"B27","doi-asserted-by":"publisher","first-page":"456","DOI":"10.1136\/jnnp.63.4.456","article-title":"Predictive value of clinical indices in detecting aspiration in patients with neurological disorders","volume":"63","author":"Mari","year":"1997","journal-title":"J. Neurol. Neurosurg. Psychiatry"},{"key":"B28","doi-asserted-by":"publisher","first-page":"769","DOI":"10.1016\/j.pmr.2008.06.004","article-title":"The videofluorographic swallowing study","volume":"19","author":"Martin-Harris","year":"2008","journal-title":"Phys. Med. Rehabil. Clin. N. Am."},{"key":"B29","doi-asserted-by":"publisher","first-page":"691","DOI":"10.1016\/j.pmr.2008.06.001","article-title":"Anatomy and physiology of feeding and swallowing: normal and abnormal","volume":"19","author":"Matsuo","year":"2008","journal-title":"Phys. Med. Rehabil. Clin. N. Am."},{"key":"B30","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1080\/16501970510026999","article-title":"Identification of a simple screening tool for dysphagia in patients with stroke using factor analysis of multiple dysphagia variables","volume":"37","author":"Nishiwaki","year":"2005","journal-title":"J. Rehabil. Med."},{"key":"B31","doi-asserted-by":"publisher","first-page":"38","DOI":"10.14639\/0392-100x-2136","article-title":"The correlation between pharyngeal residue, penetration\/aspiration and nutritional modality: a cross-sectional study in patients with neurogenic dysphagia","volume":"40","author":"Nordio","year":"2020","journal-title":"ACTA Otorhinolaryngol. Ital."},{"key":"B32","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1159\/000517144","article-title":"Advanced machine learning tools to monitor biomarkers of dysphagia: a wearable sensor proof-of-concept study","volume":"5","author":"O'brien","year":"2021","journal-title":"Digit. Biomark."},{"key":"B33","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1002\/jhm.2313","article-title":"Bedside diagnosis of dysphagia: a systematic review","volume":"10","author":"O'horo","year":"2015","journal-title":"J. Hosp. Med."},{"key":"B34","doi-asserted-by":"publisher","first-page":"16682","DOI":"10.1038\/s41598-022-20348-8","article-title":"Post-stroke respiratory complications using machine learning with voice features from mobile devices","volume":"12","author":"Park","year":"2022","journal-title":"Sci. Rep."},{"key":"B35","volume-title":"Accuracy of MP3 speech recognition under real-world conditions","author":"Pollak","year":"2011"},{"key":"B36","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1097\/ANC.0000000000000245","article-title":"Fiberoptic endoscopic evaluation of swallowing: a multidisciplinary alternative for assessment of infants with dysphagia in the neonatal intensive care unit","volume":"16","author":"Reynolds","year":"2016","journal-title":"Adv. Neonatal Care"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1900","DOI":"10.3390\/foods10081900","article-title":"A systematic and a scoping review on the psychometrics and clinical utility of the volume-viscosity swallow test (V-vst) in the clinical screening and assessment of oropharyngeal dysphagia","volume":"10","author":"Riera","year":"2021","journal-title":"Foods"},{"key":"B38","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1007\/PL00009610","article-title":"Differentiation of normal and abnormal airway protection during swallowing using the penetration-aspiration scale","volume":"14","author":"Robbins","year":"1999","journal-title":"Dysphagia"},{"key":"B39","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1159\/000339979","article-title":"The volume-viscosity swallow test for clinical screening of dysphagia and aspiration","volume-title":"Stepping stones to living well with dysphagia","author":"Rofes","year":"2012"},{"key":"B40","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1007\/BF00417897","article-title":"A penetration-aspiration scale","volume":"11","author":"Rosenbek","year":"1996","journal-title":"Dysphagia"},{"key":"B41","doi-asserted-by":"publisher","first-page":"753","DOI":"10.1097\/01.phm.0000140798.97706.a5","article-title":"Prediction of laryngeal aspiration using voice analysis","volume":"83","author":"Ryu","year":"2004","journal-title":"Am. J. Phys. Med. Rehabil."},{"key":"B42","doi-asserted-by":"crossref","DOI":"10.1007\/978-981-10-5032-9","volume-title":"Dysphagia evaluation and treatment","author":"Saitoh","year":"2018"},{"key":"B43","doi-asserted-by":"publisher","first-page":"061","DOI":"10.1055\/s-0034-1394129","article-title":"Vocal variability post swallowing in individuals with and without oropharyngeal dysphagia","volume":"19","author":"Santos","year":"2015","journal-title":"Int. Archives Otorhinolaryngology"},{"key":"B44","doi-asserted-by":"publisher","first-page":"e13100","DOI":"10.1111\/nmo.13100","article-title":"The anatomy and physiology of normal and abnormal swallowing in oropharyngeal dysphagia","volume":"29","author":"Sasegbon","year":"2017","journal-title":"Neurogastroenterol. Motil."},{"key":"B45","first-page":"1","article-title":"Efficient large-scale audio tagging via transformer-to-cnn knowledge distillation","volume-title":"ICASSP 2023-2023 IEEE international Conference on acoustics, Speech and signal processing (ICASSP)","author":"Schmid","year":""},{"key":"B11","unstructured":"SchmidF.\n            TurianJ.\n            SchreiberH.\n            AshimineI. E.\n            Joemgu7\n          Efficient pre-trained CNNs for audio pattern recognition (EfficientAT)"},{"key":"B46","doi-asserted-by":"publisher","first-page":"216","DOI":"10.1007\/BF00431413","article-title":"Airway protective mechanisms: current concepts","volume":"10","author":"Shaker","year":"1995","journal-title":"Dysphagia"},{"key":"B47","doi-asserted-by":"publisher","first-page":"1478","DOI":"10.1016\/0016-5085(90)91078-k","article-title":"Coordination of deglutitive glottic closure with oropharyngeal swallowing","volume":"98","author":"Shaker","year":"1990","journal-title":"Gastroenterology"},{"key":"B48","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1159\/000358118","article-title":"Dysphagia risk assessment in acute left-hemispheric middle cerebral artery stroke","volume":"37","author":"Somasundaram","year":"2014","journal-title":"Cerebrovasc. Dis."},{"key":"B49","doi-asserted-by":"publisher","first-page":"50","DOI":"10.34160\/jkds.2022.12.1.006","article-title":"Predicting aspiration using the functions of production and quality of voice in dysphagic patients","volume":"12","author":"Song","year":"2022","journal-title":"J. Korean Dysphagia Soc."},{"key":"B50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2021\/2388697","article-title":"Digital audio scene recognition method based on machine learning technology","volume":"2021","author":"Sun","year":"2021","journal-title":"Sci. Program."},{"key":"B51","doi-asserted-by":"publisher","first-page":"126","DOI":"10.1007\/s00455-002-0095-y","article-title":"Three tests for predicting aspiration without videofluorography","volume":"18","author":"Tohara","year":"2003","journal-title":"Dysphagia"},{"key":"B52","doi-asserted-by":"publisher","first-page":"542","DOI":"10.1016\/j.jvoice.2020.07.019","article-title":"Respiratory function and voice: the role for airflow measures","volume":"36","author":"Tong","year":"2022","journal-title":"J. Voice"},{"key":"B53","doi-asserted-by":"publisher","first-page":"778","DOI":"10.3390\/nu13030778","article-title":"Nutritional assessment in adult patients with dysphagia: a scoping review","volume":"13","author":"Ueshima","year":"2021","journal-title":"Nutrients"},{"key":"B54","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1186\/1746-160x-5-9","article-title":"Surface electromyography as a screening method for evaluation of dysphagia and odynophagia","volume":"5","author":"Vaiman","year":"2009","journal-title":"Head face Med."},{"key":"B55","doi-asserted-by":"publisher","first-page":"478","DOI":"10.1177\/000348940511400610","article-title":"Timing of glottic closure during swallowing: a combined electromyographic and endoscopic analysis","volume":"114","author":"Van Daele","year":"2005","journal-title":"Ann. Otol. Rhinol. Laryngol."},{"key":"B56","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1007\/s00455-010-9282-4","article-title":"Voice-quality abnormalities as a sign of dysphagia: validation against acoustic and videofluoroscopic data","volume":"26","author":"Waito","year":"2011","journal-title":"Dysphagia"},{"key":"B57","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1007\/s004550010005","article-title":"\u201cWet voice\u201d as a predictor of penetration and aspiration in oropharyngeal dysphagia","volume":"15","author":"Warms","year":"2000","journal-title":"Dysphagia"},{"key":"B58","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1016\/j.anl.2017.08.009","article-title":"Relationship between swallowing function and breathing\/phonation","volume":"45","author":"Yamaguchi","year":"2018","journal-title":"Auris Nasus Larynx"},{"key":"B59","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1002\/cre2.223","article-title":"Evaluation of swallowing function in patients with oropharyngeal secretions","volume":"5","author":"Yamaguchi","year":"2019","journal-title":"Clin. Exp. Dent. Res."},{"key":"B60","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1177\/0194599818762328","article-title":"Utility of eating assessment tool-10 in predicting aspiration in patients with unilateral vocal fold paralysis","volume":"159","author":"Zuniga","year":"2018","journal-title":"Otolaryngol. Head. Neck Surg."}],"container-title":["Frontiers in Bioengineering and Biotechnology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fbioe.2024.1433087\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T11:40:36Z","timestamp":1724413236000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fbioe.2024.1433087\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,2]]},"references-count":60,"alternative-id":["10.3389\/fbioe.2024.1433087"],"URL":"https:\/\/doi.org\/10.3389\/fbioe.2024.1433087","relation":{},"ISSN":["2296-4185"],"issn-type":[{"value":"2296-4185","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,2]]},"article-number":"1433087"}}