{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T12:20:09Z","timestamp":1774441209660,"version":"3.50.1"},"reference-count":36,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T00:00:00Z","timestamp":1774310400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Marcelle Gauvreau Engineering Research Chair in Multimodal Health Monitoring and Early Disease Detection with Hearables at \u00c9cole de technologie sup\u00e9rieure"},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["CREATE 543087-2020"],"award-info":[{"award-number":["CREATE 543087-2020"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Swallowing is a critical marker of neurological and emotional health. The ability to monitor it continuously and non-invasively, especially through smart ear-worn devices, holds significant promise for clinical applications. Despite this potential, no public audio datasets currently support reliable swallowing sound detection. Existing datasets focus primarily on speech and breathing, offering limited coverage and lacking detailed annotations for swallowing events. To address this gap, we introduce an in-ear audio dataset specifically designed to capture a wide range of verbal and non-verbal sounds. It includes comprehensive labeling focused on swallowing. The dataset was collected from 34 healthy adults (14 females and 20 males) between the ages of 20 and 29. Each participant performed a series of predefined tasks involving both non-verbal and verbal events. Non-verbal tasks included swallowing, clicking, forceful blinking, touching the scalp, and physical movements such as squatting or walking in place. Verbal tasks consisted of speaking (e.g., describing an image). Recordings were conducted in both quiet and noisy environments to better reflect real-world conditions. Data were captured using a combination of in-\/outer-ear microphones, a chest belt to record electrocardiogram (ECG), respiration and acceleration signals, and an ultrasound probe to track tongue movement, which served as a reference for swallowing annotation. All signals were precisely synchronized. To ensure high data quality, the recordings were reviewed using both algorithmic analysis and manual inspection. Swallowing events were identified based on ultrasound signals and validated by an expert to guarantee accurate labeling. As a proof of concept that in-ear audio supports swallow classification, we fine-tune a fully connected neural network on YAMNet embeddings plus zero-crossing rate (ZCR) features. Across the completed folds, the model reaches an F1 score of 0.875 \u00b1 0.013.<\/jats:p>","DOI":"10.3390\/s26072019","type":"journal-article","created":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T12:58:59Z","timestamp":1774357139000},"page":"2019","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Multimodal In-Ear Audio and Physiological Dataset for Swallowing and Non-Verbal Event Classification"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-1915-1319","authenticated-orcid":false,"given":"Elyes","family":"Ben Cheikh","sequence":"first","affiliation":[{"name":"The Research in Hearing Health and Assistive Devices (RHAD) Lab, \u00c9cole de technologie sup\u00e9rieure (\u00c9TS), Montreal, QC H3C 1K3, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yassine","family":"Mrabet","sequence":"additional","affiliation":[{"name":"The Research in Hearing Health and Assistive Devices (RHAD) Lab, \u00c9cole de technologie sup\u00e9rieure (\u00c9TS), Montreal, QC H3C 1K3, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Catherine","family":"Laporte","sequence":"additional","affiliation":[{"name":"Laboratoire de Traitement de l\u2019Information en Sant\u00e9 (LATIS), \u00c9cole de technologie sup\u00e9rieure (\u00c9TS), Montreal, QC H3C 1K3, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0460-5798","authenticated-orcid":false,"given":"Rachel E.","family":"Bouserhal","sequence":"additional","affiliation":[{"name":"The Research in Hearing Health and Assistive Devices (RHAD) Lab, \u00c9cole de technologie sup\u00e9rieure (\u00c9TS), Montreal, QC H3C 1K3, Canada"},{"name":"Laboratoire de Traitement de l\u2019Information en Sant\u00e9 (LATIS), \u00c9cole de technologie sup\u00e9rieure (\u00c9TS), Montreal, QC H3C 1K3, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,3,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1016\/j.pmr.2008.06.001","article-title":"Anatomy and Physiology of Feeding and Swallowing: Normal and Abnormal","volume":"19","author":"Matsuo","year":"2008","journal-title":"Phys. Med. Rehabil. Clin. N. Am."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"516","DOI":"10.3390\/signals5030028","article-title":"Biometric Vibration Signal Detection Devices for Swallowing Activity Monitoring","volume":"5","author":"Kang","year":"2024","journal-title":"Signals"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Isaacson, J., Patel, S., Torres-Yaghi, Y., and Pag\u00e1n, F. (2020). Sialorrhea in Parkinson\u2019s Disease. Toxins, 12.","DOI":"10.3390\/toxins12110691"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1093\/qjmed\/89.2.109","article-title":"Clinical measurement of swallowing in health and in neurogenic dysphagia","volume":"89","author":"Hughes","year":"1996","journal-title":"QJM Mon. J. Assoc. Physicians"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1007\/BF02065410","article-title":"Spontaneous swallowing rate and emotional state","volume":"40","author":"Cuevas","year":"1995","journal-title":"Dig. Dis. Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1080\/10408398.2011.649810","article-title":"Using Sensory Properties of Food to Trigger Swallowing: A Review","volume":"55","author":"Loret","year":"2013","journal-title":"Crit. Rev. Food Sci. Nutr."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"6013","DOI":"10.1038\/s41598-019-42422-4","article-title":"Effect of attention on chewing and swallowing behaviors in healthy humans","volume":"9","author":"Ashiga","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"116169","DOI":"10.1016\/j.sna.2024.116169","article-title":"Effect of age and bolus characteristics on the acoustic parameters of ear and neck swallowing sounds","volume":"383","author":"Asakura","year":"2025","journal-title":"Sens. Actuators A Phys."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2404211","DOI":"10.1002\/advs.202404211","article-title":"Automatic Clinical Assessment of Swallowing Behavior and Diagnosis of Silent Aspiration Using Wireless Multimodal Wearable Electronics","volume":"11","author":"Shin","year":"2024","journal-title":"Adv. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1038\/s41746-024-01417-w","article-title":"Multimodal deep ensemble classification system with wearable vibration sensor for detecting throat-related events","volume":"8","author":"Song","year":"2025","journal-title":"npj Digit. Med."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Gravellier, L., Le Coz, M., Farinas, J., and Pinquier, J. (2024, January 15\u201319). Detection of Pharyngolaryngeal Activities in Real-World Settings Using Wearable Sensors. Proceedings of the 2024 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC53108.2024.10782007"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kimura, S., Emoto, T., Suzuki, Y., Shinkai, M., Shibagaki, A., and Shichijo, F. (2024). Novel Approach Combining Shallow Learning and Ensemble Learning for the Automated Detection of Swallowing Sounds in a Clinical Database. Sensors, 24.","DOI":"10.3390\/s24103057"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1371","DOI":"10.1007\/s00455-025-10833-3","article-title":"Use of Transfer Learning for the Automated Segmentation and Detection of Swallows via Digital Cervical Auscultation in Children","volume":"40","author":"So","year":"2025","journal-title":"Dysphagia"},{"key":"ref_14","first-page":"16","article-title":"OpenEarable 2.0: Open-Source Earphone Platform for Physiological Ear Sensing","volume":"9","author":"Lepold","year":"2025","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"107643","DOI":"10.1016\/j.apacoust.2020.107643","article-title":"Detection and classification of human-produced nonverbal audio events","volume":"171","author":"Chabot","year":"2021","journal-title":"Appl. Acoust."},{"key":"ref_16","unstructured":"Butkow, K.J., Dang, T., Ferlini, A., Ma, D., and Mascolo, C. (2021). Motion-resilient Heart Rate Monitoring with In-ear Microphones. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1256","DOI":"10.1109\/TBME.2017.2720463","article-title":"In-Ear Audio Wearable: Measurement of Heart and Breathing Rates for Health and Safety Monitoring","volume":"65","author":"Martin","year":"2018","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mehrban, M.H.K., Voix, J., and Bouserhal, R.E. (2024). Classification of Breathing Phase and Path with In-Ear Microphones. Sensors, 24.","DOI":"10.3390\/s24206679"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3499","DOI":"10.1109\/TASLP.2023.3313433","article-title":"Configurable EBEN: Extreme Bandwidth Extension Network to Enhance Body-Conducted Speech Capture","volume":"31","author":"Hauret","year":"2023","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yoshimoto, T., Taniguchi, K., Kurose, S., and Kimura, Y. (2022). Validation of Earphone-Type Sensors for Non-Invasive and Objective Swallowing Function Assessment. Sensors, 22.","DOI":"10.3390\/s22145176"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.anl.2024.11.003","article-title":"Intra-aural swallowing sound analysis with simultaneous videofluoroscopy and cervical swallowing sound recording","volume":"52","author":"Haji","year":"2024","journal-title":"Auris Nasus Larynx"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jayatilake, D., Teramoto, Y., Ueno, T., and Matsuo, K. (2026). Validation of automated 5 mL thin liquid swallowing sound segmentation for estimating audio-derived pharyngeal clearance time. Sci. Rep.","DOI":"10.1038\/s41598-026-39699-7"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1109\/JBHI.2016.2625271","article-title":"A Novel Chewing Detection System Based on PPG, Audio, and Accelerometry","volume":"21","author":"Papapanagiotou","year":"2017","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1069","DOI":"10.1121\/1.5091777","article-title":"An in-ear speech database in varying conditions of the audio-phonation loop","volume":"145","author":"Bouserhal","year":"2019","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Hauret, J., Olivier, M., Joubaud, T., Langrenne, C., Poir\u00e9e, S., Zimpfer, V., and Bavu, E. (2024). Vibravox: A Dataset of French Speech Captured with Body-conduction Audio Sensors. arXiv.","DOI":"10.1016\/j.specom.2025.103238"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Liu, Y., Butkow, K.J., Stuchbury-Wass, J., Pullin, A., Ma, D., and Mascolo, C. (2024). RespEar: Earable-Based Robust Respiratory Rate Monitoring. arXiv.","DOI":"10.1109\/PerCom64205.2025.00026"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ma, D., Ferlini, A., and Mascolo, C. (2021). OESense: Employing Occlusion Effect for In-ear Human Sensing. arXiv.","DOI":"10.1145\/3458864.3467680"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"101244","DOI":"10.1016\/j.measen.2024.101244","article-title":"Measurement of respiratory rate with cardiac belt: Metrological characterization","volume":"34","author":"Panni","year":"2024","journal-title":"Meas. Sens."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1519\/JSC.0000000000003046","article-title":"Reliability of Zephyr BioHarness Respiratory Rate at Rest, During the Modified Canadian Aerobic Fitness Test and Recovery","volume":"34","author":"Nazari","year":"2019","journal-title":"J. Strength Cond. Res."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Maeda, K., Nagasaka, M., Nagano, A., Nagami, S., Hashimoto, K., Kamiya, M., Masuda, Y., Ozaki, K., and Kawamura, K. (2023). Ultrasonography for Eating and Swallowing Assessment: A Narrative Review of Integrated Insights for Noninvasive Clinical Practice. Nutrients, 15.","DOI":"10.3390\/nu15163560"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1111\/1460-6984.12703","article-title":"Swallowing assessment in patients with dysphagia: Validity and reliability of a pocket-sized ultrasound system","volume":"57","author":"Winiker","year":"2022","journal-title":"Int. J. Lang. Commun. Disord."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1007\/BF02407271","article-title":"Test-retest variability in normal swallowing","volume":"4","author":"Lof","year":"1990","journal-title":"Dysphagia"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Bernier, A., and Voix, J. (2013, January 2\u20137). An active hearing protection device for musicians. Proceedings of the Meetings on Acoustics, Montreal, QC, Canada.","DOI":"10.1121\/1.4800066"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1433","DOI":"10.1121\/1.4864484","article-title":"Three-dimensional finite element modeling of the human external ear: Simulation study of the bone conduction occlusion effect","volume":"135","author":"Brummund","year":"2014","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Gemmeke, J.F., Ellis, D.P.W., Freedman, D., Jansen, A., Lawrence, W., Moore, R.C., Plakal, M., and Ritter, M. (2017, January 5\u20139). Audio Set: An ontology and human-labeled dataset for audio events. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.","DOI":"10.1109\/ICASSP.2017.7952261"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1007\/s00455-013-9456-y","article-title":"Timing Differences Between Cued and Noncued Swallows in Healthy Young Adults","volume":"28","author":"Nagy","year":"2013","journal-title":"Dysphagia"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/7\/2019\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T09:22:05Z","timestamp":1774430525000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/7\/2019"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,24]]},"references-count":36,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,4]]}},"alternative-id":["s26072019"],"URL":"https:\/\/doi.org\/10.3390\/s26072019","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,24]]}}}