{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T02:04:07Z","timestamp":1771293847001,"version":"3.50.1"},"reference-count":59,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,1,28]],"date-time":"2025-01-28T00:00:00Z","timestamp":1738022400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Artificial intelligence (AI) models trained on audio data may have the potential to rapidly perform clinical tasks, enhancing medical decision-making and potentially improving outcomes through early detection. Existing technologies depend on limited datasets collected with expensive recording equipment in high-income countries, which challenges deployment in resource-constrained, high-volume settings where audio data may have a profound impact on health equity.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>This report introduces a novel protocol for audio data collection and a corresponding application that captures health information through guided questions.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>To demonstrate the potential of Voice EHR as a biomarker of health, initial experiments on data quality and multiple case studies are presented in this report. Large language models (LLMs) were used to compare transcribed Voice EHR data with data (from the same patients) collected through conventional techniques like multiple choice questions. Information contained in the Voice EHR samples was consistently rated as equally or more relevant to a health evaluation.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>The HEAR application facilitates the collection of an audio electronic health record (\u201cVoice EHR\u201d) that may contain complex biomarkers of health from conventional voice\/respiratory features, speech patterns, and spoken language with semantic meaning and longitudinal context\u2013potentially compensating for the typical limitations of unimodal clinical datasets.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fdgth.2024.1448351","type":"journal-article","created":{"date-parts":[[2025,1,28]],"date-time":"2025-01-28T14:06:17Z","timestamp":1738073177000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Voice EHR: introducing multimodal audio data for health"],"prefix":"10.3389","volume":"6","author":[{"given":"James","family":"Anibal","sequence":"first","affiliation":[]},{"given":"Hannah","family":"Huth","sequence":"additional","affiliation":[]},{"given":"Ming","family":"Li","sequence":"additional","affiliation":[]},{"given":"Lindsey","family":"Hazen","sequence":"additional","affiliation":[]},{"given":"Veronica","family":"Daoud","sequence":"additional","affiliation":[]},{"given":"Dominique","family":"Ebedes","sequence":"additional","affiliation":[]},{"given":"Yen Minh","family":"Lam","sequence":"additional","affiliation":[]},{"given":"Hang","family":"Nguyen","sequence":"additional","affiliation":[]},{"given":"Phuc Vo","family":"Hong","sequence":"additional","affiliation":[]},{"given":"Michael","family":"Kleinman","sequence":"additional","affiliation":[]},{"given":"Shelley","family":"Ost","sequence":"additional","affiliation":[]},{"given":"Christopher","family":"Jackson","sequence":"additional","affiliation":[]},{"given":"Laura","family":"Sprabery","sequence":"additional","affiliation":[]},{"given":"Cheran","family":"Elangovan","sequence":"additional","affiliation":[]},{"given":"Balaji","family":"Krishnaiah","sequence":"additional","affiliation":[]},{"given":"Lee","family":"Akst","sequence":"additional","affiliation":[]},{"given":"Ioan","family":"Lina","sequence":"additional","affiliation":[]},{"given":"Iqbal","family":"Elyazar","sequence":"additional","affiliation":[]},{"given":"Lenny","family":"Ekawati","sequence":"additional","affiliation":[]},{"given":"Stefan","family":"Jansen","sequence":"additional","affiliation":[]},{"given":"Richard","family":"Nduwayezu","sequence":"additional","affiliation":[]},{"given":"Charisse","family":"Garcia","sequence":"additional","affiliation":[]},{"given":"Jeffrey","family":"Plum","sequence":"additional","affiliation":[]},{"given":"Jacqueline","family":"Brenner","sequence":"additional","affiliation":[]},{"given":"Miranda","family":"Song","sequence":"additional","affiliation":[]},{"given":"Emily","family":"Ricotta","sequence":"additional","affiliation":[]},{"given":"David","family":"Clifton","sequence":"additional","affiliation":[]},{"given":"C. Louise","family":"Thwaites","sequence":"additional","affiliation":[]},{"given":"Yael","family":"Bensoussan","sequence":"additional","affiliation":[]},{"given":"Bradford","family":"Wood","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2025,1,28]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1186\/s12875-021-01522-9","article-title":"Primary care professionals\u2019 experiences during the first wave of the COVID-19 pandemic in Greece: a qualitative study","volume":"22","author":"Smyrnakis","year":"2021","journal-title":"BMC Fam Pract"},{"key":"B2","year":""},{"key":"B3","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1186\/s12960-020-0448-3","article-title":"Physician workforce in the United States of America: forecasting nationwide shortages","volume":"18","author":"Zhang","year":"2020","journal-title":"Hum Resour Health"},{"key":"B4","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1007\/s00268-013-2324-y","article-title":"Shortage of doctors, shortage of data: a review of the global surgery, obstetrics, and anesthesia workforce literature","volume":"38","author":"Hoyler","year":"2014","journal-title":"World J Surg"},{"key":"B5","doi-asserted-by":"publisher","first-page":"160","DOI":"10.7812\/TPP\/23.015","article-title":"Time out: the impact of physician burnout on patient care quality and safety in perioperative medicine","volume":"27","author":"Shin","year":"2023","journal-title":"Perm J"},{"key":"B6","doi-asserted-by":"publisher","first-page":"e2336745","DOI":"10.1001\/jamanetworkopen.2023.36745","article-title":"Patterns in physician burnout in a stable-linked cohort","volume":"6","author":"Ortega","year":"2023","journal-title":"JAMA Netw Open"},{"key":"B7","doi-asserted-by":"publisher","first-page":"579","DOI":"10.1080\/01459740.2023.2240942","article-title":"Mistrustful dependency: mistrust as risk management in an Italian emergency department","volume":"42","author":"Pasquini","year":"2023","journal-title":"Med Anthropol"},{"key":"B8","article-title":"Gpt-4 technical report","year":""},{"key":"B9","article-title":"Llama 2: open foundation and fine-tuned chat models","author":"Touvron","year":""},{"key":"B10","article-title":"Llava-med: training a large language-and-vision assistant for biomedicine in one day","volume":"36","author":"Li","year":"2024","journal-title":"Adv Neural Inf Process Syst"},{"key":"B11","year":""},{"key":"B12","doi-asserted-by":"publisher","first-page":"e0000022","DOI":"10.1371\/journal.pdig.0000022","article-title":"Sources of bias in artificial intelligence that perpetuate healthcare disparities\u2014a global review","volume":"1","author":"Celi","year":"2022","journal-title":"PLoS Digit Health"},{"key":"B13","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1038\/s41746-024-01212-7","article-title":"Disparities in clinical studies of AI enabled applications from a global perspective","volume":"7","author":"Yang","year":"2024","journal-title":"NPJ Digit Med"},{"key":"B14","doi-asserted-by":"publisher","first-page":"191","DOI":"10.1007\/s40506-020-00229-2","article-title":"Challenges in implementing surveillance tools of high-income countries (HICs) in low middle income countries (LMICs)","volume":"12","author":"Jayatilleke","year":"2020","journal-title":"Curr Treat Options Infect Dis"},{"key":"B15","doi-asserted-by":"publisher","first-page":"108769","DOI":"10.1016\/j.drugalcdep.2021.108769","article-title":"When national drug surveys \u201ctake too long\u201d: an examination of who is at risk for survey fatigue","volume":"225","author":"Le","year":"2021","journal-title":"Drug Alcohol Depend"},{"key":"B16","doi-asserted-by":"publisher","first-page":"102992","DOI":"10.1016\/j.jdeveco.2022.102992","article-title":"Exhaustive or exhausting? Evidence on respondent fatigue in long surveys","volume":"161","author":"Jeong","year":"2023","journal-title":"J Dev Econ"},{"key":"B17","doi-asserted-by":"publisher","first-page":"103362","DOI":"10.1016\/j.jbi.2019.103362","article-title":"Investigating voice as a biomarker: deep phenotyping methods for early detection of Parkinson\u2019s disease","volume":"104","author":"Tracy","year":"2020","journal-title":"J Biomed Inform"},{"key":"B18","doi-asserted-by":"publisher","first-page":"831428","DOI":"10.3389\/fneur.2022.831428","article-title":"Voice in Parkinson\u2019s disease: a machine learning study","volume":"13","author":"Suppa","year":"2022","journal-title":"Front Neurol"},{"key":"B19","doi-asserted-by":"publisher","first-page":"210","DOI":"10.4258\/hir.2022.28.3.210","article-title":"Machine learning smart system for Parkinson disease classification using the voice as a biomarker","volume":"28","author":"Tougui","year":"2022","journal-title":"Healthc Inform Res"},{"key":"B20","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1159\/000515346","article-title":"Voice for health: the use of vocal biomarkers from research to clinical practice","volume":"5","author":"Fagherazzi","year":"2021","journal-title":"Digit Biomark"},{"key":"B21","doi-asserted-by":"publisher","first-page":"92","DOI":"10.47852\/bonviewJDSIS3202831","article-title":"Voice biomarkers for Parkinson\u2019s disease prediction using machine learning models with improved feature reduction techniques","volume":"1","author":"Chintalapudi","year":"2023","journal-title":"J Data Sci Intell Syst"},{"key":"B22","doi-asserted-by":"publisher","first-page":"e6856","DOI":"10.1002\/cpe.6856","article-title":"An optimal asthma disease detection technique for voice signal using hybrid machine learning technique","volume":"34","author":"Asim Iqbal","year":"2022","journal-title":"Concurr Comput Pract Exp"},{"key":"B23","doi-asserted-by":"publisher","first-page":"4713043","DOI":"10.1016\/j.artmed.2024.102953","article-title":"COPDVD: automated classification of chronic obstructive pulmonary disease on a new developed and evaluated voice dataset","author":"Idrisoglu","year":"2024","journal-title":"Artif Intell Med"},{"key":"B24","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1007\/s42979-023-02524-3","article-title":"A novel artificial intelligence system for the prediction of interstitial lung diseases","volume":"5","author":"Raju","year":"2024","journal-title":"SN Comput Sci"},{"key":"B25","doi-asserted-by":"publisher","first-page":"500","DOI":"10.3390\/bioengineering10040500","article-title":"A review of voice-based pain detection in adults using artificial intelligence","volume":"10","author":"Borna","year":"2023","journal-title":"Bioengineering"},{"key":"B26","doi-asserted-by":"publisher","first-page":"109782","DOI":"10.1016\/j.diabres.2022.109782","article-title":"Scoping review of the relationship between diabetes and voice quality","volume":"185","author":"Saghiri","year":"2022","journal-title":"Diabetes Res Clin Pract"},{"key":"B27","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1177\/01945998221110839","article-title":"Artificial intelligence and laryngeal cancer: from screening to prognosis: a state of the art review","volume":"168","author":"Bensoussan","year":"2023","journal-title":"Otolaryngol Head Neck Surg"},{"key":"B28","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1001\/jamaoto.2023.4807","article-title":"Voice as an AI biomarker of health\u2014introducing audiomics","author":"Bensoussan","year":"2024","journal-title":"JAMA Otolaryngol Head Neck Surg"},{"key":"B29","article-title":"COVID-19 patient detection from telephone quality speech data","author":"Ritwik","year":""},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1155\/2022\/6093613","article-title":"Speech as a biomarker for COVID-19 detection using machine learning","volume":"2022","author":"Usman","year":"2022","journal-title":"Comput Intell Neurosci"},{"key":"B31","doi-asserted-by":"publisher","first-page":"65750","DOI":"10.1109\/ACCESS.2021.3075571","article-title":"Exploring the use of artificial intelligence techniques to detect the presence of coronavirus COVID-19 through speech and voice analysis","volume":"9","author":"Verde","year":"2021","journal-title":"IEEE Access"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1007\/s13369-021-06041-4","article-title":"Artificial intelligence techniques for the non-invasive detection of COVID-19 through the analysis of voice signals","volume":"48","author":"Verde","year":"2021","journal-title":"Arab J Sci Eng"},{"key":"B33","doi-asserted-by":"crossref","DOI":"10.21437\/Interspeech.2022-10389","article-title":"Analyzing the impact of SARS-CoV-2 variants on respiratory sound signals","author":"Bhattacharya","year":""},{"key":"B34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0262448","article-title":"Detection of COVID-19 in smartphone-based breathing recordings: a pre-screening deep learning tool","volume":"17","author":"Alkhodari","year":"2022","journal-title":"PLoS One"},{"key":"B35","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1038\/s41746-021-00553-x","article-title":"Sounds of COVID-19: exploring realistic performance of audio-based digital testing","volume":"5","author":"Han","year":"2022","journal-title":"NPJ Digit Med"},{"key":"B36","doi-asserted-by":"publisher","first-page":"19149","DOI":"10.1038\/s41598-021-98742-x","article-title":"Identifying individuals with recent COVID-19 through voice classification using deep learning","volume":"11","author":"Suppakitjanusant","year":"2021","journal-title":"Sci Rep"},{"key":"B37","doi-asserted-by":"crossref","DOI":"10.1101\/2022.09.13.22279673","article-title":"Omicron detection with large language models and YouTube audio data","author":"Anibal","year":""},{"key":"B38","doi-asserted-by":"publisher","first-page":"108289","DOI":"10.1016\/j.patcog.2021.108289","article-title":"AI-Based human audio processing for COVID-19: a comprehensive overview","volume":"122","author":"Deshpande","year":"2022","journal-title":"Pattern Recognit"},{"key":"B39","article-title":"Hi sigma, do I have the coronavirus? Call for a new artificial intelligence approach to support health care professionals dealing with the covid-19 pandemic","author":"Subirana","year":""},{"key":"B40","doi-asserted-by":"crossref","DOI":"10.1145\/3394486.3412865","article-title":"Exploring automatic diagnosis of COVID-19 from crowdsourced respiratory sound data","author":"Brown","year":"2020"},{"key":"B41","doi-asserted-by":"publisher","first-page":"104944","DOI":"10.1016\/j.compbiomed.2021.104944","article-title":"Detection of COVID-19 from voice, cough and breathing patterns: dataset and preliminary results","volume":"138","author":"Despotovic","year":"2021","journal-title":"Comput Biol Med"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1136\/bmjinnov-2021-000668","article-title":"End-to-end convolutional neural network enables COVID-19 detection from breath and cough audio: a pilot study","volume":"7","author":"Coppock","year":"2021","journal-title":"BMJ Innov"},{"key":"B43","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1038\/s42256-023-00773-8","article-title":"Audio-based AI classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers","volume":"6","author":"Coppock","year":"2024","journal-title":"Nat Mach Intell"},{"key":"B44","article-title":"Hundreds of AI tools have been built to catch covid","author":"Heaven","year":""},{"key":"B45","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1038\/s41597-023-02266-0","article-title":"Coswara: a respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infection","volume":"10","author":"Bhattacharya","year":"2023","journal-title":"Sci Data"},{"key":"B46","article-title":"COVYT: introducing the coronavirus YouTube and TikTok speech dataset featuring the same speakers with and without infection","author":"Triantafyllopoulos","year":""},{"key":"B47","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1159\/000316142","article-title":"The effect of smoking on the dysphonia severity index in females","volume":"63","author":"Awan","year":"2011","journal-title":"Folia Phoniatr Logop"},{"key":"B48","year":""},{"key":"B49","article-title":"Robust speech recognition via large-scale weak supervision","author":"Radford","year":"2023"},{"key":"B50","volume-title":"Voice and Articulation Drillbook","author":"Fairbanks","year":"1960"},{"key":"B51","doi-asserted-by":"publisher","first-page":"1493","DOI":"10.1109\/JBHI.2015.2480838","article-title":"Estimation of respiratory rates using the built-in microphone of a smartphone or headset","volume":"20","author":"Nam","year":"2015","journal-title":"IEEE J Biomed Health Inform"},{"key":"B52","doi-asserted-by":"crossref","DOI":"10.1101\/2024.10.17.24315634","article-title":"Transformers for rapid detection of airway stenosis and stridor","author":"Anibal","year":""},{"key":"B53","doi-asserted-by":"publisher","first-page":"e2440969","DOI":"10.1001\/jamanetworkopen.2024.40969","article-title":"Large language model influence on diagnostic reasoning: a randomized clinical trial","volume":"7","author":"Goh","year":"2024","journal-title":"JAMA Netw Open"},{"key":"B54","year":""},{"key":"B55","doi-asserted-by":"publisher","first-page":"1081","DOI":"10.1016\/j.otc.2007.05.010","article-title":"Effects of medications on the voice","volume":"40","author":"Abaza","year":"2007","journal-title":"Otolaryngol Clin North Am"},{"key":"B56","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1111\/j.1442-2050.2009.00992.x","article-title":"Effectiveness of voice therapy in reflux-related voice disorders","volume":"23","author":"Vashani","year":"2010","journal-title":"Dis Esophagus"},{"key":"B57","doi-asserted-by":"publisher","first-page":"101","DOI":"10.5152\/eurasianjmed.2018.18331","article-title":"Voice characteristics in patients with thyroid disorders","volume":"51","author":"Junuzovi\u0107-\u017duni\u0107","year":"2019","journal-title":"Eurasian J Med"},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.1530\/EC-21-0505","article-title":"Voice changes in reproductive disorders, thyroid disorders and diabetes: a review","volume":"11","author":"Stogowska","year":"2022","journal-title":"Endocr Connect"},{"key":"B59","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1016\/j.jvoice.2003.05.004","article-title":"Acoustic analysis of voice in multiple sclerosis patients","volume":"18","author":"Feij\u00f3","year":"2004","journal-title":"J Voice"}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2024.1448351\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,28]],"date-time":"2025-01-28T14:06:20Z","timestamp":1738073180000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2024.1448351\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,28]]},"references-count":59,"alternative-id":["10.3389\/fdgth.2024.1448351"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2024.1448351","relation":{},"ISSN":["2673-253X"],"issn-type":[{"value":"2673-253X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,28]]},"article-number":"1448351"}}