{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T10:16:55Z","timestamp":1784629015321,"version":"3.55.0"},"reference-count":61,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,10,1]],"date-time":"2024-10-01T00:00:00Z","timestamp":1727740800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>The development of machine learning models for symptom-based health checkers is a rapidly evolving area with significant implications for healthcare. Accurate and efficient diagnostic tools can enhance patient outcomes and optimize healthcare resources. This study focuses on evaluating and optimizing machine learning models using a dataset of 10 diseases and 9,572 samples.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>The dataset was divided into training and testing sets to facilitate model training and evaluation. The following models were selected and optimized: Decision Tree, Random Forest, Naive Bayes, Logistic Regression and K-Nearest Neighbors. Evaluation metrics included accuracy, F1 scores, and 10-fold cross-validation. ROC-AUC and precision-recall curves were also utilized to assess model performance, particularly in scenarios with imbalanced datasets. Clinical vignettes were employed to gauge the real-world applicability of the models.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The performance of the models was evaluated using accuracy, F1 scores, and 10-fold cross-validation. The use of ROC-AUC curves revealed that model performance improved with increasing complexity. Precision-recall curves were particularly useful in evaluating model sensitivity in imbalanced dataset scenarios. Clinical vignettes demonstrated the robustness of the models in providing accurate diagnoses.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>The study underscores the importance of comprehensive model evaluation techniques. The use of clinical vignette testing and analysis of ROC-AUC and precision-recall curves are crucial in ensuring the reliability and sensitivity of symptom-based health checkers. These techniques provide a more nuanced understanding of model performance and highlight areas for further improvement.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>This study highlights the significance of employing diverse evaluation metrics and methods to ensure the robustness and accuracy of machine learning models in symptom-based health checkers. The integration of clinical vignettes and the analysis of ROC-AUC and precision-recall curves are essential steps in developing reliable and sensitive diagnostic tools.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frai.2024.1397388","type":"journal-article","created":{"date-parts":[[2024,10,1]],"date-time":"2024-10-01T05:10:48Z","timestamp":1727759448000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Enhancing diagnostic accuracy in symptom-based health checkers: a comprehensive machine learning approach with clinical vignettes and benchmarking"],"prefix":"10.3389","volume":"7","author":[{"given":"Leila","family":"Aissaoui Ferhi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manel","family":"Ben Amar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fethi","family":"Choubani","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ridha","family":"Bouallegue","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,10,1]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"1268","DOI":"10.3390\/healthcare11091268","article-title":"A review on electronic health record text-Mining for Biomedical Name Entity Recognition in healthcare domain","volume":"11","author":"Ahmad","year":"2023","journal-title":"Healthcare"},{"key":"ref54","doi-asserted-by":"publisher","DOI":"10.1007\/s11036-024-02369-x","article-title":"Empowering Medical Diagnosis: A Machine Learning Approach for Symptom-Based Health Checker","author":"Aissaoui Ferhi","year":"2024","journal-title":"Mob. Netw. Appl."},{"key":"ref2","doi-asserted-by":"publisher","first-page":"1769","DOI":"10.1007\/s11277-019-06651-0","article-title":"Energy efficiency optimization for wireless body area networks under 802.15.6 standard","volume":"109","author":"Aissaoui Ferhi","year":"2019","journal-title":"Wirel. Pers. Commun."},{"key":"ref3","doi-asserted-by":"publisher","first-page":"909","DOI":"10.54393\/pbmj.v6i07.909","article-title":"Telemedicine and telehealth solutions","volume":"2023","author":"Alwazzan","year":"2023","journal-title":"Pak. Biomed. J."},{"key":"ref4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-023-00727-2","article-title":"A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications","volume":"10","author":"Alzubaidi","year":"2023","journal-title":"J. Big Data"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1109\/RBME.2021.3131358","article-title":"Interpreting deep machine learning models: an easy guide for oncologists","volume":"16","author":"Amorim","year":"2021","journal-title":"IEEE Rev. Biomed. Eng."},{"key":"ref6","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1111\/resp.13700_23","article-title":"Case study: 43-year-old male presents with PERTUSSIS (whooping cough)","volume":"24","author":"Anh","year":"2019","journal-title":"Respirology"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"603","DOI":"10.18203\/2349-3933.IJAM20211063","article-title":"Asthma related to gastroesophageal reflux disease: a case report and review","volume":"8","author":"Atmaja","year":"2021","journal-title":"Int. J. Adv. Med."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"864","DOI":"10.59287\/icpis.864","article-title":"eHealth and smart solutions framework for health monitoring in the course of the pandemic","volume":"2023","author":"Balogh","year":"2023","journal-title":"Int. Conf. Pioneer Innov. Stud."},{"key":"ref9","doi-asserted-by":"publisher","first-page":"1317","DOI":"10.1001\/jama.2017.18391","article-title":"Big data and machine learning in health care","volume":"319","author":"Beam","year":"2018","journal-title":"JAMA"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1017\/S0950268819000268","article-title":"Online symptom checker diagnostic and triage accuracy for HIV and hepatitis C","volume":"147","author":"Berry","year":"2019","journal-title":"Epidemiol. Infect."},{"key":"ref11","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1016\/j.jaad.2022.06.034","article-title":"Online symptom checkers lack diagnostic accuracy for skin rashes","volume":"2022","author":"Berry","year":"2022","journal-title":"J. Am. Acad. Dermatol."},{"key":"ref12","doi-asserted-by":"publisher","first-page":"4088","DOI":"10.1371\/journal.pone.0254088","article-title":"Accuracy of online symptom checkers and the potential impact on service utilisation","volume":"16","author":"Ceney","year":"2020","journal-title":"PLoS One"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"703","DOI":"10.3390\/jpm13121703","article-title":"Survey of transfer learning approaches in the machine learning of digital health sensing data","volume":"13","author":"Chato","year":"2023","journal-title":"J. Pers. Med."},{"key":"ref14","doi-asserted-by":"publisher","first-page":"755","DOI":"10.3390\/app131910755","article-title":"Breast cancer prediction based on differential privacy and logistic regression optimization model","volume":"2023","author":"Chen","year":"2023","journal-title":"Appl. Sci."},{"key":"ref15","doi-asserted-by":"publisher","first-page":"6413","DOI":"10.1186\/s12864-019-6413-7","article-title":"The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation","volume":"21","author":"Chicco","year":"2020","journal-title":"BMC Genomics"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"30594","DOI":"10.1609\/aaai.v38i20.30594","article-title":"Temporal logic explanations for dynamic decision systems using anchors and Monte Carlo tree search (abstract reprint)","volume":"2024","author":"Chiu","year":"2024","journal-title":"AAAI Conf. Artif. Intell."},{"key":"ref17","doi-asserted-by":"publisher","first-page":"841","DOI":"10.3390\/s21206841","article-title":"Big machinery data Preprocessing methodology for data-driven models in prognostics and health management","volume":"21","author":"Cofre-Martel","year":"2021","journal-title":"Sensors (Basel, Switzerland)"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"697","DOI":"10.1038\/s41586-022-04569-5","article-title":"SARS-CoV-2 is associated with changes in brain structure in UK biobank","volume":"604","author":"Douaud","year":"2022","journal-title":"Nature"},{"key":"ref19","author":"Fauziyyah","year":"2020"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"3063","DOI":"10.1186\/s13023-024-03063-7","article-title":"Performance and clinical utility of a new supervised machine-learning pipeline in detecting rare ciliopathy patients based on deep phenotyping from electronic health records and semantic similarity","volume":"19","author":"Faviez","year":"2024","journal-title":"Orphanet J. Rare Dis."},{"key":"ref21","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1007\/978-3-030-39512-4_68","article-title":"Narrative review of the role of wearable devices in promoting health behavior: based on health belief model","volume":"2020","author":"Fei","year":"2020","journal-title":"Int. Conf. Intell. Hum. Syst. Integr."},{"key":"ref22","author":"Gada","year":"2021"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"991","DOI":"10.1007\/s11517-020-02132-w","article-title":"Use of a K-nearest neighbors model to predict the development of type 2 diabetes within 2 years in an obese, hypertensive population","volume":"58","author":"Garc\u00eda-Carretero","year":"2020","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref24","doi-asserted-by":"publisher","first-page":"1972","DOI":"10.1117\/12.2581972","article-title":"COVID-19 pneumonia diagnosis using chest x-ray radiograph and deep learning","volume":"2021","author":"Griner","year":"2021","journal-title":"Med. Imaging"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"145369","DOI":"10.5812\/jcma-145369","article-title":"Machine learning-guided Anesthesiology: a review of recent advances and clinical applications","volume":"2024","author":"Hashemi","year":"2024","journal-title":"J. Cell. Mol. Anesth."},{"key":"ref26","author":"Heaney","year":"2020"},{"key":"ref27","author":"Jia","year":"2023"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"1899","DOI":"10.1038\/s41597-022-01899-x","article-title":"MIMIC-IV, a freely accessible electronic health record dataset","volume":"10","author":"Johnson","year":"2023","journal-title":"Sci. Data"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"1130","DOI":"10.32628\/ijsrst52411130","article-title":"Data pre-processing technique for enhancing healthcare data quality using artificial intelligence","volume":"2024","author":"Kale","year":"2024","journal-title":"Int. J. Sci. Res. Sci. Technol."},{"key":"ref30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s42979-022-01597-w","article-title":"Literature survey and an idea comprehension on prediction of hysterectomy in women using natural language processing and deep learning technique for electronic health record","volume":"4","author":"Kumar","year":"2023","journal-title":"SN Comput. Sci."},{"key":"ref31","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1080\/00949655.2023.2238235","article-title":"Implications of imbalanced datasets for empirical ROC-AUC estimation in binary classification tasks","volume":"94","author":"Liu","year":"2023","journal-title":"J. Stat. Comput. Simul."},{"key":"ref32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/10749357.2019.1659640","article-title":"Association between caregiver engagement and patient-reported healthcare utilization after stroke: a mixed-methods study","volume":"27","author":"Liu","year":"2020","journal-title":"Top. Stroke Rehabil."},{"key":"ref33","doi-asserted-by":"publisher","first-page":"2020","DOI":"10.1136\/bmjoq-2022-002020","article-title":"Thematic reviews of patient safety incidents as a tool for systems thinking: a quality improvement report","volume":"12","author":"Machen","year":"2023","journal-title":"BMJ Open Qual."},{"key":"ref34","author":"Marcio","year":""},{"key":"ref35","author":"Marcio","year":""},{"key":"ref36","doi-asserted-by":"publisher","first-page":"910","DOI":"10.1136\/thx.2003.011080","article-title":"Respiratory bronchiolitis associated interstitial lung disease (RB-ILD): a case of an acute presentation","volume":"59","author":"Mavridou","year":"2004","journal-title":"Thorax"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"737","DOI":"10.1101\/2024.03.04.24303737","article-title":"Generation of guideline-based clinical decision trees in oncology using large language models","volume":"2024","author":"Miao","year":"2024","journal-title":"medRxiv"},{"key":"ref38","doi-asserted-by":"publisher","first-page":"459","DOI":"10.1080\/07370024.2022.2057858","article-title":"Exploring the effectiveness of persuasive games for disease prevention and awareness and the impact of tailoring to the stages of change","volume":"38","author":"Mulchandani","year":"2022","journal-title":"Hum. Comput. Interact."},{"key":"ref39","doi-asserted-by":"publisher","first-page":"275","DOI":"10.47102\/annals-acadmedsg.v33n2p275","article-title":"A case report of occupational asthma due to gluteraldehyde exposure","volume":"33","author":"Ong","year":"2004","journal-title":"Ann. Acad. Med. Singap."},{"key":"ref40","doi-asserted-by":"publisher","first-page":"1982","DOI":"10.1007\/s10916-022-01892-2","article-title":"Automating electronic health record data quality assessment","volume":"47","author":"Ozonze","year":"2023","journal-title":"J. Med. Syst."},{"key":"ref41","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1101\/2023.02.09.23285633","article-title":"Explainable machine learning predictions of perceptual sensitivity for retinal prostheses","volume":"2023","author":"Pogoncheff","year":"2023","journal-title":"medRxiv"},{"key":"ref42","doi-asserted-by":"publisher","first-page":"1139","DOI":"10.52783\/jes.1139","article-title":"Web services performance prediction with confusion matrix and K-fold cross validation to provide prior service quality characteristics","volume":"2024","author":"Prakash","year":"2024","journal-title":"J. Electr. Syst."},{"key":"ref43","doi-asserted-by":"publisher","first-page":"e20190297","DOI":"10.1590\/1983-1447.2020.20190297","article-title":"Online data collection strategies used in qualitative research of the health field: a scoping review","volume":"41","author":"Salvador","year":"2020","journal-title":"Rev. Gaucha Enferm."},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.1016\/j.jaci.2006.11.502","article-title":"A case of chronic Rhinosinusitis","author":"Sattar","year":"2007","journal-title":"J. Allergy Clin. Immunol."},{"key":"ref45","doi-asserted-by":"publisher","first-page":"480","DOI":"10.1136\/bmj.h3480","article-title":"Evaluation of symptom checkers for self diagnosis and triage: audit study","volume":"351","author":"Semigran","year":"2015","journal-title":"BMJ"},{"key":"ref46","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1007\/s41133-020-00032-0","article-title":"A comparative analysis of logistic regression, random Forest and KNN models for the text classification","volume":"5","author":"Shah","year":"2020","journal-title":"Augment. Hum. Res."},{"key":"ref47","doi-asserted-by":"publisher","first-page":"571","DOI":"10.1001\/jamaophthalmol.2019.0571","article-title":"Accuracy of a popular online symptom checker for ophthalmic diagnoses","volume":"2019","author":"Shen","year":"2019","journal-title":"JAMA Ophthalmol."},{"key":"ref48","doi-asserted-by":"publisher","first-page":"1749","DOI":"10.3390\/ijerph17051749","article-title":"Effectiveness of Mobile phone-based interventions for improving health outcomes in patients with chronic heart failure: a systematic review and meta-analysis","volume":"17","author":"Son","year":"2020","journal-title":"Int. J. Environ. Res. Public Health"},{"key":"ref49","doi-asserted-by":"publisher","first-page":"473","DOI":"10.1007\/s12553-022-00643-0","article-title":"Expert system based on fuzzy rules for diagnosing breast cancer","volume":"12","author":"Thani","year":"2022","journal-title":"Heal. Technol."},{"key":"ref50","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1016\/j.mcna.2019.08.011","article-title":"Cough: a practical and multifaceted approach to diagnosis and management","volume":"104","author":"Tran","year":"2020","journal-title":"Med. Clin. North Am."},{"key":"ref51","author":"Tripathi","year":"2021"},{"key":"ref52","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1177\/1062860605274520","article-title":"Clinical vignette-based surveys: a tool for assessing physician practice variation","volume":"20","author":"Veloski","year":"2005","journal-title":"Am. J. Med. Qual."},{"key":"ref53","author":"Vida","year":"2022"},{"key":"ref55","doi-asserted-by":"publisher","DOI":"10.7861\/futurehosp.6-2-94","article-title":"The potential for artificial intelligence in healthcare","author":"Wen","year":"2023","journal-title":"J. Commer. Biotechnol."},{"key":"ref56","doi-asserted-by":"publisher","first-page":"863","DOI":"10.1186\/s12874-019-0863-0","article-title":"Clinical risk prediction with random forests for survival, longitudinal, and multivariate (RF-SLAM) data analysis","volume":"20","author":"Wongvibulsin","year":"2019","journal-title":"BMC Med. Res. Methodol."},{"key":"ref57","doi-asserted-by":"publisher","first-page":"386","DOI":"10.2196\/29386","article-title":"The impact of explanations on layperson Trust in Artificial Intelligence\u2013Driven Symptom Checker Apps: experimental study","volume":"23","author":"Woodcock","year":"2021","journal-title":"J. Med. Internet Res."},{"key":"ref58","doi-asserted-by":"publisher","first-page":"1328","DOI":"10.1136\/bmj.m1328","article-title":"Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal","volume":"369","author":"Wynants","year":"2020","journal-title":"BMJ"},{"key":"ref59","doi-asserted-by":"publisher","first-page":"1815","DOI":"10.1080\/01605682.2022.2118630","article-title":"Generalized mixed prediction chain model and its application in forecasting chronic complications","volume":"74","author":"You","year":"2022","journal-title":"J. Oper. Res. Soc."},{"key":"ref60","doi-asserted-by":"publisher","first-page":"619","DOI":"10.3390\/math10193619","article-title":"A survey on deep transfer learning and beyond","volume":"2022","author":"Yu","year":"2022","journal-title":"Mathematics"},{"key":"ref61","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1108\/el-11-2021-0204","article-title":"Analysing scientific publications in the field of mobile information systems using bibliometric analysis","volume":"40","author":"Zhang","year":"2022","journal-title":"Electron. Libr."}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1397388\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,3]],"date-time":"2024-10-03T12:00:52Z","timestamp":1727956852000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1397388\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,1]]},"references-count":61,"alternative-id":["10.3389\/frai.2024.1397388"],"URL":"https:\/\/doi.org\/10.3389\/frai.2024.1397388","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,1]]},"article-number":"1397388"}}