{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T23:10:05Z","timestamp":1790205005701,"version":"4.1.0"},"reference-count":56,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2026,2,15]],"date-time":"2026-02-15T00:00:00Z","timestamp":1771113600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background: Large language models (LLMs) such as DeepSeek-V3, Google Gemini 2.0 Pro, and ChatGPT-4o are increasingly used by patients seeking online medical information. However, their accuracy, reliability, and reproducibility in patient-facing content related to chest wall deformities (CWD) remain unclear. This study aimed to compare the performance of three contemporary LLMs in generating information on pectus excavatum, pectus carinatum, and related thoracic deformities. Methods: Eighty patient-facing questions were developed across eight thematic domains and independently submitted to each model using newly created accounts over two consecutive days. Accuracy was assessed using a validated four-point rubric by blinded physiatrists, and reproducibility was evaluated using agreement metrics and weighted Cohen\u2019s kappa. Results: ChatGPT-4o achieved the highest overall accuracy (median score: 1.20), the greatest proportion of fully accurate responses, and the lowest hallucination rate (5.0%). Gemini showed intermediate accuracy, while DeepSeek-V3 demonstrated the lowest accuracy and highest hallucination rate (11.25%). Across all models, general-information and quality-of-life domains had the best performance, whereas treatment-related questions showed the most errors. Reproducibility was highest for ChatGPT-4o (weighted \u03ba = almost perfect), followed by Gemini and DeepSeek-V3. Inter-rater reliability was substantial (Fleiss\u2019 \u03ba = 0.69). Conclusions: Contemporary LLMs can generate largely accurate and reproducible patient-facing information on CWD, with ChatGPT-4o showing the strongest overall performance. This study provides the first domain-specific comparative evaluation of LLMs in CWD and integrates reproducibility analysis alongside accuracy and reliability assessment. While these tools may support patient education, treatment-related responses require caution, and LLMs should be used as adjuncts rather than substitutes for clinical counseling.<\/jats:p>","DOI":"10.3390\/diagnostics16040589","type":"journal-article","created":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T14:09:54Z","timestamp":1771250994000},"page":"589","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Performance of ChatGPT-4o, Gemini 2.0 Pro, and DeepSeek-V3 in Patient-Facing Information on Chest Wall Deformities: A Comparative Evaluation of Accuracy, RELIABILITY, and Reproducibility"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3213-0363","authenticated-orcid":false,"given":"Deniz","family":"Oke","sequence":"first","affiliation":[{"name":"Department of Physical Medicine and Rehabilitation, Health Sciences University Gaziosmanpasa Training and Research Hospital, Istanbul 34255, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ozge Gulsum","family":"Illeez","sequence":"additional","affiliation":[{"name":"Department of Physical Medicine and Rehabilitation, Fatih Sultan Mehmet Research and Training Hospital, Istanbul 34015, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Esra","family":"Giray","sequence":"additional","affiliation":[{"name":"Department of Physical Medicine and Rehabilitation, Fatih Sultan Mehmet Research and Training Hospital, Istanbul 34015, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7184-0862","authenticated-orcid":false,"given":"Bet\u00fcl","family":"\u00c7ift\u00e7i","sequence":"additional","affiliation":[{"name":"Department of Physical Medicine and Rehabilitation, K\u0131rklareli University Faculty of Medicine, K\u0131rklareli 39060, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Saxena, A.K. (2017). Classification of Chest Wall Deformities. Chest Wall Deformities, Springer.","DOI":"10.1007\/978-3-662-53088-7"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1016\/j.ccm.2018.01.010","article-title":"Disorders of the Chest Wall","volume":"39","year":"2018","journal-title":"Clin. Chest Med."},{"key":"ref_3","first-page":"1421","article-title":"Prevalence and incidence of chest wall deformities in children below 18 years old: The first systematic review and meta-analysis","volume":"21","author":"Pu","year":"2024","journal-title":"Arch. Med. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1687","DOI":"10.21037\/jtd-23-957","article-title":"Pectus excavatum and carinatum: A narrative review of epidemiology, etiopathogenesis, clinical features, and classification","volume":"16","author":"Janssen","year":"2024","journal-title":"J. Thorac. Dis."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"David, V.L. (2022). Current Concepts in the Etiology and Pathogenesis of Pectus Excavatum in Humans\u2014A Systematic Review. J. Clin. Med., 11.","DOI":"10.3390\/jcm11051241"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.athoracsur.2023.02.059","article-title":"Pectus Excavatum: Consensus and Controversies in Clinical Practice","volume":"116","author":"Janssen","year":"2023","journal-title":"Ann. Thorac. Surg."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"692","DOI":"10.1016\/j.asjsur.2022.06.080","article-title":"Modified Nuss procedure for the treatment of pectus excavatum: Experience of 259 patients","volume":"46","author":"Li","year":"2023","journal-title":"Asian J. Surg."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Maraschi, A., Cristina Vallocchia, E., and Bhakhri, K. (2024). Chest Wall Defects: An Integrated Multidisciplinary Approach by Assessing Treatment Approaches and Tracking Clinical Progress. Common Childhood Diseases\u2014Diagnosis, Prevention and Management, IntechOpen.","DOI":"10.5772\/intechopen.1006073"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1923","DOI":"10.1177\/00031348211038585","article-title":"Analysis of Psychological Assessments Affecting Patients Undergoing Treatment for Chest Wall Deformities","volume":"89","author":"Alqadi","year":"2021","journal-title":"Am. Surg."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Fattah, F.H., Salih, A.M., Salih, A.M., Asaad, S.K., Ghafour, A.K., Bapir, R., Abdalla, B.A., Othman, S., Ahmed, S.M., and Hasan, S.J. (2025). Comparative analysis of ChatGPT and Gemini (Bard) in medical inquiry: A scoping review. Front. Digit. Health, 7.","DOI":"10.3389\/fdgth.2025.1482712"},{"key":"ref_11","first-page":"3","article-title":"Assessment of Nursing Skill and Knowledge of ChatGPT, Gemini, Microsoft Copilot, and Llama: A Comparative Study","volume":"2","author":"Dilan","year":"2024","journal-title":"Barw Med. J."},{"key":"ref_12","first-page":"14","article-title":"Role of ChatGPT and Google Bard in the Diagnosis of Psychiatric Disorders: A Comparative Study","volume":"2","author":"Yadgar","year":"2023","journal-title":"Barw Med. J."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"e27850","DOI":"10.2196\/27850","article-title":"Chatbot for Health Care and Oncology Applications Using Artificial Intelligence and Machine Learning: Systematic Review","volume":"7","author":"Xu","year":"2021","journal-title":"JMIR Cancer"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1887","DOI":"10.12669\/pjms.41.7.12183","article-title":"Exploring the role of DeepSeek-R1, ChatGPT-4, and Google Gemini in medical education: How valid and reliable are they?","volume":"41","author":"Meo","year":"2025","journal-title":"Pak. J. Med. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e43951","DOI":"10.1097\/MD.0000000000043951","article-title":"ChatGPT-4.0 or DeepSeek-V3? Comparative analysis of answers to the most frequently asked questions by total knee replacement candidate patients","volume":"104","author":"Pamuk","year":"2025","journal-title":"Medicine"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Addis, B.A., Gelaw, Y.M., Eyowas, F.A., Bogale, T.W., Aynalem, Z.B., and Guadie, H.A. (2023). \u201cTime wasted by health professionals is time not invested in patients\u201d: Time management practice and associated factors among health professionals at public hospitals in Bahir Dar, Ethiopia: A multicenter mixed method study. Front. Public Health, 11.","DOI":"10.3389\/fpubh.2023.1159275"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e188","DOI":"10.7861\/fhj.2021-0095","article-title":"Artificial intelligence in healthcare: Transforming the practice of medicine","volume":"8","author":"Bajwa","year":"2021","journal-title":"Future Healthc. J."},{"key":"ref_18","first-page":"e81871","article-title":"Evaluating the Accuracy and Reliability of Large Language Models (ChatGPT, Claude, DeepSeek, Gemini, Grok, and Le Chat) in Answering Item-Analyzed Multiple-Choice Questions on Blood Physiology","volume":"17","author":"Agarwal","year":"2025","journal-title":"Cureus"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"6","DOI":"10.3352\/jeehp.2024.21.6","article-title":"Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: A systematic scoping review","volume":"21","author":"Xu","year":"2024","journal-title":"J. Educ. Eval. Health Prof."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1276","DOI":"10.1111\/medu.15402","article-title":"A systematic review of large language models and their implications in medical education","volume":"58","author":"Lucas","year":"2024","journal-title":"Med. Educ."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1884","DOI":"10.1007\/s11606-019-05109-0","article-title":"Can Patients Trust Online Health Information? A Meta-narrative Systematic Review Addressing the Quality of Health Information on the Internet","volume":"34","author":"Daraz","year":"2019","journal-title":"J. Gen. Intern. Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.aej.2024.12.090","article-title":"AI-enhanced chatbot for improving healthcare usability and accessibility for older adults","volume":"116","author":"Khamaj","year":"2025","journal-title":"Alex. Eng. J."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"e12522","DOI":"10.2196\/12522","article-title":"Consumer Evaluation of the Quality of Online Health Information: Systematic Literature Review of Relevant Criteria and Indicators","volume":"21","author":"Sun","year":"2019","journal-title":"J. Med. Internet Res."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Xiao, Z., Liao, Q.V., Zhou, M., Grandison, T., and Li, Y. (2023, January 27\u201331). Powering an AI Chatbot with Expert Sourcing to Support Credible Health Information Access. Proceedings of the 28th International Conference on Intelligent User Interfaces, Sydney, NSW, Australia.","DOI":"10.1145\/3581641.3584031"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1007\/s11019-021-10049-w","article-title":"Chatbot breakthrough in the 2020s? An ethical reflection on the trend of automated consultations in health care","volume":"25","author":"Parviainen","year":"2021","journal-title":"Med. Health Care Philos."},{"key":"ref_26","first-page":"1","article-title":"AI-Driven Healthcare Chatbots: Enhancing Access to Medical Information and Lowering Healthcare Costs","volume":"2","author":"Kumar","year":"2023","journal-title":"J. Artif. Intell. Cloud Comput."},{"key":"ref_27","first-page":"e43314","article-title":"Assessing the Efficacy of ChatGPT in Solving Questions Based on the Core Concepts in Physiology","volume":"15","author":"Banerjee","year":"2023","journal-title":"Cureus"},{"key":"ref_28","first-page":"e42972","article-title":"Performance of Large Language Models (ChatGPT, Bing Search, and Google Bard) in Solving Case Vignettes in Physiology","volume":"15","author":"Dhanvijay","year":"2023","journal-title":"Cureus"},{"key":"ref_29","first-page":"e46222","article-title":"Evaluating ChatGPT-3.5 and Claude-2 in Answering and Explaining Conceptual Medical Physiology Multiple-Choice Questions","volume":"15","author":"Agarwal","year":"2023","journal-title":"Cureus"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"E499","DOI":"10.1503\/cjs.013222","article-title":"Indication for total knee arthroplasty based on preoperative functional score: Are we operating earlier?","volume":"66","author":"Dust","year":"2023","journal-title":"Can. J. Surg."},{"key":"ref_31","first-page":"e40977","article-title":"Analysing the Applicability of ChatGPT, Bard, and Bing to Generate Reasoning-Based Multiple-Choice Questions in Medical Physiology","volume":"15","author":"Agarwal","year":"2023","journal-title":"Cureus"},{"key":"ref_32","first-page":"94","article-title":"How Well Did ChatGPT Perform in Answering Questions on Different Topics in Gross Anatomy?","volume":"5","author":"Bolgova","year":"2023","journal-title":"Eur. J. Med. Health Sci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1441","DOI":"10.1080\/0142159X.2024.2308779","article-title":"Is ChatGPT \u2018ready\u2019 to be a learning tool for medical undergraduates and will it perform equally in different subjects? Comparative study of ChatGPT performance in tutorial and case-based learning questions in physiology and biochemistry","volume":"46","author":"Luke","year":"2024","journal-title":"Med. Teach."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Tutor, A.S., Escudero, E., Nogal \u00c1vila, M.d., Aranda, J.F., Torres, H., Yague, J.G., Borrego, M.J., Mu\u00f1oz, \u00da., S\u00e1daba, M.C., and S\u00e1nchez-Vera, I. (2023). Learning and assessment strategies to develop specific and transversal competencies for a humanized medical education. Front. Physiol., 14.","DOI":"10.3389\/fphys.2023.1212031"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1016\/j.knee.2025.06.007","article-title":"ChatGPT-4.0 and DeepSeek-R1 does not yet provide clinically supported answers for knee osteoarthritis","volume":"56","author":"Wu","year":"2025","journal-title":"Knee"},{"key":"ref_36","first-page":"1966","article-title":"A Scoping Review of Large Language Model Applications in Healthcare","volume":"329","author":"Zhang","year":"2025","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"e496","DOI":"10.4244\/EIJ-D-23-00643","article-title":"A study of ChatGPT in facilitating Heart Team decisions on severe aortic stenosis","volume":"20","author":"Salihu","year":"2024","journal-title":"EuroIntervention"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"e47621","DOI":"10.2196\/47621","article-title":"The Potential of ChatGPT as a Self-Diagnostic Tool in Common Orthopedic Diseases: Exploratory Study","volume":"25","author":"Kuroiwa","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1016\/j.ajem.2024.04.024","article-title":"How artificial intelligence could transform emergency care","volume":"81","author":"Kachman","year":"2024","journal-title":"Am. J. Emerg. Med."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Jones, N. (2025). OpenAI\u2019s \u2018deep research\u2019 tool: Is it useful for scientists?. Nature, Online ahead of print.","DOI":"10.1038\/d41586-025-00377-9"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zuccon, G., Koopman, B., and Shaik, R. (2023, January 26\u201329). ChatGPT Hallucinates when Attributing Answers. Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region 2023, Beijing, China.","DOI":"10.1145\/3624918.3625329"},{"key":"ref_42","first-page":"e44720","article-title":"A Call to Address AI \u201cHallucinations\u201d and How Healthcare Professionals Can Mitigate Their Risks","volume":"15","author":"Hatem","year":"2023","journal-title":"Cureus"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1038\/d41586-025-00229-6","article-title":"China\u2019s cheap, open AI model DeepSeek thrills scientists","volume":"638","author":"Gibney","year":"2025","journal-title":"Nature"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1412","DOI":"10.1038\/s41433-024-02958-w","article-title":"Google DeepMind\u2019s gemini AI versus ChatGPT: A comparative analysis in ophthalmology","volume":"38","author":"Masalkhi","year":"2024","journal-title":"Eye"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1038\/d41586-025-00259-0","article-title":"How China created AI model DeepSeek and shocked the world","volume":"638","author":"Conroy","year":"2025","journal-title":"Nature"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1386","DOI":"10.2106\/JBJS.N.01141","article-title":"Prevalence of Total Hip and Knee Replacement in the United States","volume":"97","author":"Larson","year":"2015","journal-title":"J. Bone Jt. Surg.-Am."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1519","DOI":"10.2106\/JBJS.23.00209","article-title":"Assessing ChatGPT Responses to Common Patient Questions Regarding Total Hip Arthroplasty","volume":"105","author":"Mika","year":"2023","journal-title":"J. Bone Jt. Surg."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Li, J., Huang, J., Zheng, L., and Li, X. (2020). Application of Artificial Intelligence in Diabetes Education and Management: Present Status and Promising Prospect. Front. Public Health, 8.","DOI":"10.3389\/fpubh.2020.00173"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"e70349","DOI":"10.1111\/liv.70349","article-title":"From Guidelines to Real-Time Conversation: Expert-Validated Retrieval-Augmented and Fine-Tuned GPT-4 for Hepatitis C Management","volume":"45","author":"Giuffre","year":"2025","journal-title":"Liver Int."},{"key":"ref_50","first-page":"e2500011","article-title":"Enhancing Oncology-Specific Question Answering With Large Language Models Through Fine-Tuned Embeddings With Synthetic Data","volume":"9","author":"Lu","year":"2025","journal-title":"JCO Clin. Cancer Inf."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Lee, R.C., Hadidchi, R., Coard, M.C., Rubinov, Y., Alamuri, T., Liaw, A., Chandrupatla, R., and Duong, T.Q. (J. Am. Coll. Radiol., 2025). Use of Large Language Models on Radiology Reports: A Scoping Review, J. Am. Coll. Radiol., in press.","DOI":"10.1016\/j.jacr.2025.10.005"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1016\/j.admecli.2004.06.002","article-title":"Chest wall anomalies: Pectus excavatum and pectus carinatum","volume":"15","author":"Goretsky","year":"2004","journal-title":"Adolesc. Med. Clin."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.yapd.2024.05.001","article-title":"Pectus Disorders: Excavatum, Carinatum and Arcuatum","volume":"71","author":"Notrica","year":"2024","journal-title":"Adv. Pediatr."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"1835","DOI":"10.1016\/j.jpedsurg.2021.01.027","article-title":"Ten-year experience with staged management of pectus carinatum: Results and lessons learned","volume":"56","author":"Edward","year":"2021","journal-title":"J. Pediatr. Surg."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1016\/S0022-3468(03)00123-4","article-title":"Apilot study of the impact of surgical repair on disease-specific quality of life among patients with pectus excavatum","volume":"38","author":"Lawson","year":"2003","journal-title":"J. Pediatr. Surg."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"2257","DOI":"10.1016\/j.jpedsurg.2019.03.019","article-title":"REJr Development validation of the Pectus Carinatum Body Image Quality of Life (PeCBI-QOL) questionnaire","volume":"54","author":"Paulson","year":"2019","journal-title":"J. Pediatr. Surg."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/4\/589\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T14:24:13Z","timestamp":1771251853000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/4\/589"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,15]]},"references-count":56,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["diagnostics16040589"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16040589","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,15]]}}}