{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,3]],"date-time":"2026-05-03T02:05:48Z","timestamp":1777773948694,"version":"3.51.4"},"reference-count":32,"publisher":"Public Library of Science (PLoS)","issue":"4","license":[{"start":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T00:00:00Z","timestamp":1777420800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:sec id=\"sec001\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Large Language Models (LLMs) have been increasingly adopted in healthcare settings, yet comparative evaluations of their performance in standardized medical assessments remain limited. This study aims to evaluate the accuracy and consistency of four LLMs in answering Advanced Cardiac Life Support (ACLS) questions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec002\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In this observational study, 50 ACLS questions were categorized as knowledge-based (n\u2009=\u200929), visual content (n\u2009=\u200912), or case-based (n\u2009=\u20099). Each question was posed to ChatGPT-4o, Gemini 2.0, Claude 3.5, and DeepSeek R1 on three separate occasions to assess consistency. Performance was evaluated using three accuracy metrics: overall accuracy (all three responses correct), strict accuracy (at least two responses correct), and ideal accuracy (at least one response correct).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec003\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>ChatGPT-4o demonstrated superior performance with 100% accuracy across all categories and perfect consistency (Fleiss\u2019 Kappa\u2009=\u20091.0). Claude 3.5 achieved 92.0% overall accuracy with excellent consistency (Fleiss\u2019 Kappa\u2009=\u20090.89). Gemini 2.0 showed 86.0% overall accuracy with moderate consistency (Fleiss\u2019 Kappa\u2009=\u20090.58). DeepSeek R1 performed lowest at 70.0% overall accuracy with moderate consistency (Fleiss\u2019 Kappa\u2009=\u20090.58) and failed completely on visual content questions (0%). All models achieved 100% accuracy on knowledge-based questions. Performance differences were statistically significant across models (p\u2009&lt;\u20090.001).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec004\">\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>LLMs demonstrate variable capabilities in ACLS knowledge assessment, with ChatGPT-4o showing exceptional performance. While these models show promise as supplementary tools in resuscitation education and clinical decision support, significant variations in visual recognition capabilities and response consistency highlight the importance of critical evaluation before clinical implementation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1371\/journal.pone.0347611","type":"journal-article","created":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T17:32:48Z","timestamp":1777483968000},"page":"e0347611","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Differential performance of large language models in advanced cardiac life support assessment: A comprehensive multi-dimensional analysis of accuracy, consistency, and visual recognition capabilities"],"prefix":"10.1371","volume":"21","author":[{"given":"Murat","family":"Genc","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5629-3143","authenticated-orcid":true,"given":"Bensu","family":"Bulut","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6320-9667","authenticated-orcid":true,"given":"Medine","family":"Akkan \u00d6z","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9521-1120","authenticated-orcid":true,"given":"Ay\u015fenur","family":"G\u00fcr","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mehmet","family":"Yortanl\u0131","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bet\u00fcl \u00c7i\u011fdem","family":"Yortanl\u0131","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Oguz","family":"Sariyildiz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ramiz","family":"Yazici","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1930-3293","authenticated-orcid":true,"given":"H\u00fcseyin","family":"Mutlu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zekeriya","family":"Uykan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2026,4,29]]},"reference":[{"key":"pone.0347611.ref001","volume-title":"Cardiovascular diseases (CVDs) [Internet]","author":"World Health Organization","year":"2025"},{"issue":"8","key":"pone.0347611.ref002","doi-asserted-by":"crossref","DOI":"10.1161\/CIR.0000000000001303","article-title":"Heart disease and stroke statistics: a report of US and global data from the American Heart Association","volume":"151","author":"SS Martin","year":"2025","journal-title":"Circulation"},{"key":"pone.0347611.ref003","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.resuscitation.2021.02.003","article-title":"European resuscitation council guidelines 2021: executive summary","volume":"161","author":"GD Perkins","year":"2021","journal-title":"Resuscitation"},{"key":"pone.0347611.ref004","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1016\/j.resuscitation.2018.06.005","article-title":"Adherence to advanced cardiovascular life support (ACLS) guidelines during in-hospital cardiac arrest is associated with improved outcomes","volume":"129","author":"K Honarmand","year":"2018","journal-title":"Resuscitation"},{"key":"pone.0347611.ref005","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/j.resuscitation.2020.05.042","article-title":"The association between ACLS guideline deviations and outcomes from in-hospital cardiac arrest","volume":"153","author":"CP Crowley","year":"2020","journal-title":"Resuscitation"},{"key":"pone.0347611.ref006","doi-asserted-by":"crossref","DOI":"10.1161\/CIR.0000000000000916","article-title":"2020 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care","volume":"142","author":"AR Panchal","year":"2020","journal-title":"Circulation"},{"issue":"11","key":"pone.0347611.ref007","doi-asserted-by":"crossref","DOI":"10.1161\/CIRCOUTCOMES.123.010491","article-title":"Ten steps toward improving in-hospital cardiac arrest quality of care and outcomes","volume":"16","author":"BK Nallamothu","year":"2023","journal-title":"Circ Cardiovasc Qual Outcomes"},{"key":"pone.0347611.ref008","doi-asserted-by":"crossref","first-page":"109732","DOI":"10.1016\/j.resuscitation.2023.109732","article-title":"Can ChatGPT pass the life support exams without entering the American Heart Association course?","volume":"185","author":"N Fija\u010dko","year":"2023","journal-title":"Resuscitation"},{"key":"pone.0347611.ref009","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1007\/s42979-022-01043-x","article-title":"AI-based modeling: techniques, applications and research issues towards automation, intelligent and smart systems","volume":"3","author":"IH Sarker","year":"2022","journal-title":"SN Comput Sci"},{"issue":"21","key":"pone.0347611.ref010","doi-asserted-by":"crossref","first-page":"2906","DOI":"10.3390\/healthcare11212906","article-title":"The role of ChatGPT in the advancement of diagnosis, management, and prognosis of cardiovascular and cerebrovascular disease","volume":"11","author":"D-D Chlorogiannis","year":"2023","journal-title":"Healthcare (Basel)"},{"issue":"1","key":"pone.0347611.ref011","doi-asserted-by":"crossref","first-page":"11429","DOI":"10.1038\/s41598-024-82948-w","article-title":"Evaluation of correctness and reliability of GPT, Bard, and Bing chatbots\u2019 responses in basic life support scenarios","volume":"15","author":"S Aqavil-Jahromi","year":"2025","journal-title":"Sci Rep"},{"key":"pone.0347611.ref012","doi-asserted-by":"crossref","DOI":"10.1016\/j.resuscitation.2020.09.010","article-title":"Adult basic life support: international consensus on cardiopulmonary resuscitation and emergency cardiovascular care science with treatment recommendations","volume":"156","author":"TM Olasveengen","year":"2020","journal-title":"Resuscitation"},{"key":"pone.0347611.ref013","volume-title":"Advanced Cardiovascular Life Support for Experienced Providers [Internet]","author":"American Heart Association","year":"2025"},{"issue":"1","key":"pone.0347611.ref014","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1038\/s43856-023-00370-1","article-title":"The future landscape of large language models in medicine","volume":"3","author":"J Clusmann","year":"2023","journal-title":"Commun Med (Lond)"},{"key":"pone.0347611.ref015","doi-asserted-by":"crossref","first-page":"110451","DOI":"10.1016\/j.resuscitation.2024.110451","article-title":"ChatGPT and pediatric advanced life support: a performance evaluation","volume":"205","author":"K Kokulu","year":"2024","journal-title":"Resuscitation"},{"key":"pone.0347611.ref016","doi-asserted-by":"crossref","first-page":"102963","DOI":"10.1016\/j.inffus.2025.102963","article-title":"A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics","volume":"118","author":"K He","year":"2025","journal-title":"Inf Fusion"},{"key":"pone.0347611.ref017","doi-asserted-by":"crossref","first-page":"109545","DOI":"10.1016\/j.compbiomed.2024.109545","article-title":"Generating credible referenced medical research: a comparative study of openAI\u2019s GPT-4 and Google\u2019s gemini","volume":"185","author":"M Omar","year":"2025","journal-title":"Comput Biol Med"},{"key":"pone.0347611.ref018","doi-asserted-by":"crossref","first-page":"1456911","DOI":"10.3389\/fdgth.2024.1456911","article-title":"Comparative study of Claude 3.5-Sonnet and human physicians in generating discharge summaries for patients with renal insufficiency: assessment of efficiency, accuracy, and quality","volume":"6","author":"H Jin","year":"2024","journal-title":"Front Digit Health"},{"key":"pone.0347611.ref019","article-title":"How does DeepSeek-R1 perform on USMLE?","author":"L Faray de Paiva","year":"2025","journal-title":"medRxiv"},{"key":"pone.0347611.ref020","article-title":"Taxonomy of educational objectives: The classification of educational goals","volume-title":"Handbook I: Cognitive domain","author":"BS Bloom","year":"1956"},{"key":"pone.0347611.ref021","volume-title":"A taxonomy for learning, teaching, and assessing: A revision of Bloom\u2019s taxonomy of educational objectives","author":"LW Anderson","year":"2001"},{"issue":"1","key":"pone.0347611.ref022","doi-asserted-by":"crossref","first-page":"1187","DOI":"10.1186\/s12903-025-06444-x","article-title":"Exploring ChatGPT\u2019s potential in diagnosing oral and maxillofacial pathologies: a study of 123 challenging cases","volume":"25","author":"M Tassoker","year":"2025","journal-title":"BMC Oral Health"},{"issue":"1","key":"pone.0347611.ref023","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/s10916-023-02019-x","article-title":"\u201cChatGPT, Can You Help Me Save My Child\u2019s Life?\u201d - Diagnostic Accuracy and Supportive Capabilities to Lay Rescuers by ChatGPT in Prehospital Basic Life Support and Paediatric Advanced Life Support Cases - An In-silico Analysis","volume":"47","author":"S Bushuven","year":"2023","journal-title":"J Med Syst"},{"issue":"2","key":"pone.0347611.ref024","doi-asserted-by":"crossref","first-page":"102644","DOI":"10.1016\/j.jormas.2025.102644","article-title":"Can large language models perform clinical anamnesis? Comparative evaluation of ChatGPT, Claude, and Gemini in diagnostic reasoning through case-based questioning in oral and maxillofacial disorders","volume":"127","author":"BE Yilmaz","year":"2026","journal-title":"J Stomatol Oral Maxillofac Surg"},{"issue":"1","key":"pone.0347611.ref025","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s00276-025-03723-8","article-title":"Evaluation of the performance of different large language models on head and neck anatomy questions in the dentistry specialization exam in Turkey","volume":"47","author":"BN Gokkurt Yilmaz","year":"2025","journal-title":"Surg Radiol Anat"},{"issue":"1","key":"pone.0347611.ref026","doi-asserted-by":"crossref","first-page":"19","DOI":"10.14744\/hnhj.2024.92979","article-title":"Assessing the competence of the ChatGPT-3.5 artificial intelligence system in executing the ACLS protocol of the AHA 2020","volume":"65","author":"\u0130 Altunda\u011f","year":"2025","journal-title":"Haydarpasa Numune Med J"},{"key":"pone.0347611.ref027","doi-asserted-by":"crossref","first-page":"109783","DOI":"10.1016\/j.resuscitation.2023.109783","article-title":"ChatGPT can pass the AHA exams: open-ended questions outperform multiple-choice format","volume":"188","author":"L Zhu","year":"2023","journal-title":"Resuscitation"},{"issue":"6","key":"pone.0347611.ref028","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1016\/j.annemergmed.2023.06.022","article-title":"The natural language artificial intelligence revolution in emergency medicine: unleash the potential, addressing the concerns","volume":"82","author":"E Bignami","year":"2023","journal-title":"Ann Emerg Med"},{"key":"pone.0347611.ref029","doi-asserted-by":"crossref","DOI":"10.2196\/48785","article-title":"Opportunities, challenges, and future directions of generative artificial intelligence in medical education: scoping review","volume":"9","author":"C Preiksaitis","year":"2023","journal-title":"JMIR Med Educ"},{"issue":"6","key":"pone.0347611.ref030","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1017\/S1049023X23006568","article-title":"Large Language Model (LLM)-powered chatbots fail to generate guideline-consistent content on resuscitation and may provide potentially harmful advice","volume":"38","author":"AA Birkun","year":"2023","journal-title":"Prehosp Disaster Med"},{"issue":"5","key":"pone.0347611.ref031","doi-asserted-by":"crossref","DOI":"10.1161\/JAHA.117.006124","article-title":"Cardiopulmonary resuscitation training disparities in the United States","volume":"6","author":"AL Blewer","year":"2017","journal-title":"J Am Heart Assoc"},{"issue":"2","key":"pone.0347611.ref032","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1111\/1742-6723.12720","article-title":"Public cardiopulmonary resuscitation training rates and awareness of hands-only cardiopulmonary resuscitation: a cross-sectional survey of Victorians","volume":"29","author":"JE Bray","year":"2017","journal-title":"Emerg Med Australas"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0347611","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T17:32:54Z","timestamp":1777483974000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0347611"}},"subtitle":[],"editor":[{"given":"Carla","family":"Pegoraro","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2026,4,29]]},"references-count":32,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,4,29]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0347611","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,29]]}}}