{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T10:00:06Z","timestamp":1779098406244,"version":"3.51.4"},"reference-count":52,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T00:00:00Z","timestamp":1767830400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NIH","award":["R01-CA246704"],"award-info":[{"award-number":["R01-CA246704"]}]},{"name":"NIH","award":["R01-CA240639"],"award-info":[{"award-number":["R01-CA240639"]}]},{"name":"NIH","award":["U01-DK127384-02S1"],"award-info":[{"award-number":["U01-DK127384-02S1"]}]},{"name":"NIH","award":["U01-CA268808"],"award-info":[{"award-number":["U01-CA268808"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Medical Sciences"],"abstract":"<jats:p>Background: The clinical applicability of large language models (LLMs) in high-stakes cardiac emergencies remains unexplored. This study evaluated how well advanced LLMs perform in managing complex catheterization laboratory (Cath lab) scenarios and compared their performance with that of interventional cardiologists. Methods and Results: A cross-sectional study was conducted from 20 June to 2 December 2024. Twelve challenging inferior myocardial infarction scenarios were presented to seven LLMs (ChatGPT, Gemini, LLAMA, Qwen, Bing, Claude, DeepSeek) and five early-career interventional cardiologists. Responses were standardized, anonymized, and evaluated by thirty experienced interventional cardiologists. Performance comparisons were analyzed using a linear mixed-effects model with correlation and reliability statistics. Physicians had an average reference score of 80.68 (95% CI 76.3\u201385.0). Among LLMs, ChatGPT ranked highest (87.4, 95% CI 82.5\u201392.3), followed by Claude (80.8, 95% CI 75.7\u201385.9) and DeepSeek (78.7, 95% CI 72.9\u201384.6). LLAMA (73.7), Qwen (66.2), and Bing (64.3) ranked lower, while Gemini scored the lowest (59.0). ChatGPT scored higher than the early-career physician comparator group (difference 6.69, 95% CI 0.00\u201313.37; p &lt; 0.05), whereas Gemini, LLAMA, Qwen, and Bing performed significantly worse; Claude and DeepSeek showed no significant difference. Conclusions: This expanded assessment reveals significant variability in LLM performance. In this simulated setting, ChatGPT demonstrated performance comparable to that of early-career interventional cardiologists. These results suggest that LLMs could serve as supplementary decision-support tools in interventional cardiology under simulated conditions.<\/jats:p>","DOI":"10.3390\/medsci14010033","type":"journal-article","created":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T09:01:29Z","timestamp":1767862889000},"page":"33","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["AI in the Hot Seat: Head-to-Head Comparison of Large Language Models and Cardiologists in Emergency Scenarios"],"prefix":"10.3390","volume":"14","author":[{"given":"Vedat","family":"Cicek","sequence":"first","affiliation":[{"name":"Machine & Hybrid Intelligence Lab, Department of Radiology, Northwestern University, Chicago, IL 60611, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6366-8206","authenticated-orcid":false,"given":"Lili","family":"Zhao","sequence":"additional","affiliation":[{"name":"Department of Preventive Medicine, Biostatistics and Informatics Division, Northwestern University, Chicago, IL 60611, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yalcin","family":"Tur","sequence":"additional","affiliation":[{"name":"Machine & Hybrid Intelligence Lab, Department of Radiology, Northwestern University, Chicago, IL 60611, USA"},{"name":"Department of Computer Science, Stanford University, Stanford, CA 94305, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0268-9641","authenticated-orcid":false,"given":"Ahmet","family":"Oz","sequence":"additional","affiliation":[{"name":"Department of Cardiology, Istanbul Education and Research Hospital, Istanbul 34098, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3524-5396","authenticated-orcid":false,"given":"Sahhan","family":"Kilic","sequence":"additional","affiliation":[{"name":"Corlu State Hospital, Tekirdag 59860, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1608-1955","authenticated-orcid":false,"given":"Gorkem","family":"Durak","sequence":"additional","affiliation":[{"name":"Machine & Hybrid Intelligence Lab, Department of Radiology, Northwestern University, Chicago, IL 60611, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3165-6769","authenticated-orcid":false,"given":"Faysal","family":"Saylik","sequence":"additional","affiliation":[{"name":"Department of Cardiology, Van Training and Research Hospital, Health Sciences University, Van 65100, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mert Ilker","family":"Hayiroglu","sequence":"additional","affiliation":[{"name":"Department of Cardiology, Dr. Siyami Ersek Cardiovascular and Thoracic Surgery Research and Training Hospital, Istanbul 34668, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8188-5020","authenticated-orcid":false,"given":"Tufan","family":"Cinar","sequence":"additional","affiliation":[{"name":"School of Medicine, University of Maryland, Baltimore, MD 21201, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ulas","family":"Bagci","sequence":"additional","affiliation":[{"name":"Machine & Hybrid Intelligence Lab, Department of Radiology, Northwestern University, Chicago, IL 60611, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,1,8]]},"reference":[{"key":"ref_1","first-page":"370","article-title":"Comparative Evaluation of Chatbot Responses on Coronary Artery Disease","volume":"53","author":"Pay","year":"2025","journal-title":"Arch. Turk. Soc. Cardiol.\/T\u00fcrk Kardiyol. Derne\u011fi Ar\u015fivi"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1093\/eurheartj\/ehad838","article-title":"Artificial intelligence: Revolutionizing cardiology with large language models","volume":"45","author":"Boonstra","year":"2024","journal-title":"Eur. Heart J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"109713","DOI":"10.1016\/j.isci.2024.109713","article-title":"The application of large language models in medicine: A scoping review","volume":"27","author":"Meng","year":"2024","journal-title":"Iscience"},{"key":"ref_4","first-page":"533","article-title":"Large Language Models in Intracardiac Electrogram Interpretation: A New Frontier in Cardiac Diagnostics for Pacemaker Patients","volume":"29","author":"Bozyel","year":"2025","journal-title":"Anatol. J. Cardiol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"e687","DOI":"10.1161\/CIR.0000000000001300","article-title":"Evolution of critical care cardiology: An update on structure, care delivery, training, and research paradigms: A scientific statement from the American Heart Association","volume":"151","author":"Sinha","year":"2025","journal-title":"Circulation"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1038\/s43856-023-00370-1","article-title":"The future landscape of large language models in medicine","volume":"3","author":"Clusmann","year":"2023","journal-title":"Commun. Med."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Cross, J., Choma, M., and Onofrey, J. (2024). Bias in medical AI: Implications for clinical decision-making. PLoS Digit. Health, 3.","DOI":"10.1371\/journal.pdig.0000651"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Safranek, C.W., Sidamon-Eristoff, A.E., Gilson, A., and Chartash, D. (2023). The Role of Large Language Models in Medical Education: Applications and Implications, JMIR Publications.","DOI":"10.2196\/preprints.50945"},{"key":"ref_9","first-page":"371","article-title":"Large language models: Could they be the next generation of clinical decision support systems in cardiovascular diseases?","volume":"28","author":"Cesur","year":"2024","journal-title":"Anatol. J. Cardiol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"100289","DOI":"10.1016\/j.patter.2021.100289","article-title":"Structuring clinical text with AI: Old versus new natural language processing techniques evaluated on eight common cardiovascular diseases","volume":"2","author":"Zhan","year":"2021","journal-title":"Patterns"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1093\/jamia\/ocy178","article-title":"Criteria2Query: A natural language interface to clinical databases for cohort definition","volume":"26","author":"Yuan","year":"2019","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dewaswala, N., Chen, D., Bhopalwala, H., Kaggal, V.C., Murphy, S.P., Bos, J.M., Geske, J.B., Gersh, B.J., Ommen, S.R., and Araoz, P.A. (2022). Natural language processing for identification of hypertrophic cardiomyopathy patients from cardiac magnetic resonance reports. BMC Med. Inform. Decis. Mak., 22.","DOI":"10.1186\/s12911-022-02017-y"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"674","DOI":"10.1016\/S0735-1097(21)02033-7","article-title":"The use of natural language processing-based algorithms and outpatient clinical encounters for worsening heart failure: Insights from the utilize-WHF study","volume":"77","author":"Ambrosy","year":"2021","journal-title":"J. Am. Coll. Cardiol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1038\/s41746-022-00590-0","article-title":"Cohort design and natural language processing to reduce bias in electronic health records research","volume":"5","author":"Khurshid","year":"2022","journal-title":"NPJ Digit. Med."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Patterson, O.V., Freiberg, M.S., Skanderson, M., Fodeh, J.S., Brandt, C.A., and DuVall, S.L. (2017). Unlocking echocardiogram measurements for heart disease research through natural language processing. BMC Cardiovasc. Disord., 17.","DOI":"10.1186\/s12872-017-0580-8"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Shool, S., Adimi, S., Saboori Amleshi, R., Bitaraf, E., Golpira, R., and Tara, M. (2025). A systematic review of large language model (LLM) evaluations in clinical medicine. BMC Med. Inform. Decis. Mak., 25.","DOI":"10.1186\/s12911-025-02954-4"},{"key":"ref_17","unstructured":"Zhou, H., Liu, F., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S.S., Zhou, P., and Liu, J. (2023). A survey of large language models in medicine: Progress, application, and challenge. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Maity, S., and Saikia, M.J. (2025). Large Language Models in Healthcare and Medical Applications: A Review. Bioengineering, 12.","DOI":"10.3390\/bioengineering12060631"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Alkalbani, A.M., Alrawahi, A.S., Salah, A., Haghighi, V., Zhang, Y., Alkindi, S., and Sheng, Q.Z. (2025). A Systematic Review of Large Language Models in Medical Specialties: Applications, Challenges and Future Directions. Information, 16.","DOI":"10.21203\/rs.3.rs-5128451\/v2"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Artsi, Y., Sorin, V., Konen, E., Glicksberg, B.S., Nadkarni, G., and Klang, E. (2024). Large language models for generating medical examinations: Systematic review. BMC Med. Educ., 24.","DOI":"10.1186\/s12909-024-05239-y"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Pressman, S.M., Borna, S., Gomez-Cabello, C.A., Haider, S.A., Haider, C.R., and Forte, A.J. (2024). Clinical and surgical applications of large language models: A systematic review. J. Clin. Med., 13.","DOI":"10.3390\/jcm13113041"},{"key":"ref_22","first-page":"1239","article-title":"2023 ACC\/AHA\/SCAI Advanced Training Statement on Interventional Cardiology (Coronary, Peripheral Vascular, and Structural Heart Interventions) A Report of the ACC Competency Management Committee","volume":"16","author":"Committee","year":"2023","journal-title":"Cardiovasc. Interv."},{"key":"ref_23","first-page":"255","article-title":"SCAI expert consensus update on best practices in the cardiac catheterization laboratory: This statement was endorsed by the American College of Cardiology (ACC), the American Heart Association (AHA), and the Heart Rhythm Society (HRS) in April 2021","volume":"98","author":"Naidu","year":"2021","journal-title":"Catheter. Cardiovasc. Interv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Neo, J.R.E., Ser, J.S., and Tay, S.S. (2024). Use of large language model-based chatbots in managing the rehabilitation concerns and education needs of outpatient stroke survivors and caregivers. Front. Digit. Health, 6.","DOI":"10.3389\/fdgth.2024.1395501"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"102797","DOI":"10.1016\/j.cpcardiol.2024.102797","article-title":"Evaluating ChatGPT platform in delivering heart failure educational material: A comparison with the leading national cardiology institutes","volume":"49","author":"Anaya","year":"2024","journal-title":"Curr. Probl. Cardiol."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Li, Y. (2023). A practical survey on zero-shot prompt design for in-context learning. arXiv.","DOI":"10.26615\/978-954-452-092-2_069"},{"key":"ref_27","first-page":"125469","article-title":"Instance-adaptive zero-shot chain-of-thought prompting","volume":"37","author":"Yuan","year":"2024","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Gu, Y., Han, X., Liu, Z., and Huang, M. (2021). Ppt: Pre-trained prompt tuning for few-shot learning. arXiv.","DOI":"10.18653\/v1\/2022.acl-long.576"},{"key":"ref_29","first-page":"70757","article-title":"Towards revealing the mystery behind chain of thought: A theoretical perspective","volume":"36","author":"Feng","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ahmed, T., and Devanbu, P. (2023, January 11\u201315). Better patching using llm prompting, via self-consistency. Proceedings of the 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE), Luxembourg.","DOI":"10.1109\/ASE56229.2023.00065"},{"key":"ref_31","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Subedi, K. (2025). The reliability of llms for medical diagnosis: An examination of consistency, manipulation, and contextual awareness. arXiv.","DOI":"10.56147\/aaiet.1.1.3"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"113503","DOI":"10.1016\/j.knosys.2025.113503","article-title":"A comprehensive survey on integrating large language models with knowledge-based methods","volume":"318","author":"Yang","year":"2025","journal-title":"Knowl.-Based Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1015","DOI":"10.1093\/jamia\/ocaf045","article-title":"Evaluating the effectiveness of biomedical fine-tuning for large language models on clinical tasks","volume":"32","author":"Dorfner","year":"2025","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Xu, F., Hao, Q., Zong, Z., Wang, J., Zhang, Y., Wang, J., Lan, X., Gong, J., Ouyang, T., and Meng, F. (2025). Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv.","DOI":"10.1016\/j.patter.2025.101370"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"110022","DOI":"10.1016\/j.compbiomed.2025.110022","article-title":"DALL-M: Context-aware clinical data augmentation with large language models","volume":"190","author":"Hsieh","year":"2025","journal-title":"Comput. Biol. Med."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Nazi, Z.A., and Peng, W. (2024). Large language models in healthcare and medical domain: A review. Informatics, 11.","DOI":"10.3390\/informatics11030057"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"e2440969","DOI":"10.1001\/jamanetworkopen.2024.40969","article-title":"Large language model influence on diagnostic reasoning: A randomized clinical trial","volume":"7","author":"Goh","year":"2024","journal-title":"JAMA Netw. Open"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"101945","DOI":"10.1016\/j.nuclcard.2024.101945","article-title":"Comparative Evaluation of Large Language Models as Clinical Decision Support Tool for Cardiac Amyloidosis","volume":"38","author":"Azevedo","year":"2024","journal-title":"J. Nucl. Cardiol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"74","DOI":"10.14744\/AnatolJCardiol.2023.3685","article-title":"Artificial intelligence-based clinical decision support systems in cardiovascular diseases","volume":"28","author":"Bozyel","year":"2024","journal-title":"Anatol. J. Cardiol."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.ajem.2024.07.043","article-title":"The accuracy of Gemini, GPT-4, and GPT-4o in ECG analysis: A comparison with cardiologists and emergency medicine specialists","volume":"84","year":"2024","journal-title":"Am. J. Emerg. Med."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"e54607","DOI":"10.2196\/54607","article-title":"Multimodal ChatGPT-4V for Electrocardiogram Interpretation: Promise and Limitations","volume":"26","author":"Zhu","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"e033584","DOI":"10.1161\/JAHA.123.033584","article-title":"Potential Multidisciplinary Use of Large Language Models for Addressing Queries in Cardio-Oncology","volume":"13","author":"Li","year":"2024","journal-title":"J. Am. Heart Assoc."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"189","DOI":"10.4258\/hir.2025.31.2.189","article-title":"Generative Pre-trained Transformer: Trends, Applications, Strengths and Challenges in Dentistry: A Systematic Review","volume":"31","author":"Siluvai","year":"2025","journal-title":"Healthc. Inform. Res."},{"key":"ref_45","unstructured":"Nori, H., Lee, Y.T., Zhang, S., Carignan, D., Edgar, R., Fusi, N., King, N., Larson, J., Li, Y., and Liu, W. (2023). Can generalist foundation models outcompete special-purpose tuning? case study in medicine. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"e49","DOI":"10.3399\/bjgp15X683161","article-title":"Early diagnostic suggestions improve accuracy of GPs: A randomised controlled trial using computer-simulated patients","volume":"65","author":"Kostopoulou","year":"2014","journal-title":"Br. J. Gen. Pract."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1136\/bmjqs-2021-013493","article-title":"Should electronic differential diagnosis support be used early or late in the diagnostic process? A multicentre experimental study of Isabel","volume":"31","author":"Sibbald","year":"2022","journal-title":"BMJ Qual. Saf."},{"key":"ref_48","unstructured":"Ataee, S., and Popescu-Belis, A. (2025). Chain-of-Thought Reasoning Improves Context-Aware Translation with Large Language Models. arXiv."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"100213","DOI":"10.1016\/j.caeai.2024.100213","article-title":"Applying large language models and chain-of-thought for automatic scoring","volume":"6","author":"Lee","year":"2024","journal-title":"Comput. Educ. Artif. Intell."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"102323","DOI":"10.1016\/j.xcrm.2025.102323","article-title":"Large language model as clinical decision support system augments medication safety in 16 clinical specialties","volume":"6","author":"Ong","year":"2025","journal-title":"Cell Rep. Med."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"388","DOI":"10.1598\/RRQ.45.4.2","article-title":"Text complexity and oral reading prosody in young readers","volume":"45","author":"Benjamin","year":"2010","journal-title":"Read. Res. Q."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"AIcs2400639","DOI":"10.1056\/AIcs2400639","article-title":"Cognitive biases and artificial intelligence","volume":"1","author":"Wang","year":"2024","journal-title":"NEJM AI"}],"container-title":["Medical Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-3271\/14\/1\/33\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,10]],"date-time":"2026-01-10T05:19:53Z","timestamp":1768022393000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-3271\/14\/1\/33"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,8]]},"references-count":52,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["medsci14010033"],"URL":"https:\/\/doi.org\/10.3390\/medsci14010033","relation":{},"ISSN":["2076-3271"],"issn-type":[{"value":"2076-3271","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,8]]}}}