{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T03:04:42Z","timestamp":1788231882109,"version":"build-2803163510"},"posted":{"date-parts":[[2024,2,18]]},"group-title":"Cardiovascular Medicine","reference-count":44,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2024,2,18]],"date-time":"2024-02-18T00:00:00Z","timestamp":1708214400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2025,4,22]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                <jats:sec>\n                  <jats:title>Background<\/jats:title>\n                  <jats:p>Timely and accurate assessment of electrocardiograms (ECGs) is crucial for diagnosing, triaging, and clinically managing patients. Current workflows rely on computerized ECG interpretation tools built into ECG signal acquisition systems, which use rule-based algorithms that are unreliable and frequently not available in low-resource settings. We developed and validated a format-independent vision encoder-decoder model \u2013 ECG-GPT \u2013 that can generate free-text, expert-level interpretations directly from 12-lead ECG images.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>Using 12-lead ECGs and their corresponding diagnosis statements collected at the Yale-New Haven Health System (YNHHS) between 2000 and 2022, we developed a vision-text transformer model to generate interpretation statements from images of ECGs. Using structured clinical assessment, semantic similarity, and conventional natural language generation metrics, we validated ECG-GPT across 7 geographically distinct health settings. These include (1) 3 large and diverse US health systems, (2) consecutive ECGs from a central reading system in Minas Gerais, Brazil, (3) the prospective cohort study, UK Biobank, (4) a Germany-based, publicly available repository, PTB-XL, and (5) a community hospital in Missouri.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Overall, 2.9 million ECGs were used for model development. The model performed well in clinical assessment across 26 extracted labels: for atrial fibrillation, sinus tachycardia, sinus bradycardia, premature atrial contractions, and premature ventricular contractions, AUROCs and AUPRCs ranged from 0.80-0.95 and 0.50-0.86, respectively. For left bundle branch block, right bundle branch block, first degree atrioventricular block, left anterior fascicular block, and left posterior fascicular block, AUROCs and AUPRCs ranged from 0.88-0.96 and 0.23-0.86, respectively. Across all 26 conditions, diagnostic accuracy ranged between 0.93-0.99. ECG-GPT identified the full context of the diagnosis statements with allied conditions. It had a median pairwise cosine similarity of 0.90 (IQR 0.83-0.97), significantly greater than the median baseline similarity of 0.73 (IQR 0.67-0.78, p&lt;0.001). This separation between median pairwise and baseline similarity remained consistent across all 26 condition-specific subsets. The results were comparable across external validation sites.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>We developed and extensively validated a vision encoder-decoder model that generates expert-level interpretations from ECG images. This represents a scalable and accessible strategy for automated ECG analysis, especially in low-resource settings.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>CLINICAL PERSPECTIVE<\/jats:title>\n                  <jats:sec>\n                    <jats:title>What is New?<\/jats:title>\n                    <jats:list list-type=\"bullet\">\n                      <jats:list-item>\n                        <jats:p>ECG-GPT is a vision encoder-decoder model capable of generating full-text ECG interpretations directly from ECG images, regardless of layout or format.<\/jats:p>\n                      <\/jats:list-item>\n                      <jats:list-item>\n                        <jats:p>The model was trained on over 2.7 million ECGs and externally validated across 3.8 million additional ECGs from demographically and geographically diverse populations.<\/jats:p>\n                      <\/jats:list-item>\n                    <\/jats:list>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>What are the clinical implications?<\/jats:title>\n                    <jats:list list-type=\"bullet\">\n                      <jats:list-item>\n                        <jats:p>ECG-GPT enables automated, expert-level ECG interpretation directly from images, eliminating the need for signal data or device integration.<\/jats:p>\n                      <\/jats:list-item>\n                      <jats:list-item>\n                        <jats:p>The model demonstrates consistent performance across diverse patient populations, ECG formats, and care settings.<\/jats:p>\n                      <\/jats:list-item>\n                      <jats:list-item>\n                        <jats:p>This scalable, image-based approach may expand access to accurate ECG interpretation in low-resource settings.<\/jats:p>\n                      <\/jats:list-item>\n                    <\/jats:list>\n                  <\/jats:sec>\n                <\/jats:sec>","DOI":"10.1101\/2024.02.17.24302976","type":"posted-content","created":{"date-parts":[[2024,2,18]],"date-time":"2024-02-18T16:05:20Z","timestamp":1708272320000},"source":"Crossref","is-referenced-by-count":12,"title":["Artificial Intelligence-Based Automated Interpretation of Images of Electrocardiograms: Development and Multinational Validation of ECG-GPT"],"prefix":"10.64898","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3812-3260","authenticated-orcid":false,"given":"Akshay","family":"Khunte","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8524-1203","authenticated-orcid":false,"given":"Veer","family":"Sangha","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4362-0720","authenticated-orcid":false,"given":"Evangelos K","family":"Oikonomou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5664-4126","authenticated-orcid":false,"given":"Lovedeep S","family":"Dhingra","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3197-2657","authenticated-orcid":false,"given":"Arya","family":"Aminorroaya","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5243-552X","authenticated-orcid":false,"given":"Andreas","family":"Coppi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-3034-0641","authenticated-orcid":false,"given":"Sumukh Vasisht","family":"Shankar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1488-7661","authenticated-orcid":false,"given":"Elijah","family":"Rockers","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2655-2095","authenticated-orcid":false,"given":"Bobak J","family":"Mortazavi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1278-6245","authenticated-orcid":false,"given":"Deepak L","family":"Bhatt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2046-127X","authenticated-orcid":false,"given":"Harlan M","family":"Krumholz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sadeer","family":"Al-Kindi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6319-4314","authenticated-orcid":false,"given":"Girish N","family":"Nadkarni","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3343-744X","authenticated-orcid":false,"given":"Akhil","family":"Vaid","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9467-6199","authenticated-orcid":false,"given":"Rohan","family":"Khera","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","reference":[{"key":"2025042309451251000_2024.02.17.24302976v2.1","doi-asserted-by":"crossref","first-page":"1221","DOI":"10.1161\/01.CIR.85.3.1221","article-title":"Guidelines for electrocardiography. A report of the American college of cardiology\/American heart association task force on assessment of diagnostic and therapeutic cardiovascular procedures (committee on electrocardiography)","volume":"85","year":"1992","journal-title":"Circulation"},{"key":"2025042309451251000_2024.02.17.24302976v2.2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jelectrocard.2007.03.008"},{"key":"2025042309451251000_2024.02.17.24302976v2.3","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijcard.2005.02.007"},{"key":"2025042309451251000_2024.02.17.24302976v2.4","doi-asserted-by":"publisher","DOI":"10.1016\/j.jacc.2017.07.723"},{"key":"2025042309451251000_2024.02.17.24302976v2.5","doi-asserted-by":"crossref","first-page":"728","DOI":"10.1016\/j.jelectrocard.2016.04.010","article-title":"Electrocardiographic diagnosis of ST segment elevation myocardial infarction: An evaluation of three automated interpretation algorithms","volume":"49","year":"2016","journal-title":"J. Electrocardiol"},{"key":"2025042309451251000_2024.02.17.24302976v2.6","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCEP.111.000097"},{"key":"2025042309451251000_2024.02.17.24302976v2.7","doi-asserted-by":"publisher","DOI":"10.1016\/j.amjmed.2004.06.024"},{"key":"2025042309451251000_2024.02.17.24302976v2.8","unstructured":"Jones, C. , Park, T. & United Economic Research Service (Ers). Health Status and Health Care Access of Farm and Rural Populations. (Bibliogov, 2012)."},{"key":"2025042309451251000_2024.02.17.24302976v2.9","doi-asserted-by":"publisher","DOI":"10.1377\/hlthaff.2011.0255"},{"key":"2025042309451251000_2024.02.17.24302976v2.10","doi-asserted-by":"publisher","DOI":"10.1089\/tmj.2015.0234"},{"key":"2025042309451251000_2024.02.17.24302976v2.11","doi-asserted-by":"crossref","first-page":"1461","DOI":"10.1001\/jamainternmed.2020.3989","article-title":"Accuracy of physicians\u2019 electrocardiogram interpretations","volume":"180","year":"2020","journal-title":"JAMA Intern. Med"},{"key":"2025042309451251000_2024.02.17.24302976v2.12","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-15432-4"},{"key":"2025042309451251000_2024.02.17.24302976v2.13","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0268-3"},{"key":"2025042309451251000_2024.02.17.24302976v2.14","doi-asserted-by":"crossref","first-page":"1285","DOI":"10.1001\/jamacardio.2021.2746","article-title":"Performance of a convolutional neural network and explainability technique for 12-lead electrocardiogram interpretation","volume":"6","year":"2021","journal-title":"JAMA Cardiol"},{"key":"2025042309451251000_2024.02.17.24302976v2.15","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-29153-3"},{"key":"2025042309451251000_2024.02.17.24302976v2.16","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCULATIONAHA.122.062646"},{"key":"2025042309451251000_2024.02.17.24302976v2.17","unstructured":"Ecg-plot. PyPI https:\/\/pypi.org\/project\/ecg-plot\/ (2021)."},{"key":"2025042309451251000_2024.02.17.24302976v2.18","unstructured":"Pyenchant. PyPI https:\/\/pypi.org\/project\/pyenchant\/ (2021)."},{"key":"2025042309451251000_2024.02.17.24302976v2.19","unstructured":"Wolf, T. et al. Transformers: State-of-the-Art Natural Language Processing. in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations 38\u201345 (Association for Computational Linguistics, Online, 2020)."},{"key":"2025042309451251000_2024.02.17.24302976v2.20","doi-asserted-by":"crossref","unstructured":"Deng, J. et al. ImageNet: A large-scale hierarchical image database. in 2009 IEEE Conference on Computer Vision and Pattern Recognition 248\u2013255 (2009).","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"2025042309451251000_2024.02.17.24302976v2.21","unstructured":"Bao, H. , Dong, L. , Piao, S. & Wei, F . BEiT: BERT Pre-Training of Image Transformers. arXiv [cs.CV ] (2021)."},{"key":"2025042309451251000_2024.02.17.24302976v2.22","unstructured":"Loshchilov, I. & Hutter, F . Decoupled Weight Decay Regularization. arXiv [cs.LG ] (2017)."},{"key":"2025042309451251000_2024.02.17.24302976v2.23","doi-asserted-by":"crossref","unstructured":"Ribeiro, A. L. P. et al. Tele-electrocardiography and bigdata: The CODE (Clinical Outcomes in Digital Electrocardiography) study. J. Electrocardiol. 57S , S75\u2013S78 (2019).","DOI":"10.1016\/j.jelectrocard.2019.09.008"},{"key":"2025042309451251000_2024.02.17.24302976v2.24","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCULATIONAHA.106.180200"},{"key":"2025042309451251000_2024.02.17.24302976v2.25","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-020-0495-6"},{"key":"2025042309451251000_2024.02.17.24302976v2.26","unstructured":"Devlin, J. , Chang, M.-W. , Lee, K. & Toutanova, K . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv [cs.CL ] (2018)."},{"key":"2025042309451251000_2024.02.17.24302976v2.27","unstructured":"Sanh, V. , Debut, L. , Chaumond, J. & Wolf, T . DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv [cs.CL ] (2019)."},{"key":"2025042309451251000_2024.02.17.24302976v2.28","doi-asserted-by":"crossref","first-page":"2633","DOI":"10.1038\/s41591-023-02552-9","article-title":"Optimized glycemic control of type 2 diabetes with reinforcement learning: a proof-of-concept trial","volume":"29","year":"2023","journal-title":"Nat. Med"},{"key":"2025042309451251000_2024.02.17.24302976v2.29","doi-asserted-by":"crossref","unstructured":"Alsentzer, E. , et al. Publicly Available Clinical BERT Embeddings. arXiv [cs.CL] (2019).","DOI":"10.18653\/v1\/W19-1909"},{"key":"2025042309451251000_2024.02.17.24302976v2.30","unstructured":"Lin, C.-Y . ROUGE: A Package for Automatic Evaluation of Summaries. in Text Summarization Branches Out 74\u201381 (Association for Computational Linguistics, Barcelona, Spain, 2004)."},{"key":"2025042309451251000_2024.02.17.24302976v2.31","unstructured":"Papineni, K. , Roukos, S. , Ward, T. & Zhu, W.-J . BLEU: A method for automatic evaluation of machine translation. https:\/\/aclanthology.org\/P02-1040.pdf."},{"key":"2025042309451251000_2024.02.17.24302976v2.32","unstructured":"Banerjee, S. & Lavie, A . METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. in Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization 65\u201372 (Association for Computational Linguistics, Ann Arbor, Michigan, 2005)."},{"key":"2025042309451251000_2024.02.17.24302976v2.33","doi-asserted-by":"crossref","unstructured":"Vedantam, R. , Zitnick, C. L. & Parikh, D . CIDEr: Consensus-based image description evaluation. arXiv [cs.CV ] 4566\u20134575 (2014).","DOI":"10.1109\/CVPR.2015.7299087"},{"key":"2025042309451251000_2024.02.17.24302976v2.34","doi-asserted-by":"crossref","unstructured":"von Werra, L. , et al. Evaluate & evaluation on the Hub: Better best practices for data and model measurements. arXiv [cs.LG] (2022).","DOI":"10.18653\/v1\/2022.emnlp-demos.13"},{"key":"2025042309451251000_2024.02.17.24302976v2.35","unstructured":"Chen, X. et al. Microsoft COCO captions: Data collection and evaluation server. arXiv [cs.CV ] (2015)."},{"key":"2025042309451251000_2024.02.17.24302976v2.36","doi-asserted-by":"publisher","DOI":"10.1109\/lsp.2014.2337313"},{"key":"2025042309451251000_2024.02.17.24302976v2.37","doi-asserted-by":"publisher","DOI":"10.2307\/2531595"},{"key":"2025042309451251000_2024.02.17.24302976v2.38","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176344552"},{"key":"2025042309451251000_2024.02.17.24302976v2.39","doi-asserted-by":"crossref","first-page":"4171","DOI":"10.1038\/s41598-023-31223-5","article-title":"Medical image captioning via generative pretrained transformers","volume":"13","year":"2023","journal-title":"Sci. Rep"},{"key":"2025042309451251000_2024.02.17.24302976v2.40","doi-asserted-by":"crossref","unstructured":"Ayesha, H. et al. Automatic medical image interpretation: State of the art and future directions. Pattern Recognit. 114, 107856 (2021).","DOI":"10.1016\/j.patcog.2021.107856"},{"key":"2025042309451251000_2024.02.17.24302976v2.41","unstructured":"Bartels, M. G. G. et al. Learning to Automatically Generate Accurate ECG Captions. in Proceedings of The 5th International Conference on Medical Imaging with Deep Learning (eds. Konukoglu, E. et al.) vol. 172 86\u2013102 (PMLR, 06--08 Jul 2022)."},{"key":"2025042309451251000_2024.02.17.24302976v2.42","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.cvdhj.2020.08.005","article-title":"A comprehensive artificial intelligence-enabled electrocardiogram interpretation program","volume":"1","year":"2020","journal-title":"Cardiovasc Digit Health J"},{"key":"2025042309451251000_2024.02.17.24302976v2.43","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1038\/s41569-020-00503-2","article-title":"Artificial intelligence-enhanced electrocardiography in cardiovascular disease management","volume":"18","year":"2021","journal-title":"Nat. Rev. Cardiol"},{"key":"2025042309451251000_2024.02.17.24302976v2.44","unstructured":"ECG-GPT. CarDS Lab cards-lab.org\/ecg-gpt (2024)."}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1101\/2024.02.17.24302976","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T19:52:14Z","timestamp":1768506734000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.1101\/2024.02.17.24302976"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,18]]},"references-count":44,"URL":"https:\/\/doi.org\/10.1101\/2024.02.17.24302976","relation":{},"subject":[],"published":{"date-parts":[[2024,2,18]]},"subtype":"preprint"}}