{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T02:56:06Z","timestamp":1777690566863,"version":"3.51.4"},"reference-count":33,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,1,17]],"date-time":"2025-01-17T00:00:00Z","timestamp":1737072000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCM"],"abstract":"<jats:p>Background: Several artificial intelligence systems based on large language models (LLMs) have been commercially developed, with recent interest in integrating them for clinical questions. Recent versions now include image analysis capacity, but their performance in gastroenterology remains untested. This study assesses ChatGPT-4\u2019s performance in interpreting gastroenterology images. Methods: A total of 740 images from five procedures\u2014capsule endoscopy (CE), device-assisted enteroscopy (DAE), endoscopic ultrasound (EUS), digital single-operator cholangioscopy (DSOC), and high-resolution anoscopy (HRA)\u2014were included and analyzed by ChatGPT-4 using a predefined prompt for each. ChatGPT-4 predictions were compared to gold standard diagnoses. Statistical analyses included accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the curve (AUC). Results: For CE, ChatGPT-4 demonstrated accuracies ranging from 50.0% to 90.0%, with AUCs of 0.50\u20130.90. For DAE, the model demonstrated an accuracy of 67.0% (AUC 0.670). For EUS, the system showed AUCs of 0.488 and 0.550 for the differentiation between pancreatic cystic and solid lesions, respectively. The LLM differentiated benign from malignant biliary strictures with an AUC of 0.550. For HRA, ChatGPT-4 showed an overall accuracy between 47.5% and 67.5%. Conclusions: ChatGPT-4 demonstrated suboptimal diagnostic accuracies for image interpretation across several gastroenterology techniques, highlighting the need for continuous improvement before clinical adoption.<\/jats:p>","DOI":"10.3390\/jcm14020572","type":"journal-article","created":{"date-parts":[[2025,1,17]],"date-time":"2025-01-17T11:24:56Z","timestamp":1737113096000},"page":"572","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Evaluating ChatGPT-4 for the Interpretation of Images from Several Diagnostic Techniques in Gastroenterology"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0340-0830","authenticated-orcid":false,"given":"Miguel Mascarenhas","family":"Saraiva","sequence":"first","affiliation":[{"name":"Precision Medicine Unit, Department of Gastroenterology, S\u00e3o Jo\u00e3o University Hospital, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"WGO Gastroenterology and Hepatology Training Center, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"Faculty of Medicine, University of Porto, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"}]},{"given":"Tiago","family":"Ribeiro","sequence":"additional","affiliation":[{"name":"Precision Medicine Unit, Department of Gastroenterology, S\u00e3o Jo\u00e3o University Hospital, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"WGO Gastroenterology and Hepatology Training Center, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"Faculty of Medicine, University of Porto, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"}]},{"given":"Bel\u00e9n","family":"Agudo","sequence":"additional","affiliation":[{"name":"Department of Gastroenterology, Hospital Universitario Puerta de Hierro Majadahonda, C\/Joaqu\u00edn Rodrigo, 28220 Madrid, Spain"}]},{"given":"Jo\u00e3o","family":"Afonso","sequence":"additional","affiliation":[{"name":"Precision Medicine Unit, Department of Gastroenterology, S\u00e3o Jo\u00e3o University Hospital, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"WGO Gastroenterology and Hepatology Training Center, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"Faculty of Medicine, University of Porto, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5890-7049","authenticated-orcid":false,"given":"Francisco","family":"Mendes","sequence":"additional","affiliation":[{"name":"Precision Medicine Unit, Department of Gastroenterology, S\u00e3o Jo\u00e3o University Hospital, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"WGO Gastroenterology and Hepatology Training Center, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"Faculty of Medicine, University of Porto, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"}]},{"given":"Miguel","family":"Martins","sequence":"additional","affiliation":[{"name":"Precision Medicine Unit, Department of Gastroenterology, S\u00e3o Jo\u00e3o University Hospital, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"WGO Gastroenterology and Hepatology Training Center, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"Faculty of Medicine, University of Porto, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9427-5635","authenticated-orcid":false,"given":"Pedro","family":"Cardoso","sequence":"additional","affiliation":[{"name":"Precision Medicine Unit, Department of Gastroenterology, S\u00e3o Jo\u00e3o University Hospital, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"WGO Gastroenterology and Hepatology Training Center, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"Faculty of Medicine, University of Porto, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"}]},{"given":"Joana","family":"Mota","sequence":"additional","affiliation":[{"name":"Precision Medicine Unit, Department of Gastroenterology, S\u00e3o Jo\u00e3o University Hospital, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"WGO Gastroenterology and Hepatology Training Center, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"Faculty of Medicine, University of Porto, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"}]},{"given":"Maria Joao","family":"Almeida","sequence":"additional","affiliation":[{"name":"Precision Medicine Unit, Department of Gastroenterology, S\u00e3o Jo\u00e3o University Hospital, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"WGO Gastroenterology and Hepatology Training Center, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"Faculty of Medicine, University of Porto, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"}]},{"given":"Ant\u00f3nio","family":"Costa","sequence":"additional","affiliation":[{"name":"Department of Gastroenterology, Hospital Universitario Puerta de Hierro Majadahonda, C\/Joaqu\u00edn Rodrigo, 28220 Madrid, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8025-1704","authenticated-orcid":false,"given":"Mariano","family":"Gonzalez Haba Ruiz","sequence":"additional","affiliation":[{"name":"Department of Gastroenterology, Hospital Universitario Puerta de Hierro Majadahonda, C\/Joaqu\u00edn Rodrigo, 28220 Madrid, Spain"}]},{"given":"Jessica","family":"Widmer","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology, NYU Langone Hospital\u2014Long Island, 259 First Street Mineola, New York, NY 11501, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8023-3722","authenticated-orcid":false,"given":"Eduardo","family":"Moura","sequence":"additional","affiliation":[{"name":"Department of Gastrointestinal Endoscopy, Hospital das Cl\u00ednicas da Faculdade de Medicina da Universidade de S\u00e3o Paulo, Rua Dr. Ov\u00eddio Pires de Campos 225, Sao Paulo 05403-010, Brazil"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0402-2285","authenticated-orcid":false,"given":"Ahsan","family":"Javed","sequence":"additional","affiliation":[{"name":"Department of Colorectal Surgery, Royal Liverpool University Hospital, Liverpool L7 8YE, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1914-0129","authenticated-orcid":false,"given":"Thiago","family":"Manzione","sequence":"additional","affiliation":[{"name":"Department of Surgery, Instituto de Infectologia Em\u00edlio Ribas, S\u00e3o Paulo 01246-900, Brazil"}]},{"given":"Sidney","family":"Nadal","sequence":"additional","affiliation":[{"name":"Department of Surgery, Instituto de Infectologia Em\u00edlio Ribas, S\u00e3o Paulo 01246-900, Brazil"}]},{"given":"Luis F.","family":"Barroso","sequence":"additional","affiliation":[{"name":"Internal Medicine\/Infectious Diseases, Wake Forest University Health Sciences, Winston-Salem, NC 27109, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1498-9089","authenticated-orcid":false,"given":"Vincent","family":"de Parades","sequence":"additional","affiliation":[{"name":"Department of Proctology, H\u00f4pital Paris Saint-Joseph, 85, Rue Raymond Losserand, 75014 Paris, France"}]},{"given":"Jo\u00e3o","family":"Ferreira","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Faculty of Engineering of the University of Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9387-9872","authenticated-orcid":false,"given":"Guilherme","family":"Macedo","sequence":"additional","affiliation":[{"name":"Precision Medicine Unit, Department of Gastroenterology, S\u00e3o Jo\u00e3o University Hospital, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"WGO Gastroenterology and Hepatology Training Center, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"},{"name":"Faculty of Medicine, University of Porto, Alameda Professor Hern\u00e2ni Monteiro, 4200-427 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2025,1,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/j.surg.2024.04.003","article-title":"AI Versus MD: Evaluating the surgical decision-making accuracy of ChatGPT-4","volume":"176","author":"Palenzuela","year":"2024","journal-title":"Surgery"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1007\/s11604-024-01619-y","article-title":"Diagnostic performances of GPT-4o, Claude 3 Opus, and Gemini 1.5 Pro in \u201cDiagnosis Please\u201d cases","volume":"42","author":"Sonoda","year":"2024","journal-title":"Jpn. J. Radiol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Lahat, A., Shachar, E., Avidan, B., Shatz, Z., Glicksberg, B.S., and Klang, E. (2023). Evaluating the use of large language model in identifying top research questions in gastroenterology. Sci. Rep., 13.","DOI":"10.1038\/s41598-023-31412-2"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"487","DOI":"10.4103\/NRR.NRR-D-24-00165","article-title":"From text to image: Challenges in integrating vision into ChatGPT for medical image interpretation","volume":"20","author":"Koga","year":"2025","journal-title":"Neural Regen. Res."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1108","DOI":"10.21037\/qims-23-892","article-title":"The role of large language models in medical image processing: A narrative review","volume":"14","author":"Tian","year":"2024","journal-title":"Quant. Imaging Med. Surg."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e59505","DOI":"10.2196\/59505","article-title":"Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook","volume":"26","author":"AlSaad","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1053\/j.gastro.2023.04.033","article-title":"ChatGPT Answers Common Patient Questions About Colonoscopy","volume":"165","author":"Lee","year":"2023","journal-title":"Gastroenterology"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2276","DOI":"10.14309\/ajg.0000000000002397","article-title":"Evaluation of the Potential Utility of an Artificial Intelligence Chatbot in Gastroesophageal Reflux Disease Management","volume":"118","author":"Henson","year":"2023","journal-title":"Am. J. Gastroenterol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"831","DOI":"10.1016\/j.gie.2020.04.039","article-title":"Deep learning for wireless capsule endoscopy: A systematic review and meta-analysis","volume":"92","author":"Soffer","year":"2020","journal-title":"Gastrointest. Endosc."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1038\/s41746-023-00813-y","article-title":"Explainable artificial intelligence incorporated with domain knowledge diagnosing early gastric neoplasms under white light endoscopy","volume":"6","author":"Dong","year":"2023","journal-title":"NPJ Digit. Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1709","DOI":"10.14309\/ajg.0000000000002689","article-title":"Prompt Engineering for Generative Artificial Intelligence in Gastroenterology and Hepatology","volume":"119","author":"Ge","year":"2024","journal-title":"Am. J. Gastroenterol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2146","DOI":"10.14309\/ajg.0000000000002573","article-title":"Generative Artificial Intelligence for Gastroenterology: Neither Friend nor Foe","volume":"118","author":"Dang","year":"2023","journal-title":"Am. J. Gastroenterol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1053\/j.gastro.2019.08.058","article-title":"Application of Artificial Intelligence to Gastroenterology and Hepatology","volume":"158","author":"Sandborn","year":"2020","journal-title":"Gastroenterology"},{"key":"ref_14","first-page":"e68298","article-title":"GPT-4 Vision: Multi-Modal Evolution of ChatGPT and Potential Role in Radiology","volume":"16","author":"Javan","year":"2024","journal-title":"Cureus"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e55627","DOI":"10.2196\/55627","article-title":"Evaluating ChatGPT-4\u2019s Diagnostic Accuracy: Impact of Visual Data Integration","volume":"12","author":"Hirosawa","year":"2024","journal-title":"JMIR Med. Inform."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"e48808","DOI":"10.2196\/48808","article-title":"ChatGPT-Generated Differential Diagnosis Lists for Complex Case-Derived Clinical Vignettes: Diagnostic Accuracy Evaluation","volume":"11","author":"Hirosawa","year":"2023","journal-title":"JMIR Med. Inform."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"E600","DOI":"10.1055\/a-2289-9334","article-title":"Using a customized GPT to provide guideline-based recommendations for management of pancreatic cystic lesions","volume":"12","author":"Gorelik","year":"2024","journal-title":"Endosc. Int. Open"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1168","DOI":"10.1007\/s11604-024-01606-3","article-title":"Evaluating ChatGPT-4V in chest CT diagnostics: A critical image interpretation assessment","volume":"42","author":"Dehdab","year":"2024","journal-title":"Jpn. J. Radiol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1057","DOI":"10.1016\/j.jaad.2023.12.062","article-title":"Can ChatGPT vision diagnose melanoma? An exploratory diagnostic accuracy study","volume":"90","author":"Shifai","year":"2024","journal-title":"J. Am. Acad. Dermatol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e000753","DOI":"10.1136\/bmjgast-2021-000753","article-title":"Deep learning and capsule endoscopy: Automatic identification and differentiation of small bowel lesions with distinct haemorrhagic potential using a convolutional neural network","volume":"8","author":"Afonso","year":"2021","journal-title":"BMJ Open Gastroenterol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1044","DOI":"10.1053\/j.gastro.2019.06.025","article-title":"Gastroenterologist-Level Identification of Small-Bowel Diseases and Normal Variants by Capsule Endoscopy Using a Deep-Learning Model","volume":"157","author":"Ding","year":"2019","journal-title":"Gastroenterology"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Vilas-Boas, F., Ribeiro, T., Afonso, J., Cardoso, H., Lopes, S., Moutinho-Ribeiro, P., Ferreira, J., Mascarenhas-Saraiva, M., and Macedo, G. (2022). Deep Learning for Automatic Differentiation of Mucinous versus Non-Mucinous Pancreatic Cystic Lesions: A Pilot Study. Diagnostics, 12.","DOI":"10.3390\/diagnostics12092041"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"719","DOI":"10.1055\/a-2034-3803","article-title":"Artificial intelligence for diagnosing neoplasia on digital cholangioscopy: Development and multicenter validation of a convolutional neural network model","volume":"55","author":"Kahaleh","year":"2023","journal-title":"Endoscopy"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1016\/j.gie.2022.08.021","article-title":"Identification of patients with malignant biliary strictures using a cholangioscopy-based deep learning artificial intelligence (with video)","volume":"97","author":"Marya","year":"2023","journal-title":"Gastrointest. Endosc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/j.gie.2023.02.026","article-title":"A real-time interpretable artificial intelligence model for the cholangioscopic diagnosis of malignant biliary stricture (with videos)","volume":"98","author":"Zhang","year":"2023","journal-title":"Gastrointest. Endosc."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1016\/j.gie.2021.08.027","article-title":"Artificial intelligence for automatic diagnosis of biliary stricture malignancy status in single-operator cholangioscopy: A pilot study","volume":"95","author":"Saraiva","year":"2022","journal-title":"Gastrointest. Endosc."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"893","DOI":"10.1007\/s10151-022-02684-z","article-title":"Artificial intelligence and high-resolution anoscopy: Automatic identification of anal squamous cell carcinoma precursors using a convolutional neural network","volume":"26","author":"Saraiva","year":"2022","journal-title":"Tech. Coloproctol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e00681","DOI":"10.14309\/ctg.0000000000000681","article-title":"Deep Learning in High-Resolution Anoscopy: Assessing the Impact of Staining and Therapeutic Manipulation on Automated Detection of Anal Cancer Precursors","volume":"15","author":"Spindler","year":"2024","journal-title":"Clin. Transl. Gastroenterol."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Saraiva, M.M., Spindler, L., Manzione, T., Ribeiro, T., Fathallah, N., Martins, M., Cardoso, P., Mendes, F., Fernandes, J., and Ferreira, J. (2024). Deep Learning and High-Resolution Anoscopy: Development of an Interoperable Algorithm for the Detection and Differentiation of Anal Squamous Cell Carcinoma Precursors-A Multicentric Study. Cancers, 16.","DOI":"10.3390\/cancers16101909"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"e50853","DOI":"10.2196\/50853","article-title":"The Effect of Artificial Intelligence on Patient-Physician Trust: Cross-Sectional Vignette Study","volume":"26","author":"Zondag","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"e47031","DOI":"10.2196\/47031","article-title":"Trust in and Acceptance of Artificial Intelligence Applications in Medicine: Mixed Methods Study","volume":"11","author":"Shevtsova","year":"2024","journal-title":"JMIR Hum. Factors"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"e50638","DOI":"10.2196\/50638","article-title":"Prompt Engineering as an Important Emerging Skill for Medical Professionals: Tutorial","volume":"25","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"e2440969","DOI":"10.1001\/jamanetworkopen.2024.40969","article-title":"Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial","volume":"7","author":"Goh","year":"2024","journal-title":"JAMA Netw. Open"}],"container-title":["Journal of Clinical Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/2\/572\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T10:30:33Z","timestamp":1759919433000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/2\/572"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,17]]},"references-count":33,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,1]]}},"alternative-id":["jcm14020572"],"URL":"https:\/\/doi.org\/10.3390\/jcm14020572","relation":{},"ISSN":["2077-0383"],"issn-type":[{"value":"2077-0383","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,17]]}}}