{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,4]],"date-time":"2026-11-04T00:31:19Z","timestamp":1793752279424,"version":"4.6.1"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,3,22]],"date-time":"2022-03-22T00:00:00Z","timestamp":1647907200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,22]],"date-time":"2022-03-22T00:00:00Z","timestamp":1647907200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["SFB CRC1382"],"award-info":[{"award-number":["SFB CRC1382"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["SFB-TRR57"],"award-info":[{"award-number":["SFB-TRR57"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003107","name":"Bundesministerium f\u00fcr Gesundheit","doi-asserted-by":"publisher","award":["DEEP LIVER"],"award-info":[{"award-number":["DEEP LIVER"]}],"id":[{"id":"10.13039\/501100003107","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003107","name":"Bundesministerium f\u00fcr Gesundheit","doi-asserted-by":"publisher","award":["ZMVI1-2520DAT111"],"award-info":[{"award-number":["ZMVI1-2520DAT111"]}],"id":[{"id":"10.13039\/501100003107","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005972","name":"Deutsche Krebshilfe","doi-asserted-by":"publisher","award":["70113864"],"award-info":[{"award-number":["70113864"]}],"id":[{"id":"10.13039\/501100005972","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Artificial intelligence (AI) is widely used to analyze gastrointestinal (GI) endoscopy image data. AI has led to several clinically approved algorithms for polyp detection, but application of AI beyond this specific task is limited by the high cost of manual annotations. Here, we show that a weakly supervised AI can be trained on data from a clinical routine database to learn visual patterns of GI diseases without any manual labeling or annotation. We trained a deep neural network on a dataset of N\u2009=\u200929,506 gastroscopy and N\u2009=\u200918,942 colonoscopy examinations from a large endoscopy unit serving patients in Germany, the Netherlands and Belgium, using only routine diagnosis data for the 42 most common diseases. Despite a high data heterogeneity, the AI system reached a high performance for diagnosis of multiple diseases, including inflammatory, degenerative, infectious and neoplastic diseases. Specifically, a cross-validated area under the receiver operating curve (AUROC) of above 0.70 was reached for 13 diseases, and an AUROC of above 0.80 was reached for two diseases in the primary data set. In an external validation set including six disease categories, the AI system was able to significantly predict the presence of diverticulosis, candidiasis, colon and rectal cancer with AUROCs above 0.76. Reverse engineering the predictions demonstrated that plausible patterns were learned on the level of images and within images and potential confounders were identified. In summary, our study demonstrates the potential of weakly supervised AI to generate high-performing classifiers and identify clinically relevant visual patterns based on non-annotated routine image data in GI endoscopy and potentially other clinical imaging modalities.<\/jats:p>","DOI":"10.1038\/s41598-022-08773-1","type":"journal-article","created":{"date-parts":[[2022,3,22]],"date-time":"2022-03-22T11:03:15Z","timestamp":1647946995000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Weakly supervised end-to-end artificial intelligence in gastrointestinal endoscopy"],"prefix":"10.1038","volume":"12","author":[{"given":"Lukas","family":"Buendgens","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Didem","family":"Cifci","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Narmin","family":"Ghaffari Laleh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marko","family":"van Treeck","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maria T.","family":"Koenen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Henning W.","family":"Zimmermann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Till","family":"Herbold","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas Joachim","family":"Lux","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexander","family":"Hann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"Trautwein","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jakob Nikolas","family":"Kather","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,3,22]]},"reference":[{"key":"8773_CR1","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","volume":"542","author":"A Esteva","year":"2017","unstructured":"Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115\u2013118 (2017).","journal-title":"Nature"},{"key":"8773_CR2","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1016\/j.ejca.2018.12.016","volume":"111","author":"TJ Brinker","year":"2019","unstructured":"Brinker, T. J. et al. Comparing artificial intelligence algorithms to 157 German dermatologists: The melanoma classification benchmark. Eur. J. Cancer 111, 30\u201337 (2019).","journal-title":"Eur. J. Cancer"},{"key":"8773_CR3","doi-asserted-by":"publisher","DOI":"10.1038\/s41416-020-01122-x","author":"A Echle","year":"2020","unstructured":"Echle, A. et al. Deep learning in cancer pathology: A new generation of clinical biomarkers. Br. J. Cancer https:\/\/doi.org\/10.1038\/s41416-020-01122-x (2020).","journal-title":"Br. J. Cancer"},{"key":"8773_CR4","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1038\/s41568-018-0016-5","volume":"18","author":"A Hosny","year":"2018","unstructured":"Hosny, A. et al. Artificial intelligence in radiology. Nat. Rev. Cancer 18, 500\u2013510 (2018).","journal-title":"Nat. Rev. Cancer"},{"key":"8773_CR5","doi-asserted-by":"publisher","first-page":"2035","DOI":"10.1136\/gutjnl-2019-320466","volume":"69","author":"F van der Sommen","year":"2020","unstructured":"van der Sommen, F. et al. Machine learning in GI endoscopy: Practical guidance in how to interpret a novel field. Gut 69, 2035\u20132045 (2020).","journal-title":"Gut"},{"key":"8773_CR6","doi-asserted-by":"publisher","first-page":"471","DOI":"10.1159\/000512440","volume":"36","author":"D Wilhelm","year":"2020","unstructured":"Wilhelm, D. et al. Artificial intelligence in visceral medicine. Visc. Med. 36, 471\u2013475 (2020).","journal-title":"Visc. Med."},{"key":"8773_CR7","doi-asserted-by":"publisher","first-page":"1183","DOI":"10.1136\/gutjnl-2020-322880","volume":"70","author":"J Calderaro","year":"2021","unstructured":"Calderaro, J. & Kather, J. N. Artificial intelligence-based pathology for gastrointestinal and hepatobiliary cancers. Gut 70, 1183\u20131193 (2021).","journal-title":"Gut"},{"key":"8773_CR8","doi-asserted-by":"publisher","DOI":"10.1038\/s41575-020-0343-3","author":"JN Kather","year":"2020","unstructured":"Kather, J. N. & Calderaro, J. Development of AI-based pathology biomarkers in gastrointestinal and liver cancer. Nat. Rev. Gastroenterol. Hepatol. https:\/\/doi.org\/10.1038\/s41575-020-0343-3 (2020).","journal-title":"Nat. Rev. Gastroenterol. Hepatol."},{"key":"8773_CR9","doi-asserted-by":"crossref","first-page":"263177452110178","DOI":"10.1177\/26317745211017809","volume":"14","author":"S Sundaram","year":"2021","unstructured":"Sundaram, S. et al. Artificial intelligence in inflammatory bowel disease endoscopy: Current landscape and the road ahead. Ther. Adv. Gastrointest. Endosc. 14, 26317745211017810 (2021).","journal-title":"Ther. Adv. Gastrointest. Endosc."},{"key":"8773_CR10","doi-asserted-by":"publisher","first-page":"1145","DOI":"10.1136\/gut.2005.087130","volume":"55","author":"H Brenner","year":"2006","unstructured":"Brenner, H. et al. Does a negative screening colonoscopy ever need to be repeated?. Gut 55, 1145\u20131150 (2006).","journal-title":"Gut"},{"key":"8773_CR11","doi-asserted-by":"publisher","first-page":"1002","DOI":"10.1016\/S1470-2045(21)00199-6","volume":"22","author":"R Cardoso","year":"2021","unstructured":"Cardoso, R. et al. Colorectal cancer incidence, mortality, and stage distribution in European countries in the colorectal cancer screening era: An international population-based study. Lancet Oncol. 22, 1002\u20131013 (2021).","journal-title":"Lancet Oncol."},{"key":"8773_CR12","doi-asserted-by":"publisher","first-page":"14465","DOI":"10.1038\/s41598-019-50567-5","volume":"9","author":"M Yamada","year":"2019","unstructured":"Yamada, M. et al. Development of a real-time endoscopic image diagnosis support system using deep learning technology in colonoscopy. Sci. Rep. 9, 14465 (2019).","journal-title":"Sci. Rep."},{"key":"8773_CR13","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1016\/S2468-1253(19)30411-X","volume":"5","author":"P Wang","year":"2020","unstructured":"Wang, P. et al. Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): A double-blind randomised study. Lancet Gastroenterol. Hepatol. 5, 343\u2013351 (2020).","journal-title":"Lancet Gastroenterol. Hepatol."},{"key":"8773_CR14","first-page":"100622","volume":"29\u201330","author":"BP Mohan","year":"2020","unstructured":"Mohan, B. P. et al. Real-time computer aided colonoscopy versus standard colonoscopy for improving adenoma detection rate: A meta-analysis of randomized-controlled trials. EClin. Med. 29\u201330, 100622 (2020).","journal-title":"EClin. Med."},{"key":"8773_CR15","doi-asserted-by":"publisher","first-page":"1813","DOI":"10.1136\/gutjnl-2018-317500","volume":"68","author":"P Wang","year":"2019","unstructured":"Wang, P. et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: A prospective randomised controlled study. Gut 68, 1813\u20131819 (2019).","journal-title":"Gut"},{"key":"8773_CR16","doi-asserted-by":"publisher","first-page":"527","DOI":"10.1002\/ueg2.12108","volume":"9","author":"A Hann","year":"2021","unstructured":"Hann, A., Troya, J. & Fitting, D. Current status and limitations of artificial intelligence in colonoscopy. United Eur. Gastroenterol. J. 9, 527\u2013533 (2021).","journal-title":"United Eur. Gastroenterol. J."},{"key":"8773_CR17","doi-asserted-by":"publisher","first-page":"428","DOI":"10.1097\/MOG.0000000000000774","volume":"37","author":"M Moore","year":"2021","unstructured":"Moore, M. & Sharma, P. Updates in artificial intelligence in gastroenterology endoscopy in 2020. Curr. Opin. Gastroenterol. 37, 428\u2013433 (2021).","journal-title":"Curr. Opin. Gastroenterol."},{"key":"8773_CR18","doi-asserted-by":"publisher","first-page":"e21983","DOI":"10.2196\/21983","volume":"22","author":"CS Bang","year":"2020","unstructured":"Bang, C. S., Lee, J. J. & Baik, G. H. Artificial intelligence for the prediction of helicobacter pylori infection in endoscopic images: Systematic Review and meta-analysis of diagnostic test accuracy. J. Med. Internet. Res. 22, e21983 (2020).","journal-title":"J. Med. Internet. Res."},{"key":"8773_CR19","doi-asserted-by":"publisher","first-page":"5408","DOI":"10.3748\/wjg.v26.i36.5408","volume":"26","author":"P-H Niu","year":"2020","unstructured":"Niu, P.-H. et al. Artificial intelligence in gastric cancer: Application and future perspectives. World J. Gastroenterol. 26, 5408\u20135419 (2020).","journal-title":"World J. Gastroenterol."},{"key":"8773_CR20","doi-asserted-by":"publisher","first-page":"538","DOI":"10.1177\/2050640619837443","volume":"7","author":"J de Groof","year":"2019","unstructured":"de Groof, J. et al. The Argos project: The development of a computer-aided detection system to improve detection of Barrett\u2019s neoplasia on white light endoscopy. United Eur. Gastroenterol. J. 7, 538\u2013547 (2019).","journal-title":"United Eur. Gastroenterol. J."},{"key":"8773_CR21","doi-asserted-by":"publisher","first-page":"915","DOI":"10.1053\/j.gastro.2019.11.030","volume":"158","author":"AJ de Groof","year":"2020","unstructured":"de Groof, A. J. et al. Deep-learning system detects neoplasia in patients with barrett\u2019s esophagus with higher accuracy than endoscopists in a multistep training and validation study with benchmarking. Gastroenterology 158, 915\u201329.e4 (2020).","journal-title":"Gastroenterology"},{"key":"8773_CR22","doi-asserted-by":"publisher","first-page":"132","DOI":"10.5946\/ce.2020.038","volume":"53","author":"AP Abadir","year":"2020","unstructured":"Abadir, A. P. et al. Artificial intelligence in gastrointestinal endoscopy. Clin. Endosc. 53, 132\u2013141 (2020).","journal-title":"Clin. Endosc."},{"key":"8773_CR23","doi-asserted-by":"publisher","first-page":"1151","DOI":"10.1016\/j.gie.2020.09.022","volume":"92","author":"R Pannala","year":"2020","unstructured":"Pannala, R. et al. Emerging role of artificial intelligence in GI endoscopy. Gastrointest. Endosc. 92, 1151\u20131152 (2020).","journal-title":"Gastrointest. Endosc."},{"key":"8773_CR24","doi-asserted-by":"publisher","first-page":"388","DOI":"10.5009\/gnl18384","volume":"13","author":"JK Min","year":"2019","unstructured":"Min, J. K., Kwak, M. S. & Cha, J. M. Overview of deep learning in gastrointestinal endoscopy. Gut Liver 13, 388\u2013393 (2019).","journal-title":"Gut Liver"},{"key":"8773_CR25","doi-asserted-by":"publisher","first-page":"789","DOI":"10.1038\/s43018-020-0087-6","volume":"1","author":"JN Kather","year":"2020","unstructured":"Kather, J. N. et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat. Cancer 1, 789\u2013799 (2020).","journal-title":"Nat. Cancer"},{"key":"8773_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s43018-020-0085-8","volume":"1","author":"Y Fu","year":"2020","unstructured":"Fu, Y. et al. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat. Cancer 1, 1\u201311 (2020).","journal-title":"Nat. Cancer"},{"key":"8773_CR27","doi-asserted-by":"publisher","DOI":"10.1101\/2020.12.03.410845","author":"FM Howard","year":"2020","unstructured":"Howard, F. M. et al. The impact of digital histopathology batch effect on deep learning model accuracy and bias. BioRxiv https:\/\/doi.org\/10.1101\/2020.12.03.410845 (2020).","journal-title":"BioRxiv"},{"key":"8773_CR28","doi-asserted-by":"publisher","DOI":"10.1101\/2021.12.19.473344","author":"M van Treeck","year":"2021","unstructured":"van Treeck, M. et al. DeepMed: A unified, modular pipeline for end-to-end deep learning in computational pathology. BioRxiv https:\/\/doi.org\/10.1101\/2021.12.19.473344 (2021).","journal-title":"BioRxiv"},{"key":"8773_CR29","doi-asserted-by":"publisher","first-page":"2701","DOI":"10.1109\/JBHI.2020.2974425","volume":"24","author":"D Hao","year":"2020","unstructured":"Hao, D. et al. Inaccurate labels in weakly-supervised deep learning: Automatic identification and correction and their impact on classification performance. IEEE J. Biomed. Health Inf. 24, 2701\u20132710 (2020).","journal-title":"IEEE J. Biomed. Health Inf."},{"key":"8773_CR30","doi-asserted-by":"publisher","DOI":"10.1136\/gutjnl-2020-323115","author":"R Schmitz","year":"2020","unstructured":"Schmitz, R. et al. Artificial intelligence in GI endoscopy: Stumbling blocks, gold standards and the role of endoscopy societies. Gut https:\/\/doi.org\/10.1136\/gutjnl-2020-323115 (2020).","journal-title":"Gut"},{"key":"8773_CR31","doi-asserted-by":"publisher","DOI":"10.1038\/s41568-021-00408-3","author":"KM Boehm","year":"2021","unstructured":"Boehm, K. M. et al. Harnessing multimodal data integration to advance precision oncology. Nat. Rev. Cancer https:\/\/doi.org\/10.1038\/s41568-021-00408-3 (2021).","journal-title":"Nat. Rev. Cancer"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-08773-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-08773-1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-08773-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,25]],"date-time":"2022-11-25T00:39:56Z","timestamp":1669336796000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-08773-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,22]]},"references-count":31,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["8773"],"URL":"https:\/\/doi.org\/10.1038\/s41598-022-08773-1","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,22]]},"assertion":[{"value":"26 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 March 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 March 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"JNK declares consulting services for Owkin, France and Panakeia, UK. No other potential conflicts of interest are reported by any of the authors.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"4829"}}