{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,3]],"date-time":"2026-11-03T14:25:30Z","timestamp":1793715930969,"version":"4.6.1"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,4,16]],"date-time":"2021-04-16T00:00:00Z","timestamp":1618531200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,16]],"date-time":"2021-04-16T00:00:00Z","timestamp":1618531200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The advancement of artificial intelligence (AI) has facilitated its application in medical fields. However, there has been little research for AI-assisted endoscopy, despite the clinical significance of the efficiency and safety of cannulation in the endoscopic retrograde cholangiopancreatography (ERCP). In this study, we aim to assist endoscopists performing ERCP through automatic detection of the ampulla and the identification of cannulation difficulty. We developed a novel AI-assisted system based on convolutional neural networks that predict the location of the ampulla and the difficulty of cannulation to the ampulla. ERCP data of 531 and 451 patients were utilized in the evaluation of our model for each task. Our model detected the ampulla with mean intersection-over-union 64.1%, precision 76.2%, recall 78.4%, and centroid distance 0.021. In classifying the cannulation difficulty, it achieved the recall of 71.9% for the class of easy cases and that of 61.1% for that of difficult cases. Remarkably, our model accurately detected AOV with varying morphological shape, size, and texture on par with the level of a human expert and showed promising results for recognizing cannulation difficulty. It demonstrated its potential to improve the quality of ERCP by assisting endoscopists.<\/jats:p>","DOI":"10.1038\/s41598-021-87737-3","type":"journal-article","created":{"date-parts":[[2021,4,16]],"date-time":"2021-04-16T11:36:09Z","timestamp":1618572969000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Artificial intelligence-assisted analysis of endoscopic retrograde cholangiopancreatography image for identifying ampulla and difficulty of selective cannulation"],"prefix":"10.1038","volume":"11","author":[{"given":"Taesung","family":"Kim","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinhee","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hyuk Soon","family":"Choi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eun Sun","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bora","family":"Keum","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yoon Tae","family":"Jeen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hong Sik","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hoon Jai","family":"Chun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sung Yong","family":"Han","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dong Uk","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Soonwook","family":"Kwon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jaegul","family":"Choo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jae Min","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,4,16]]},"reference":[{"key":"87737_CR1","unstructured":"Baron, T. H., Kozarek, R. A. & Carr-Locke, D. L. ERCP. Third edition. (ed. Philadelphia) 108\u2013121 (Elsevier, 2019)."},{"issue":"2","key":"87737_CR2","doi-asserted-by":"publisher","first-page":"250","DOI":"10.1016\/S0016-5107(00)70164-7","volume":"51","author":"JD Wave","year":"2000","unstructured":"Wave, J. D. Basic techniques of ERCP. Gastrointest. Endosc. 51(2), 250\u2013253 (2000).","journal-title":"Gastrointest. Endosc."},{"key":"87737_CR3","doi-asserted-by":"publisher","first-page":"657","DOI":"10.1055\/s-0042-108641","volume":"48","author":"PA Testoni","year":"2016","unstructured":"Testoni, P. A. et al. Papillary cannulation and sphincterotomy techniques at ERCP: European Society of Gastrointestinal Endoscopy (ESGE) Clinical Guideline. Endoscopy 48, 657\u2013683 (2016).","journal-title":"Endoscopy"},{"issue":"1","key":"87737_CR4","doi-asserted-by":"publisher","first-page":"5","DOI":"10.4253\/wjge.v11.i1.5","volume":"11","author":"R Berry","year":"2019","unstructured":"Berry, R., Han, J. Y. & Tabibian, J. H. Difficult biliary cannulation: Historical perspective, practical updates, and guide for the endoscopist. World J. Gastrointest. Endosc. 11(1), 5 (2019).","journal-title":"World J. Gastrointest. Endosc."},{"issue":"3","key":"87737_CR5","doi-asserted-by":"publisher","first-page":"394","DOI":"10.1016\/j.gie.2006.03.933","volume":"65","author":"D Verma","year":"2007","unstructured":"Verma, D. et al. Establishing a true assessment of endoscopic competence in ERCP during training and beyond: A single-operator learning curve for deep biliary cannulation in patients with native papillary anatomy. Gastrointest. Endosc. 65(3), 394\u2013400 (2007).","journal-title":"Gastrointest. Endosc."},{"key":"87737_CR6","unstructured":"Geirhos, R. et al. Comparing deep neural networks against humans: object recognition when the signal gets weaker. Preprint at https:\/\/arxiv.org\/abs\/1706.06969 (2017)."},{"key":"87737_CR7","unstructured":"Gatys, L. A., Ecker, A. S. & Bethge, M. A neural algorithm of artistic style. Preprint at https:\/\/arxiv.org\/abs\/1508.06576 (2015)."},{"key":"87737_CR8","doi-asserted-by":"crossref","unstructured":"Choi, Y. et al. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 8789\u20138797 (2018).","DOI":"10.1109\/CVPR.2018.00916"},{"key":"87737_CR9","doi-asserted-by":"crossref","unstructured":"He, K. et al. Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision, 2980\u20132988 (2017).","DOI":"10.1109\/ICCV.2017.322"},{"key":"87737_CR10","doi-asserted-by":"crossref","unstructured":"Karras, T., Laine, S. & Aila, T. A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4401\u20134410 (2019).","DOI":"10.1109\/CVPR.2019.00453"},{"key":"87737_CR11","doi-asserted-by":"crossref","unstructured":"Khan, M. A. et al. Automatic detection of tympanic membrane and middle ear infection from oto-endoscopic images via convolutional neural networks. Neural Netw. (2020).","DOI":"10.1016\/j.neunet.2020.03.023"},{"issue":"7","key":"87737_CR12","doi-asserted-by":"publisher","first-page":"1562","DOI":"10.1109\/TMI.2018.2791721","volume":"37","author":"G Wang","year":"2018","unstructured":"Wang, G. et al. Interactive medical image segmentation using deep learning with image-specific fine tuning. IEEE Trans. Med. Imaging 37(7), 1562\u20131573 (2018).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"87737_CR13","doi-asserted-by":"crossref","unstructured":"Nie, D. et al. 3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients. In: International conference on medical image computing and computer-assisted intervention, 212\u2013220 (2016).","DOI":"10.1007\/978-3-319-46723-8_25"},{"key":"87737_CR14","doi-asserted-by":"crossref","unstructured":"Fu, H. et al. Deepvessel: Retinal vessel segmentation via deep learning and conditional random field. In: International conference on medical image computing and computer-assisted intervention, 132\u2013139 (2016).","DOI":"10.1007\/978-3-319-46723-8_16"},{"key":"87737_CR15","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1016\/j.media.2016.07.007","volume":"35","author":"T Kooi","year":"2017","unstructured":"Kooi, T. et al. Large scale deep learning for computer aided detection of mammographic lesions. Med. Image Anal. 35, 303\u2013312 (2017).","journal-title":"Med. Image Anal."},{"issue":"1","key":"87737_CR16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-018-25842-6","volume":"8","author":"H Takiyama","year":"2018","unstructured":"Takiyama, H. et al. Automatic anatomical classification of esophagogastroduodenoscopy images using deep convolutional neural networks. Sci. Rep. 8(1), 1\u20138 (2018).","journal-title":"Sci. Rep."},{"issue":"4","key":"87737_CR17","doi-asserted-by":"publisher","first-page":"653","DOI":"10.1007\/s10120-018-0793-2","volume":"21","author":"T Hirasawa","year":"2018","unstructured":"Hirasawa, T. et al. Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer 21(4), 653\u2013660 (2018).","journal-title":"Gastric Cancer"},{"key":"87737_CR18","unstructured":"Kim, S. et al. Learning to focus and track extreme climate events. In Proceedings of the British Machine Vision Conference (BMVC), (2019)."},{"key":"87737_CR19","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P. & Brox, T. U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention, 234\u2013241 (2015).","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"87737_CR20","unstructured":"Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. Preprint at https:\/\/arxiv.org\/abs\/1409.1556 (2014)."},{"key":"87737_CR21","doi-asserted-by":"crossref","unstructured":"Deng, J. et al. Imagenet: A large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition, 248\u2013255 (2009).","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"87737_CR22","doi-asserted-by":"crossref","unstructured":"He, K. et al. Identity mappings in deep residual networks. In: European conference on computer vision, 630\u2013645 (2016).","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"87737_CR23","doi-asserted-by":"crossref","unstructured":"Huang, G. et al. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 4700\u20134708 (2017).","DOI":"10.1109\/CVPR.2017.243"},{"key":"87737_CR24","unstructured":"Paszke, A. et al. Pytorch: An imperative style, high-performance deep learning library. In: Advances in neural information processing systems, 8026\u20138037,"},{"issue":"6","key":"87737_CR25","doi-asserted-by":"publisher","first-page":"1374","DOI":"10.2214\/AJR.17.18224","volume":"209","author":"JR Kim","year":"2017","unstructured":"Kim, J. R. et al. Computerized bone age estimation using deep learning based program: Evaluation of the accuracy and efficiency. Am. J. Roentgenol. 209(6), 1374\u20131380 (2017).","journal-title":"Am. J. Roentgenol."},{"key":"87737_CR26","doi-asserted-by":"publisher","first-page":"e00321","DOI":"10.1016\/j.btre.2019.e00321","volume":"22","author":"A Buetti-Dinh","year":"2019","unstructured":"Buetti-Dinh, A. et al. Deep neural networks outperform human expert\u2019s capacity in characterizing bioleaching bacterial biofilm composition. Biotechnol. Rep. 22, e00321 (2019).","journal-title":"Biotechnol. Rep."},{"issue":"3","key":"87737_CR27","first-page":"979","volume":"57","author":"AF Constantinescu","year":"2016","unstructured":"Constantinescu, A. F. et al. A computer-aided diagnostic system for intestinal polyps identified by wireless capsule endoscopy. Rom. J. Morphol. Embryol. 57(3), 979\u2013984 (2016).","journal-title":"Rom. J. Morphol. Embryol."},{"issue":"06","key":"87737_CR28","doi-asserted-by":"publisher","first-page":"522","DOI":"10.1055\/a-0855-3532","volume":"51","author":"L Wu","year":"2019","unstructured":"Wu, L. et al. A deep neural network improves endoscopic detection of early gastric cancer without blind spots. Endoscopy 51(06), 522\u2013531 (2019).","journal-title":"Endoscopy"},{"key":"87737_CR29","doi-asserted-by":"crossref","unstructured":"Saito, H. et al. Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest. Endosc. (2020).","DOI":"10.1016\/j.gie.2020.01.054"},{"key":"87737_CR30","doi-asserted-by":"crossref","unstructured":"Selvaraju, R. R. et al. Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, 618\u2013626 (2017).","DOI":"10.1109\/ICCV.2017.74"},{"issue":"1","key":"87737_CR31","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1186\/s40537-019-0192-5","volume":"6","author":"JM Johnson","year":"2019","unstructured":"Johnson, J. M. & Khoshgoftaar, T. M. Survey on deep learning with class imbalance. J. Big Data 6(1), 27 (2019).","journal-title":"J. Big Data"},{"key":"87737_CR32","doi-asserted-by":"crossref","unstructured":"Wang, S. et al. Training deep neural networks on imbalanced data sets. In: International joint conference on neural networks, 4368\u20134374 (2016).","DOI":"10.1109\/IJCNN.2016.7727770"},{"issue":"8","key":"87737_CR33","first-page":"3574","volume":"29","author":"SH Khan","year":"2017","unstructured":"Khan, S. H. et al. Cost-sensitive learning of deep feature representations from imbalanced data. IEEE Trans. Neural Netw. Learn. Syst. 29(8), 3574\u20133587 (2017).","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"87737_CR34","doi-asserted-by":"crossref","unstructured":"Lee, H., Park, M. & Kim, J. Plankton classification on imbalanced large scale database via convolutional neural networks with transfer learning. In: IEEE international conference on image processing, 3713\u20133717 (2016).","DOI":"10.1109\/ICIP.2016.7533053"},{"key":"87737_CR35","doi-asserted-by":"crossref","unstructured":"Pouyanfar, S. et al. Dynamic sampling in convolutional neural networks for imbalanced data classification. In: IEEE conference on multimedia information processing and retrieval, 112\u2013177 (2018).","DOI":"10.1109\/MIPR.2018.00027"},{"key":"87737_CR36","doi-asserted-by":"crossref","unstructured":"Huang, C. et al. Learning deep representation for imbalanced classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 5375\u20135384 (2016).","DOI":"10.1109\/CVPR.2016.580"},{"key":"87737_CR37","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1016\/j.neunet.2018.07.011","volume":"106","author":"M Buda","year":"2018","unstructured":"Buda, M., Maki, A. & Mazurowski, M. A. A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw. 106, 249\u2013259 (2018).","journal-title":"Neural Netw."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-021-87737-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-021-87737-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-021-87737-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,3]],"date-time":"2022-12-03T06:51:16Z","timestamp":1670050276000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-021-87737-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,16]]},"references-count":37,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["87737"],"URL":"https:\/\/doi.org\/10.1038\/s41598-021-87737-3","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,16]]},"assertion":[{"value":"4 November 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 March 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 April 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"8381"}}