{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:25:52Z","timestamp":1760149552129,"version":"build-2065373602"},"reference-count":28,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2023,8,22]],"date-time":"2023-08-22T00:00:00Z","timestamp":1692662400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Union\u2019s Horizon2020 research and innovation programme","award":["732111","RED2022-134964-T"],"award-info":[{"award-number":["732111","RED2022-134964-T"]}]},{"name":"MCIN\/AEI\/10.13039\/501100011033","award":["732111","RED2022-134964-T"],"award-info":[{"award-number":["732111","RED2022-134964-T"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Colorectal cancer is one of the leading death causes worldwide, but, fortunately, early detection highly increases survival rates, with the adenoma detection rate being one surrogate marker for colonoscopy quality. Artificial intelligence and deep learning methods have been applied with great success to improve polyp detection and localization and, therefore, the adenoma detection rate. In this regard, a comparison with clinical experts is required to prove the added value of the systems. Nevertheless, there is no standardized comparison in a laboratory setting before their clinical validation. The ClinExpPICCOLO comprises 65 unedited endoscopic images that represent the clinical setting. They include white light imaging and narrow band imaging, with one third of the images containing a lesion but, differently to another public datasets, the lesion does not appear well-centered in the image. Together with the dataset, an expert clinical performance baseline has been established with the performance of 146 gastroenterologists, who were required to locate the lesions in the selected images. Results shows statistically significant differences between experience groups. Expert gastroenterologists\u2019 accuracy was 77.74, while sensitivity and specificity were 86.47 and 74.33, respectively. These values can be established as minimum values for a DL method before performing a clinical trial in the hospital setting.<\/jats:p>","DOI":"10.3390\/jimaging9090167","type":"journal-article","created":{"date-parts":[[2023,8,22]],"date-time":"2023-08-22T08:58:54Z","timestamp":1692694734000},"page":"167","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Clinical Validation Benchmark Dataset and Expert Performance Baseline for Colorectal Polyp Localization Methods"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7630-353X","authenticated-orcid":false,"given":"Luisa F.","family":"S\u00e1nchez-Peralta","sequence":"first","affiliation":[{"name":"Jes\u00fas Us\u00f3n Minimally Invasive Surgery Centre, E-10071 C\u00e1ceres, Spain"},{"name":"AI4polypNET Thematic Network, E-08193 Barcelona, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3043-0012","authenticated-orcid":false,"given":"Ben","family":"Glover","sequence":"additional","affiliation":[{"name":"Imperial College London, London SW7 2BU, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1429-7936","authenticated-orcid":false,"given":"Cristina L.","family":"Saratxaga","sequence":"additional","affiliation":[{"name":"TECNALIA, Basque Research and Technology Alliance (BRTA), E-48160 Derio, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0287-5498","authenticated-orcid":false,"given":"Juan Francisco","family":"Ortega-Mor\u00e1n","sequence":"additional","affiliation":[{"name":"Jes\u00fas Us\u00f3n Minimally Invasive Surgery Centre, E-10071 C\u00e1ceres, Spain"},{"name":"AI4polypNET Thematic Network, E-08193 Barcelona, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1409-5253","authenticated-orcid":false,"given":"Scarlet","family":"Nazarian","sequence":"additional","affiliation":[{"name":"Imperial College London, London SW7 2BU, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3316-6571","authenticated-orcid":false,"given":"Artzai","family":"Pic\u00f3n","sequence":"additional","affiliation":[{"name":"TECNALIA, Basque Research and Technology Alliance (BRTA), E-48160 Derio, Spain"},{"name":"Department of Automatic Control and Systems Engineering, University of the Basque Country, E-48013 Bilbao, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4382-5075","authenticated-orcid":false,"given":"J. Blas","family":"Pagador","sequence":"additional","affiliation":[{"name":"Jes\u00fas Us\u00f3n Minimally Invasive Surgery Centre, E-10071 C\u00e1ceres, Spain"},{"name":"AI4polypNET Thematic Network, E-08193 Barcelona, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2138-988X","authenticated-orcid":false,"given":"Francisco M.","family":"S\u00e1nchez-Margallo","sequence":"additional","affiliation":[{"name":"Jes\u00fas Us\u00f3n Minimally Invasive Surgery Centre, E-10071 C\u00e1ceres, Spain"},{"name":"AI4polypNET Thematic Network, E-08193 Barcelona, Spain"},{"name":"RICORS-TERAV Network, ISCIII, E-28029 Madrid, Spain"},{"name":"Centro de Investigaci\u00f3n Biom\u00e9dica en Red de Enfermedades Cardiovasculares (CIBERCV), Instituto de Salud Carlos III, E-28029 Madrid, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"209","DOI":"10.3322\/caac.21660","article-title":"Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"71","author":"Sung","year":"2021","journal-title":"CA Cancer J. Clin."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1056\/NEJMoa1100370","article-title":"Colonoscopic Polypectomy and Long-Term Prevention of Colorectal-Cancer Deaths","volume":"366","author":"Zauber","year":"2012","journal-title":"N. Engl. J. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1661","DOI":"10.1053\/j.gastro.2019.01.260","article-title":"Magnitude, Risk Factors, and Factors Associated With Adenoma Miss Rate of Tandem Colonoscopy: A Systematic Review and Meta-analysis","volume":"156","author":"Zhao","year":"2019","journal-title":"Gastroenterology"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1298","DOI":"10.1056\/NEJMoa1309086","article-title":"Adenoma Detection Rate and Risk of Colorectal Cancer and Death","volume":"370","author":"Corley","year":"2014","journal-title":"N. Engl. J. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1038\/ajg.2012.417","article-title":"An endoscopic quality improvement program improves detection of colorectal adenomas","volume":"108","author":"Coe","year":"2013","journal-title":"Am. J. Gastroenterol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"616","DOI":"10.1136\/gutjnl-2014-307503","article-title":"Leadership training to improve adenoma detection rate in screening colonoscopy: A randomised trial","volume":"65","author":"Kaminski","year":"2016","journal-title":"Gut"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"820","DOI":"10.1109\/JPROC.2021.3054390","article-title":"A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises","volume":"109","author":"Zhou","year":"2021","journal-title":"Proc. IEEE"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1109\/TMI.2017.2664042","article-title":"Comparative Validation of Polyp Detection Methods in Video Colonoscopy: Results from the MICCAI 2015 Endoscopic Vision Challenge","volume":"36","author":"Bernal","year":"2017","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_9","first-page":"53","article-title":"Non-informative frame classification in colonoscopy videos using CNNs","volume":"2","author":"Alammari","year":"2018","journal-title":"ACM Int. Conf. Proceeding Ser."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1311","DOI":"10.1007\/s11548-023-02936-9","article-title":"Sources of performance variability in deep learning-based polyp detection","volume":"18","author":"Tran","year":"2023","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Li, M.D., Huang, Z.R., Shan, Q.Y., Chen, S.L., Zhang, N., Hu, H.T., and Wang, W. (2022). Performance and comparison of artificial intelligence and human experts in the detection and classification of colonic polyps. BMC Gastroenterol., 22.","DOI":"10.1186\/s12876-022-02605-2"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Xu, Y., Ding, W., Wang, Y., Tan, Y., Xi, C., Ye, N., Wu, D., and Xu, X. (2021). Comparison of diagnostic performance between convolutional neural networks and human endoscopists for diagnosis of colorectal polyp: A systematic review and meta-analysis. PLoS ONE, 16.","DOI":"10.1371\/journal.pone.0246892"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1874","DOI":"10.1016\/j.cgh.2019.09.009","article-title":"Artificial Intelligence-assisted System Improves Endoscopic Identification of Colorectal Neoplasms","volume":"18","author":"Kudo","year":"2019","journal-title":"Clin. Gastroenterol. Hepatol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"7184","DOI":"10.1002\/cam4.4261","article-title":"Artificial intelligence-assisted colonoscopy: A prospective, multicenter, randomized controlled trial of polyp detection","volume":"10","author":"Xu","year":"2021","journal-title":"Cancer Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"6598","DOI":"10.1038\/s41598-022-10597-y","article-title":"Clinical evaluation of a real-time artificial intelligence-based polyp detection system: A US multi-center pilot study","volume":"12","author":"Quan","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.gie.2020.06.059","article-title":"Performance of artificial intelligence in colonoscopy for adenoma and polyp detection: A systematic review and meta-analysis","volume":"93","author":"Hassan","year":"2021","journal-title":"Gastrointest. Endosc."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1186\/s13063-017-2099-9","article-title":"Barriers to the conduct of randomised clinical trials within all disease areas","volume":"18","author":"Djurisic","year":"2017","journal-title":"Trials"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"S\u00e1nchez-Peralta, L.F., Pagador, J.B., Pic\u00f3n, A., Calder\u00f3n, \u00c1.J., Andraka, N., Bilbao, R., Glover, B., Saratxaga, C.L., and S\u00e1nchez-Margallo, F.M. (2020). PICCOLO White-Light and Narrow-Band Imaging Colonoscopic Dataset: A Performance Comparative of Models and Datasets. Appl. Sci., 10.","DOI":"10.3390\/app10238501"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1007\/s11548-018-1864-x","article-title":"GTCreator: A flexible annotation tool for image-based datasets","volume":"14","author":"Bernal","year":"2019","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"886","DOI":"10.3109\/00365521.2016.1157892","article-title":"Il Comparison of adenoma detection rate and adenoma per colonoscopy as a quality indicator of colonoscopy","volume":"51","author":"Park","year":"2016","journal-title":"Scand. J. Gastroenterol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1055\/s-0042-100185","article-title":"Endoscopist characteristics that influence the quality of colonoscopy","volume":"48","author":"Jover","year":"2016","journal-title":"Endoscopy"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1016\/j.bpg.2017.07.002","article-title":"Adenoma detection rate and risk of colorectal cancer","volume":"31","author":"Wieszczy","year":"2017","journal-title":"Best Pract. Res. Clin. Gastroenterol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1016\/j.gie.2014.12.038","article-title":"Adenoma detection rate varies greatly during colonoscopy training","volume":"82","author":"Klanderman","year":"2015","journal-title":"Gastrointest. Endosc."},{"key":"ref_24","first-page":"815","article-title":"Time-of-day variation in the diagnostic quality of screening colonoscopies: A registry-based study","volume":"34","author":"Jaho","year":"2021","journal-title":"Ann. Gastroenterol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"900","DOI":"10.1016\/j.gie.2020.06.021","article-title":"Computer-aided detection-assisted colonoscopy: Classification and relevance of false positives","volume":"92","author":"Hassan","year":"2020","journal-title":"Gastrointest. Endosc."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1465","DOI":"10.1126\/science.aap8731","article-title":"Prevalence-induced concept change in human judgment","volume":"360","author":"Levari","year":"2018","journal-title":"Science"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1906","DOI":"10.3758\/s13423-021-01956-3","article-title":"Feedback moderates the effect of prevalence on perceptual decisions","volume":"28","author":"Lyu","year":"2021","journal-title":"Psychon. Bull. Rev."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1457","DOI":"10.1038\/ajg.2011.87","article-title":"Queue position in the endoscopic schedule impacts effectiveness of colonoscopy","volume":"106","author":"Lee","year":"2011","journal-title":"Am. J. Gastroenterol."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/9\/9\/167\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:39:45Z","timestamp":1760128785000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/9\/9\/167"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,22]]},"references-count":28,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["jimaging9090167"],"URL":"https:\/\/doi.org\/10.3390\/jimaging9090167","relation":{},"ISSN":["2313-433X"],"issn-type":[{"type":"electronic","value":"2313-433X"}],"subject":[],"published":{"date-parts":[[2023,8,22]]}}}