{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T06:25:31Z","timestamp":1767853531032,"version":"3.49.0"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,6,14]],"date-time":"2021-06-14T00:00:00Z","timestamp":1623628800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,14]],"date-time":"2021-06-14T00:00:00Z","timestamp":1623628800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The Gleason score contributes significantly in predicting prostate cancer outcomes and selecting the appropriate treatment option, which is affected by well-known inter-observer variations. We present a novel deep learning-based automated Gleason grading system that does not require extensive region-level manual annotations by experts and\/or complex algorithms for the automatic generation of region-level annotations. A total of 6664 and 936 prostate needle biopsy single-core slides (689 and 99 cases) from two institutions were used for system discovery and validation, respectively. Pathological diagnoses were converted into grade groups and used as the reference standard. The grade group prediction accuracy of the system was 77.5% (95% confidence interval (CI): 72.3\u201382.7%), the Cohen\u2019s kappa score (<jats:italic>\u03ba<\/jats:italic>) was 0.650 (95% CI: 0.570\u20130.730), and the quadratic-weighted kappa score (<jats:italic>\u03ba<\/jats:italic><jats:sub>quad<\/jats:sub>) was 0.897 (95% CI: 0.815\u20130.979). When trained on 621 cases from one institution and validated on 167 cases from the other institution, the system\u2019s accuracy reached 67.4% (95% CI: 63.2\u201371.6%), <jats:italic>\u03ba<\/jats:italic> 0.553 (95% CI: 0.495\u20130.610), and the <jats:italic>\u03ba<\/jats:italic><jats:sub>quad<\/jats:sub> 0.880 (95% CI: 0.822\u20130.938). In order to evaluate the impact of the proposed method, performance comparison with several baseline methods was also performed. While limited by case volume and a few more factors, the results of this study can contribute to the potential development of an artificial intelligence system to diagnose other cancers without extensive region-level annotations.<\/jats:p>","DOI":"10.1038\/s41746-021-00469-6","type":"journal-article","created":{"date-parts":[[2021,6,14]],"date-time":"2021-06-14T10:02:37Z","timestamp":1623664957000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":47,"title":["Yet Another Automated Gleason Grading System (YAAGGS) by weakly supervised deep learning"],"prefix":"10.1038","volume":"4","author":[{"given":"Yechan","family":"Mun","sequence":"first","affiliation":[]},{"given":"Inyoung","family":"Paik","sequence":"additional","affiliation":[]},{"given":"Su-Jin","family":"Shin","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9171-7258","authenticated-orcid":false,"given":"Tae-Yeong","family":"Kwak","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4274-4652","authenticated-orcid":false,"given":"Hyeyoon","family":"Chang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,6,14]]},"reference":[{"key":"469_CR1","unstructured":"Prostate Cancer\u2014Cancer Stat Facts. https:\/\/seer.cancer.gov\/statfacts\/html\/prost.html. Accessed 6 Apr 2020."},{"key":"469_CR2","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1186\/1746-1596-1-4","volume":"1","author":"A Buhmeida","year":"2006","unstructured":"Buhmeida, A., Pyrh\u00f6nen, S., Laato, M. & Collan, Y. Prognostic factors in prostate cancer. Diagn. Pathol. 1, 4, https:\/\/doi.org\/10.1186\/1746-1596-1-4 (2006).","journal-title":"Diagn. Pathol."},{"key":"469_CR3","unstructured":"National Comprehensive Cancer Network (NCCN). Practice Guidelines in Oncology: Prostate Cancer Early Detection. Version 2. 2019. https:\/\/www.nccn.org\/professionals\/physician_gls\/pdf\/prostate_detection.pdf Accessed 12 Mar 2020 (2019)."},{"key":"469_CR4","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1186\/s13000-016-0478-2","volume":"11","author":"J Gordetsky","year":"2016","unstructured":"Gordetsky, J. & Epstein, J. Grading of prostatic adenocarcinoma: current state and prognostic implications. Diagn. Pathol. 11, 25, https:\/\/doi.org\/10.1186\/s13000-016-0478-2 (2016).","journal-title":"Diagn. Pathol."},{"key":"469_CR5","first-page":"125","volume":"50","author":"DF Gleason","year":"1966","unstructured":"Gleason, D. F. Classification of prostatic carcinomas. Cancer Chemother. Rep. 50, 125\u20138 (1966).","journal-title":"Cancer Chemother. Rep."},{"key":"469_CR6","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1053\/hupa.2001.21135","volume":"32","author":"WC Allsbrook","year":"2001","unstructured":"Allsbrook, W. C. et al. Interobserver reproducibility of Gleason grading of prostatic carcinoma: general pathologist. Hum. Pathol. 32, 81\u201388, https:\/\/doi.org\/10.1053\/hupa.2001.21135 (2001).","journal-title":"Hum. Pathol."},{"key":"469_CR7","doi-asserted-by":"publisher","first-page":"420","DOI":"10.1080\/21681805.2016.1206619","volume":"50","author":"TA Ozkan","year":"2016","unstructured":"Ozkan, T. A. et al. Interobserver variability in Gleason histological grading of prostate cancer. Scand. J. Urol. 50, 420\u2013424, https:\/\/doi.org\/10.1080\/21681805.2016.1206619 (2016).","journal-title":"Scand. J. Urol."},{"issue":"5","key":"469_CR8","doi-asserted-by":"publisher","first-page":"753","DOI":"10.1111\/j.1464-410x.2012.11611.x","volume":"111","author":"PM Pierorazio","year":"2013","unstructured":"Pierorazio, P. M., Walsh, P. C., Partin, A. W. & Epstein, J. I. Prognostic Gleason Grade Grouping: data based on the modified Gleason scoring system. BJU Int. 111(5), 753\u2013760, https:\/\/doi.org\/10.1111\/j.1464-410x.2012.11611.x (2013).","journal-title":"BJU Int."},{"key":"469_CR9","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1097\/pas.0000000000000530","volume":"40","author":"JI Epstein","year":"2016","unstructured":"Epstein, J. I. et al. The 2014 International Society of Urological Pathology (ISUP) Consensus Conference on Gleason Grading of Prostatic Carcinoma. Am. J. Surg. Pathol. 40, 2, https:\/\/doi.org\/10.1097\/pas.0000000000000530 (2016).","journal-title":"Am. J. Surg. Pathol."},{"key":"469_CR10","doi-asserted-by":"crossref","unstructured":"Moch, H., Humphrey, P. A., Ulbright, T. M. & Reuter, V. E. WHO Classification of Tumours of the Urinary System and Male Genital Organs. Fourth edition. (IARC, Lyon, 2016).","DOI":"10.1016\/j.eururo.2016.02.029"},{"issue":"1","key":"469_CR11","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1146\/annurev-bioeng-071516-044442","volume":"19","author":"D Shen","year":"2017","unstructured":"Shen, D., Wu, G. & Suk, H.-I. Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19(1), 221\u2013248, https:\/\/doi.org\/10.1146\/annurev-bioeng-071516-044442 (2017).","journal-title":"Annu. Rev. Biomed. Eng."},{"key":"469_CR12","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","volume":"42","author":"G Litjens","year":"2017","unstructured":"Litjens, G. et al. A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60\u201388, https:\/\/doi.org\/10.1016\/j.media.2017.07.005 (2017).","journal-title":"Med. Image Anal."},{"key":"469_CR13","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1177\/2F0300985813503570","volume":"51","author":"JD Webster","year":"2014","unstructured":"Webster, J. D. & Dunstan, R. W. Whole-slide imaging and automated image analysis: considerations and opportunities in the practice of pathology. Vet. Pathol. 51, 211\u2013223, https:\/\/doi.org\/10.1177\/2F0300985813503570 (2014).","journal-title":"Vet. Pathol."},{"issue":"12","key":"469_CR14","doi-asserted-by":"publisher","first-page":"1860,","DOI":"10.3390\/cancers11121860","volume":"11","author":"H-S Ryu","year":"2019","unstructured":"Ryu, H.-S. et al. Automated Gleason scoring and tumor quantification in prostate core needle biopsy images using deep neural networks and its comparison with pathologist-based assessment. Cancers 11(12), 1860, https:\/\/doi.org\/10.3390\/cancers11121860 (2019).","journal-title":"Cancers"},{"key":"469_CR15","doi-asserted-by":"publisher","unstructured":"Nagpal, K. et al. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. NPJ Digit. Med. 2 (1) https:\/\/doi.org\/10.1038\/s41746-019-0112-2 (2019).","DOI":"10.1038\/s41746-019-0112-2"},{"issue":"2","key":"469_CR16","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1016\/s1470-2045(19)30738-7","volume":"21","author":"P Str\u00f6m","year":"2020","unstructured":"Str\u00f6m, P. et al. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. Lancet Oncol. 21(2), 222\u2013232. https:\/\/doi.org\/10.1016\/s1470-2045(19)30738-7 (2020).","journal-title":"Lancet Oncol."},{"issue":"2","key":"469_CR17","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1016\/s1470-2045(19)30739-9","volume":"21","author":"W Bulten","year":"2020","unstructured":"Bulten, W. et al. Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol. 21(2), 233\u2013241, https:\/\/doi.org\/10.1016\/s1470-2045(19)30739-9 (2020).","journal-title":"Lancet Oncol."},{"key":"469_CR18","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1016\/j.media.2018.09.005","volume":"50","author":"G Nir","year":"2018","unstructured":"Nir, G. et al. Automatic grading of prostate cancer in digitized histopathology images: learning from multiple experts. Med Image Anal. 50, 167\u2013180 (2018).","journal-title":"Med Image Anal."},{"issue":"5","key":"469_CR19","doi-asserted-by":"publisher","first-page":"1413","DOI":"10.1109\/JBHI.2019.2944643","volume":"24","author":"D Karimi","year":"2020","unstructured":"Karimi, D. et al. Deep learning-based Gleason grading of prostate cancer from histopathology images\u2014role of multiscale decision aggregation and data augmentation. IEEE J. Biomed. Health Inform. 24(5), 1413\u201326 (2020).","journal-title":"IEEE J. Biomed. Health Inform."},{"issue":"1","key":"469_CR20","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen, J. A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20(1), 37\u201346, https:\/\/doi.org\/10.1177\/001316446002000104 (1960).","journal-title":"Educ. Psychol. Meas."},{"issue":"4","key":"469_CR21","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1037\/h0026256","volume":"70","author":"J Cohen","year":"1968","unstructured":"Cohen, J. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol. Bull. 70(4), 213\u2013220 (1968). 10.1037\/h0026256.","journal-title":"Psychol. Bull."},{"key":"469_CR22","unstructured":"Pathak, D., Shelhamer, E., Long, J. & Darrell, T. Fully convolutional multi-class multiple instance learning. In Proceedings of the International Conference on Learning Representations (ICLR) Workshop (2015)."},{"key":"469_CR23","doi-asserted-by":"publisher","unstructured":"Lu, M., Williamson, D., Chen, T., Chen, R. & Barbieri, M. Data efficient and weakly supervised computational pathology on whole slide images. Nat. Biomed. Eng. (2021). https:\/\/doi.org\/10.1038\/s41551-020-00682-w.","DOI":"10.1038\/s41551-020-00682-w"},{"key":"469_CR24","doi-asserted-by":"publisher","first-page":"1007","DOI":"10.1043\/1543-2165(2003)127<1007:AGGOPA>2.0.CO;2","volume":"127","author":"AA Renshaw","year":"2003","unstructured":"Renshaw, A. A. et al. Accurate Gleason grading of prostatic adenocarcinoma in prostate needle biopsies by general pathologists. Arch. Pathol. Lab. Med. 127, 1007\u20138, https:\/\/doi.org\/10.1043\/1543-2165(2003)127<1007:AGGOPA>2.0.CO;2 (2003).","journal-title":"Arch. Pathol. Lab. Med."},{"key":"469_CR25","doi-asserted-by":"publisher","first-page":"1004","DOI":"10.1043\/1543-2165(2005)129[1004:ACOIRO]2.0.CO;2","volume":"129","author":"T Oyama","year":"2005","unstructured":"Oyama, T. et al. A comparison of interobserver reproducibility of Gleason grading of prostatic carcinoma in Japan and the United States. Arch. Pathol. Lab. Med. 129, 1004\u201310, https:\/\/doi.org\/10.1043\/1543-2165(2005)129[1004:ACOIRO]2.0.CO;2 (2005).","journal-title":"Arch. Pathol. Lab. Med."},{"key":"469_CR26","unstructured":"Li, J. et al. An attention-based multi-resolution model for prostate whole-slide image classification and localization. https:\/\/arXiv.org\/abs\/1905.13208 (2019)."},{"key":"469_CR27","doi-asserted-by":"crossref","unstructured":"Mormont R, Geurts P, Mar\u00e9e R. Comparison of deep transfer learning strategies for digital pathology. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops 2018 (pp. 2262\u20132271).","DOI":"10.1109\/CVPRW.2018.00303"},{"key":"469_CR28","unstructured":"Ilse M, Tomczak J, Welling M. Attention-based deep multiple instance learning. In Proceedings of the International Conference on Machine Learning 2018 (pp. 2127\u20132136). PMLR."},{"key":"469_CR29","doi-asserted-by":"publisher","first-page":"e1914645","DOI":"10.1001\/jamanetworkopen.2019.14645","volume":"2","author":"N Tomita","year":"2019","unstructured":"Tomita, N. et al. Attention-based deep neural networks for detection of cancerous and precancerous esophagus tissue on histopathological slides. JAMA Netw. Open 2, e1914645, https:\/\/doi.org\/10.1001\/jamanetworkopen.2019.14645 (2019).","journal-title":"JAMA Netw. Open"},{"key":"469_CR30","doi-asserted-by":"publisher","unstructured":"Epstein, J. I. et al. The 2014 International Society of Urological Pathology (ISUP) Consensus Conference on Gleason Grading of Prostatic Carcinoma: definition of grading patterns and proposal for a new grading system. Am. J. Surg. Pathol. 40, 244-52 (2016). https:\/\/doi.org\/10.1097\/PAS.0000000000000530","DOI":"10.1097\/PAS.0000000000000530"},{"issue":"8","key":"469_CR31","doi-asserted-by":"publisher","first-page":"1301","DOI":"10.1038\/s41591-019-0508-1","volume":"25","author":"G Campanella","year":"2019","unstructured":"Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25(8), 1301\u20131309, https:\/\/doi.org\/10.1038\/s41591-019-0508-1 (2019).","journal-title":"Nat. Med."},{"key":"469_CR32","doi-asserted-by":"publisher","unstructured":"Huang, G., Liu, Z., Maaten, L. V. D. & Weinberger, K. Q. Densely connected convolutional networks. 2017 In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. https:\/\/doi.org\/10.1109\/cvpr.2017.243","DOI":"10.1109\/cvpr.2017.243"},{"key":"469_CR33","doi-asserted-by":"publisher","unstructured":"He, K. et al. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. https:\/\/doi.org\/10.1109\/cvpr.2016.90","DOI":"10.1109\/cvpr.2016.90"},{"key":"469_CR34","first-page":"2579","volume":"9","author":"L Van der Maaten","year":"2008","unstructured":"Van der Maaten, L. & Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579\u20132605 (2008).","journal-title":"J. Mach. Learn. Res."},{"issue":"3","key":"469_CR35","doi-asserted-by":"publisher","first-page":"276","DOI":"10.11613\/BM.2012.031","volume":"22","author":"ML McHugh","year":"2012","unstructured":"McHugh, M. L. Interrater reliability: the Kappa statistic. Biochem. Med. 22(3), 276\u2013282 (2012).","journal-title":"Biochem. Med."},{"issue":"3","key":"469_CR36","doi-asserted-by":"publisher","first-page":"178","DOI":"10.1080\/09296174.2013.799918","volume":"20","author":"SA Wallis","year":"2013","unstructured":"Wallis, S. A. Binomial confidence intervals and contingency tests: mathematical fundamentals and the evaluation of alternative methods. J. Quant. Linguist 20(3), 178\u2013208, https:\/\/doi.org\/10.1080\/09296174.2013.799918 (2013).","journal-title":"J. Quant. Linguist"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00469-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00469-6","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00469-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,3]],"date-time":"2022-12-03T18:54:51Z","timestamp":1670093691000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00469-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,14]]},"references-count":36,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["469"],"URL":"https:\/\/doi.org\/10.1038\/s41746-021-00469-6","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,14]]},"assertion":[{"value":"9 April 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 May 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 June 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"H.C., I.P. and Y.M. are employees of Deep Bio Inc. T.-Y.K. is the chief technology officer of Deep Bio Inc. S.-J.S. declares no conflict of interest.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"99"}}