{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,15]],"date-time":"2025-12-15T14:18:23Z","timestamp":1765808303968,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T00:00:00Z","timestamp":1708646400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European FEDER fund","award":["PID2021-127567NB-I00"],"award-info":[{"award-number":["PID2021-127567NB-I00"]}]},{"DOI":"10.13039\/501100004837","name":"Spanish Ministry of Science, Innovation, and Universities","doi-asserted-by":"publisher","award":["PID2021-127567NB-I00"],"award-info":[{"award-number":["PID2021-127567NB-I00"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>HER2 overexpression is a prognostic and predictive factor observed in about 15% to 20% of breast cancer cases. The assessment of its expression directly affects the selection of treatment and prognosis. The measurement of HER2 status is performed by an expert pathologist who assigns a score of 0, 1, 2+, or 3+ based on the gene expression. There is a high probability of interobserver variability in this evaluation, especially when it comes to class 2+. This is reasonable as the primary cause of error in multiclass classification problems typically arises in the intermediate classes. This work proposes a novel approach to expand the decision limit and divide it into two additional classes, that is 1.5+ and 2.5+. This subdivision facilitates both feature learning and pathology assessment. The method was evaluated using various neural networks models capable of performing patch-wise grading of HER2 whole slide images (WSI). Then, the outcomes of the 7-class classification were merged back into 5 classes in accordance with the pathologists\u2019 criteria and to compare the results with the initial 5-class model. Optimal outcomes were achieved by employing colour transfer for data augmentation, and the ResNet-101 architecture with 7 classes. A sensitivity of 0.91 was achieved for class 2+ and 0.97 for 3+. Furthermore, this model offers the highest level of confidence, ranging from 92% to 94% for 2+ and 96% to 97% for 3+. In contrast, a dataset containing only 5 classes demonstrates a sensitivity performance that is 5% lower for the same network.<\/jats:p>","DOI":"10.3390\/a17030097","type":"journal-article","created":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T07:33:56Z","timestamp":1708673636000},"page":"97","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Deep Neural Networks for HER2 Grading of Whole Slide Images with Subclasses Levels"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7748-6756","authenticated-orcid":false,"given":"Anibal","family":"Pedraza","sequence":"first","affiliation":[{"name":"VISILAB Group (Vision and Artificial Intelligence Group), Universidad de Castilla-La Mancha, ETSII, 13071 Ciudad Real, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4808-9385","authenticated-orcid":false,"given":"Lucia","family":"Gonzalez","sequence":"additional","affiliation":[{"name":"Hospital General Universitario de Ciudad Real, 13005 Ciudad Real, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0841-4131","authenticated-orcid":false,"given":"Oscar","family":"Deniz","sequence":"additional","affiliation":[{"name":"VISILAB Group (Vision and Artificial Intelligence Group), Universidad de Castilla-La Mancha, ETSII, 13071 Ciudad Real, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7345-4869","authenticated-orcid":false,"given":"Gloria","family":"Bueno","sequence":"additional","affiliation":[{"name":"VISILAB Group (Vision and Artificial Intelligence Group), Universidad de Castilla-La Mancha, ETSII, 13071 Ciudad Real, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,23]]},"reference":[{"key":"ref_1","unstructured":"Taylor, C.R., and Rudbeck, L. (2013). Education Guide-Immunohistochemical Staining Methods, Dako Denmark A\/S."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1364","DOI":"10.5858\/arpa.2018-0902-SA","article-title":"Human epidermal growth factor receptor 2 testing in breast cancer: American Society of Clinical Oncology\/College of American Pathologists clinical practice guideline focused update","volume":"142","author":"Wolff","year":"2018","journal-title":"Arch. Pathol. Lab. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3744","DOI":"10.1200\/JCO.2014.55.5730","article-title":"Trastuzumab plus adjuvant chemotherapy for human epidermal growth factor receptor 2\u2013positive breast cancer: Planned joint analysis of overall survival from NSABP B-31 and NCCTG N9831","volume":"32","author":"Perez","year":"2014","journal-title":"J. Clin. Oncol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"100009","DOI":"10.1016\/j.modpat.2022.100009","article-title":"Interobserver Variation in the Assessment of Immunohistochemistry Expression Levels in HER2-Negative Breast Cancer: Can We Improve the Identification of Low Levels of HER2 Expression by Adjusting the Criteria? An International Interobserver Study","volume":"36","author":"Nawawi","year":"2023","journal-title":"Mod. Pathol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"102261","DOI":"10.1016\/j.compmedimag.2023.102261","article-title":"Interpretable HER2 scoring by evaluating clinical guidelines through a weakly supervised, constrained deep learning approach","volume":"108","author":"Pham","year":"2023","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"122051","DOI":"10.1016\/j.eswa.2023.122051","article-title":"The utility of a deep learning-based approach in Her-2\/neu assessment in breast cancer","volume":"238","author":"Kabir","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_7","first-page":"1","article-title":"HER2 classification in breast cancer cells: A new explainable machine learning application for immunohistochemistry","volume":"25","author":"Cordova","year":"2023","journal-title":"Oncol. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.csbj.2018.01.001","article-title":"Machine learning methods for histopathological image analysis","volume":"16","author":"Komura","year":"2018","journal-title":"Comput. Struct. Biotechnol. J."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1636","DOI":"10.1097\/PAS.0000000000001151","article-title":"Impact of deep learning assistance on the histopathologic review of lymph nodes for metastatic breast cancer","volume":"42","author":"Steiner","year":"2018","journal-title":"Am. J. Surg. Pathol."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Jain, E., Patel, A., Parwani, A.V., Shafi, S., Brar, Z., Sharma, S., and Mohanty, S.K. (2023). Whole Slide Imaging Technology and Its Applications: Current and Emerging Perspectives. Int. J. Surg. Pathol., Online ahead of print.","DOI":"10.1177\/10668969231185089"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"22","DOI":"10.4103\/2153-3539.157788","article-title":"Whole slide imaging for human epidermal growth factor receptor 2 immunohistochemistry interpretation: Accuracy, Precision, and reproducibility studies for digital manual and paired glass slide manual interpretation","volume":"6","author":"Wilbur","year":"2015","journal-title":"J. Pathol. Inform."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"440","DOI":"10.5858\/arpa.2020-0723-CP","article-title":"Validating Whole Slide Imaging Systems for Diagnostic Purposes in Pathology: Guideline Update From the College of American Pathologists in Collaboration With the American Society for Clinical Pathology and the Association for Pathology Informatics","volume":"146","author":"Evans","year":"2021","journal-title":"Arch. Pathol. Lab. Med."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2189","DOI":"10.1109\/TIP.2018.2795742","article-title":"Her2Net: A Deep Framework for Semantic Segmentation and Classification of Cell Membranes and Nuclei in Breast Cancer Evaluation","volume":"27","author":"Saha","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Cordeiro, C.Q., Ioshii, S.O., Alves, J.H., and Oliveira, L.F. (2018). An automatic patch-based approach for her-2 scoring in immunohistochemical breast cancer images using color features. arXiv.","DOI":"10.5753\/sbcas.2018.3685"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1016\/j.compbiomed.2019.05.020","article-title":"Automated segmentation of cell membranes to evaluate HER2 status in whole slide images using a modified deep learning network","volume":"110","author":"Khameneh","year":"2019","journal-title":"Comput. Biol. Med."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1111\/his.13333","article-title":"HER2 challenge contest: A detailed assessment of automated her 2 scoring algorithms in whole slide images of breast cancer tissues","volume":"72","author":"Qaiser","year":"2018","journal-title":"Histopathology"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kabak\u00e7\u0131, K.A., \u00c7ak\u0131r, A., T\u00fcrkmen, \u0130., T\u00f6reyin, B.U., and \u00c7apar, A. (2021). Automated scoring of CerbB2\/HER2 receptors using histogram based analysis of immunohistochemistry breast cancer tissue images. Biomed. Signal Process. Control, 69.","DOI":"10.1016\/j.bspc.2021.102924"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chen, Z., Zhang, J., Che, S., Huang, J., Han, X., and Yuan, Y. (2021, January 19\u201321). Diagnose like a pathologist: Weakly-supervised pathologist-tree network for slide-level immunohistochemical scoring. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual Event.","DOI":"10.1609\/aaai.v35i1.16076"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"B\u00f3rquez, S., Pezoa, R., Salinas, L., and Torres, C.E. (2023). Uncertainty estimation in the classification of histopathological images with HER2 overexpression using Monte Carlo Dropout. Biomed. Signal Process. Control, 85.","DOI":"10.1016\/j.bspc.2023.104864"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Che, Y., Ren, F., Zhang, X., Cui, L., Wu, H., and Zhao, Z. (2023). Immunohistochemical HER2 Recognition and Analysis of Breast Cancer Based on Deep Learning. Diagnostics, 13.","DOI":"10.3390\/diagnostics13020263"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Vallez, N., Espinosa-Aranda, J.L., Pedraza, A., Deniz, O., and Bueno, G. (2023). Deep Learning within a DICOM WSI Viewer for Histopathology. Appl. Sci., 13.","DOI":"10.3390\/app13179527"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"102486","DOI":"10.1016\/j.media.2022.102486","article-title":"SlideGraph+: Whole slide image level graphs to predict HER2 status in breast cancer","volume":"80","author":"Lu","year":"2022","journal-title":"Med. Image Anal."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"42964","DOI":"10.1038\/srep42964","article-title":"Image processing in digital pathology: An opportunity to solve inter-batch variability of immunohistochemical staining","volume":"7","author":"Allard","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_24","unstructured":"Magee, D., Treanor, D., Crellin, D., Shires, M., Smith, K., Mohee, K., and Quirke, P. (2009, January 13\u201317). Colour normalisation in digital histopathology images. Proceedings of the Proc Optical Tissue Image Analysis in Microscopy, Histopathology and Endoscopy (MICCAI Workshop), Daniel Elson, Shenzhen, China."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/38.946629","article-title":"Color transfer between images","volume":"21","author":"Reinhard","year":"2001","journal-title":"IEEE Comput. Graph. Appl."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Macenko, M., Niethammer, M., Marron, J.S., Borl, D., Woosley, J.T., Guan, X., Schmitt, C., and Thomas, N.E. (July, January 28). A method for normalizing histology slides for quantitative analysis. Proceedings of the 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Boston, MA, USA.","DOI":"10.1109\/ISBI.2009.5193250"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Shapira, D., Avidan, S., and Hel-Or, Y. (2013, January 15\u201318). Multiple histogram matching. Proceedings of the 2013 IEEE International Conference on Image Processing, IEEE, Melbourne, Australia.","DOI":"10.1109\/ICIP.2013.6738468"},{"key":"ref_28","first-page":"657","article-title":"Automatic batch-invariant color segmentation of histological cancer images","volume":"2011","author":"Kothari","year":"2011","journal-title":"Proc. IEEE Int. Symp. Biomed. Imaging"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1729","DOI":"10.1109\/TBME.2014.2303294","article-title":"A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution","volume":"61","author":"Khan","year":"2014","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.compmedimag.2017.06.002","article-title":"Automatic quantification of IHC stain in breast TMA using colour analysis","volume":"61","author":"Bueno","year":"2017","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1016\/j.neunet.2018.07.011","article-title":"A systematic study of the class imbalance problem in convolutional neural networks","volume":"106","author":"Buda","year":"2018","journal-title":"Neural Netw."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1369","DOI":"10.1038\/s41416-023-02143-y","article-title":"Predicting the HER2 status in oesophageal cancer from tissue microarrays using convolutional neural networks","volume":"128","author":"Pisula","year":"2023","journal-title":"Br. J. Cancer"},{"key":"ref_33","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_35","unstructured":"Simonyan, K., and Zisserman, A. (2014, January 14\u201316). Very deep convolutional networks for large-scale image recognition. Proceedings of the International Conference on Learning Representations, Banff, AB, Canada."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the CVPR Conference, the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_38","unstructured":"Yu, W., Yang, K., Bai, Y., Xiao, T., Yao, H., and Rui, Y. (2016, January 20\u201322). Visualizing and comparing AlexNet and VGG using deconvolutional layers. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_39","unstructured":"Bishop, C. (2006). Pattern Recognition and Machine Learning, Springer."},{"key":"ref_40","unstructured":"Murphy, K.P. (2012). Machine Learning: A Probabilistic Perspective, MIT Press."},{"key":"ref_41","unstructured":"Pascanu, R., Mikolov, T., and Bengio, Y. (2013, January 17\u201319). On the difficulty of training recurrent neural networks. Proceedings of the International conference on Machine Learning, Pmlr, Atlanta, GA, USA."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/3\/97\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:03:44Z","timestamp":1760105024000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/3\/97"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,23]]},"references-count":41,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["a17030097"],"URL":"https:\/\/doi.org\/10.3390\/a17030097","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2024,2,23]]}}}