{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T06:10:15Z","timestamp":1775887815060,"version":"3.50.1"},"reference-count":48,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2020,8,23]],"date-time":"2020-08-23T00:00:00Z","timestamp":1598140800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Breast cancer is the most frequently diagnosed cancer in woman. The correct identification of the HER2 receptor is a matter of major importance when dealing with breast cancer: an over-expression of HER2 is associated with aggressive clinical behaviour; moreover, HER2 targeted therapy results in a significant improvement in the overall survival rate. In this work, we employ a pipeline based on a cascade of deep neural network classifiers and multi-instance learning to detect the presence of HER2 from Haematoxylin\u2013Eosin slides, which partly mimics the pathologist\u2019s behaviour by first recognizing cancer and then evaluating HER2. Our results show that the proposed system presents a good overall effectiveness. Furthermore, the system design is prone to further improvements that can be easily deployed in order to increase the effectiveness score.<\/jats:p>","DOI":"10.3390\/jimaging6090082","type":"journal-article","created":{"date-parts":[[2020,8,23]],"date-time":"2020-08-23T21:28:06Z","timestamp":1598218086000},"page":"82","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Detection of HER2 from Haematoxylin-Eosin Slides Through a Cascade of Deep Learning Classifiers via Multi-Instance Learning"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8215-5502","authenticated-orcid":false,"given":"David","family":"La Barbera","sequence":"first","affiliation":[{"name":"Department of Mathematics, Computer Science and Physics, University of Udine, 33100 Udine, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8312-1681","authenticated-orcid":false,"given":"Ant\u00f3nio","family":"Pol\u00f3nia","sequence":"additional","affiliation":[{"name":"Department of Pathology, Ipatimup Diagnostics, Institute of Molecular Pathology and Immunology, University of Porto, 4169-007 Porto, Portugal"},{"name":"i3S\u2014Instituto de Investiga\u00e7\u00e3o e Inova\u00e7\u00e3o em Sa\u00fade, Universidade do Porto, 4169-007 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9191-3280","authenticated-orcid":false,"given":"Kevin","family":"Roitero","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Computer Science and Physics, University of Udine, 33100 Udine, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6591-5063","authenticated-orcid":false,"given":"Eduardo","family":"Conde-Sousa","sequence":"additional","affiliation":[{"name":"i3S\u2014Instituto de Investiga\u00e7\u00e3o e Inova\u00e7\u00e3o em Sa\u00fade, Universidade do Porto, 4169-007 Porto, Portugal"},{"name":"INEB\u2014Instituto de Engenharia Biom\u00e9dica, Universidade do Porto, 4169-007 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0144-3802","authenticated-orcid":false,"given":"Vincenzo","family":"Della Mea","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Computer Science and Physics, University of Udine, 33100 Udine, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2020,8,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"7","DOI":"10.3322\/caac.21590","article-title":"Cancer statistics, 2020","volume":"70","author":"Siegel","year":"2020","journal-title":"CA A Cancer J. Clin."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1364","DOI":"10.5858\/arpa.2018-0902-SA","article-title":"Human Epidermal Growth Factor Receptor 2 Testing in Breast Cancer: American Society of Clinical Oncology\/College of American Pathologists Clinical Practice Guideline Focused Update","volume":"142","author":"Wolff","year":"2018","journal-title":"Arch. Pathol. Lab. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2806","DOI":"10.1016\/j.ejca.2008.09.013","article-title":"Her2-positive breast cancer: Herceptin and beyond","volume":"44","author":"Esteva","year":"2008","journal-title":"Eur. J. Cancer"},{"key":"ref_4","first-page":"5078","article-title":"HER-2-Targeted Therapy","volume":"9","author":"Nahta","year":"2003","journal-title":"Clin. Cancer Res."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"55","DOI":"10.5858\/2010-0454-RAR.1","article-title":"HER2: Biology, detection, and clinical implications","volume":"135","author":"Gutierrez","year":"2011","journal-title":"Arch. Pathol. Lab. Med."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1002\/elan.201100501","article-title":"An electrochemical immunoassay for HER2 detection","volume":"24","author":"Harris","year":"2012","journal-title":"Electroanalysis"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2634","DOI":"10.1158\/1078-0432.CCR-09-2042","article-title":"Detection and HER2 expression of circulating tumor cells: Prospective monitoring in breast cancer patients treated in the neoadjuvant GeparQuattro trial","volume":"16","author":"Riethdorf","year":"2010","journal-title":"Clin. Cancer Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3276","DOI":"10.1158\/1078-0432.CCR-12-3768","article-title":"Noninvasive detection of HER2 amplification with plasma DNA digital PCR","volume":"19","author":"Gevensleben","year":"2013","journal-title":"Clin. Cancer Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1016\/j.snb.2010.01.067","article-title":"Detection of HER2 breast cancer biomarker using the opto-fluidic ring resonator biosensor","volume":"146","author":"Gohring","year":"2010","journal-title":"Sens. Actuators B Chem."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"45938","DOI":"10.1038\/srep45938","article-title":"Relevance of deep learning to facilitate the diagnosis of HER2 status in breast cancer","volume":"7","author":"Vandenberghe","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1972","DOI":"10.1001\/jama.291.16.1972","article-title":"HER-2 testing in breast cancer using parallel tissue-based methods","volume":"291","author":"Yaziji","year":"2004","journal-title":"JAMA"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"642","DOI":"10.1109\/TBME.2009.2035305","article-title":"Computerized image-based detection and grading of lymphocytic infiltration in HER2+ breast cancer histopathology","volume":"57","author":"Basavanhally","year":"2009","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"46450","DOI":"10.1038\/srep46450","article-title":"Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent","volume":"7","author":"Gilmore","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_14","first-page":"4","article-title":"Deep learning for health informatics","volume":"21","author":"Wong","year":"2016","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med Image Anal."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1146\/annurev-bioeng-071516-044442","article-title":"Deep learning in medical image analysis","volume":"19","author":"Shen","year":"2017","journal-title":"Annu. Rev. Biomed. Eng."},{"key":"ref_17","unstructured":"Zha, Z.J., Hua, X.S., Mei, T., Wang, J., Qi, G.J., and Wang, Z. (2008, January 23\u201328). Joint multi-label multi-instance learning for image classification. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhou, Z.H., and Xu, J.M. (2007, January 20\u201324). On the relation between multi-instance learning and semi-supervised learning. Proceedings of the 24th International Conference on Machine Learning, Corvallis, OR, USA.","DOI":"10.1145\/1273496.1273643"},{"key":"ref_19","unstructured":"Zhou, Z.H., and Zhang, M.L. (2002, January 22\u201325). Neural networks for multi-instance learning. Proceedings of the International Conference on Intelligent Information Technology, Beijing, China."},{"key":"ref_20","unstructured":"Zhou, Z.H. (2004). Multi-instance learning: A survey. Dep. Comput. Sci. Technol. Nanjing Univ. Tech. Rep., 2."},{"key":"ref_21","unstructured":"Zhou, Z.H., and Zhang, M.L. (2003, January 22\u201326). Ensembles of multi-instance learners. Proceedings of the 14th European Conference on Machine Learning, Cavtat-Dubrovnik, Croatia."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sze, V., Chen, Y.H., Emer, J., Suleiman, A., and Zhang, Z. (May, January 30). Hardware for machine learning: Challenges and opportunities. Proceedings of the 2017 IEEE Custom Integrated Circuits Conference (CICC), Austin, TX, USA.","DOI":"10.1109\/CICC.2017.7993626"},{"key":"ref_23","unstructured":"Carbonneau, M.A., Cheplygina, V., Granger, E., and Gagnon, G. (2016). Multiple Instance Learning: A Survey of Problem Characteristics and Applications. Pattern Recognit."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Herrera, F., Ventura, S., Bello, R., Cornelis, C., Zafra, A., Sanchez Tarrago, D., and Vluymans, S. (2016). Multiple Instance Learning. Foundations and Algorithms, Springer.","DOI":"10.1007\/978-3-319-47759-6"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Foulds, J., and Frank, E. (2010). A Review of Multi-Instance Learning Assumptions. Knowl. Eng. Rev., 25.","DOI":"10.1017\/S026988890999035X"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41591-019-0508-1","article-title":"Clinical-grade computational pathology using weakly supervised deep learning on whole slide images","volume":"25","author":"Campanella","year":"2019","journal-title":"Nat. Med."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sudharshan, P.J., Petitjean, C., Spanhol, F., Oliveira, L., Heutte, L., and Honeine, P. (2018). Multiple Instance Learning for Histopathological Breast Cancer Image Classification. Expert Syst. Appl.","DOI":"10.1016\/j.eswa.2018.09.049"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Couture, H.D., Williams, L.A., Geradts, J., Nyante, S.J., Butler, E.N., Marron, J.S., Perou, C.M., Troester, M.A., and Niethammer, M. (2018). Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype. NPJ Breast Cancer.","DOI":"10.1038\/s41523-018-0079-1"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1111\/his.13333","article-title":"HER2 challenge contest: A detailed assessment of automated HER2 scoring algorithms in whole slide images of breast cancer tissues","volume":"72","author":"Qaiser","year":"2017","journal-title":"Histopathology"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2087","DOI":"10.1016\/j.prp.2018.10.015","article-title":"Comparison between digital image analysis and visual assessment of immunohistochemical HER2 expression in breast cancer","volume":"214","author":"Jakobsen","year":"2018","journal-title":"Pathol.-Res. Pract."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Mukundan, R. (2019). Analysis of Image Feature Characteristics for Automated Scoring of HER2 in Histology Slides. J. Imaging, 5.","DOI":"10.3390\/jimaging5030035"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2620","DOI":"10.1109\/TMI.2019.2907049","article-title":"Learning Where to See: A Novel Attention Model for Automated Immunohistochemical Scoring","volume":"38","author":"Qaiser","year":"2019","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"158","DOI":"10.3349\/ymj.2019.60.2.158","article-title":"Image Analysis of HER2 Immunohistochemical Staining of Surgical Breast Cancer Specimens","volume":"60","author":"Yim","year":"2019","journal-title":"Yonsei Med. J."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1007\/s10549-020-05546-0","article-title":"Quantitative digital imaging analysis of HER2 immunohistochemistry predicts the response to anti-HER2 neoadjuvant chemotherapy in HER2-positive breast carcinoma","volume":"180","author":"Li","year":"2020","journal-title":"Breast Cancer Res. Treat."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1007\/s11760-015-0828-7","article-title":"Optimizing the color-to-grayscale conversion for image classification","volume":"10","author":"Kalkan","year":"2016","journal-title":"Signal Image Video Process."},{"key":"ref_36","first-page":"1223","article-title":"Classification of Histologic Images Using a Single Staining: Experiments With Deep Learning on Deconvolved Images","volume":"270","author":"Pilutti","year":"2020","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_37","unstructured":"Iandola, F., Moskewicz, M., Karayev, S., Girshick, R., Darrell, T., and Keutzer, K. (2014). Densenet: Implementing efficient convnet descriptor pyramids. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1007\/s00138-019-01042-8","article-title":"Utilization of DenseNet201 for diagnosis of breast abnormality","volume":"30","author":"Yu","year":"2019","journal-title":"Mach. Vis. Appl."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Yilmaz, F., Kose, O., and Demir, A. (2019, January 3\u20135). Comparison of two different deep learning architectures on breast cancer. Proceedings of the 2019 Medical Technologies Congress (TIPTEKNO), Izmir, Turkey.","DOI":"10.1109\/TIPTEKNO47231.2019.8972042"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1016\/j.media.2019.05.010","article-title":"Bach: Grand challenge on breast cancer histology images","volume":"56","author":"Aresta","year":"2019","journal-title":"Med. Image Anal."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Della Mea, V., Baroni, G.L., Pilutti, D., and Loreto, C.D. (2017). SlideJ: An ImageJ plugin for automated processing of whole slide images. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0180540"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017, January 4\u20139). Inception-v4, inception-resnet and the impact of residual connections on learning. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_45","unstructured":"Targ, S., Almeida, D., and Lyman, K. (2016). Resnet in resnet: Generalizing residual architectures. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.patcog.2019.01.006","article-title":"Wider or deeper: Revisiting the resnet model for visual recognition","volume":"90","author":"Wu","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_47","unstructured":"Smith, L.N. (2018). A disciplined approach to neural network hyper-parameters: Part 1\u2014Learning rate, batch size, momentum, and weight decay. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Nguyen, L.D., Lin, D., Lin, Z., and Cao, J. (2018, January 27\u201330). Deep CNNs for microscopic image classification by exploiting transfer learning and feature concatenation. Proceedings of the 2018 IEEE International Symposium on Circuits and Systems (ISCAS), Florence, Italy.","DOI":"10.1109\/ISCAS.2018.8351550"}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/6\/9\/82\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:05:22Z","timestamp":1760177122000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/6\/9\/82"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,23]]},"references-count":48,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["jimaging6090082"],"URL":"https:\/\/doi.org\/10.3390\/jimaging6090082","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,23]]}}}