{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T20:45:45Z","timestamp":1775249145891,"version":"3.50.1"},"reference-count":45,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2020,8,5]],"date-time":"2020-08-05T00:00:00Z","timestamp":1596585600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Government of the Basque Country, Spain","award":["MIFLUDAN Project"],"award-info":[{"award-number":["MIFLUDAN Project"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Breast cancer is one of the major public health issues and is considered a leading cause of cancer-related deaths among women worldwide. Its early diagnosis can effectively help in increasing the chances of survival rate. To this end, biopsy is usually followed as a gold standard approach in which tissues are collected for microscopic analysis. However, the histopathological analysis of breast cancer is non-trivial, labor-intensive, and may lead to a high degree of disagreement among pathologists. Therefore, an automatic diagnostic system could assist pathologists to improve the effectiveness of diagnostic processes. This paper presents an ensemble deep learning approach for the definite classification of non-carcinoma and carcinoma breast cancer histopathology images using our collected dataset. We trained four different models based on pre-trained VGG16 and VGG19 architectures. Initially, we followed 5-fold cross-validation operations on all the individual models, namely, fully-trained VGG16, fine-tuned VGG16, fully-trained VGG19, and fine-tuned VGG19 models. Then, we followed an ensemble strategy by taking the average of predicted probabilities and found that the ensemble of fine-tuned VGG16 and fine-tuned VGG19 performed competitive classification performance, especially on the carcinoma class. The ensemble of fine-tuned VGG16 and VGG19 models offered sensitivity of 97.73% for carcinoma class and overall accuracy of 95.29%. Also, it offered an F1 score of 95.29%. These experimental results demonstrated that our proposed deep learning approach is effective for the automatic classification of complex-natured histopathology images of breast cancer, more specifically for carcinoma images.<\/jats:p>","DOI":"10.3390\/s20164373","type":"journal-article","created":{"date-parts":[[2020,8,5]],"date-time":"2020-08-05T15:13:18Z","timestamp":1596640398000},"page":"4373","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":237,"title":["Breast Cancer Histopathology Image Classification Using an Ensemble of Deep Learning Models"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2514-1064","authenticated-orcid":false,"given":"Zabit","family":"Hameed","sequence":"first","affiliation":[{"name":"eVida Research Group, University of Deusto, 48007 Bilbao, Spain"}]},{"given":"Sofia","family":"Zahia","sequence":"additional","affiliation":[{"name":"eVida Research Group, University of Deusto, 48007 Bilbao, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9356-1186","authenticated-orcid":false,"given":"Begonya","family":"Garcia-Zapirain","sequence":"additional","affiliation":[{"name":"eVida Research Group, University of Deusto, 48007 Bilbao, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9946-7521","authenticated-orcid":false,"given":"Jos\u00e9","family":"Javier Aguirre","sequence":"additional","affiliation":[{"name":"Biokeralty Reseach Institute, 01510 Vitoria, Spain"},{"name":"Department of Pathological Anatomy, University Hospital of Araba, 01009 Vitoria, Spain"}]},{"given":"Ana","family":"Mar\u00eda Vanegas","sequence":"additional","affiliation":[{"name":"Clinica Colsanitas, Bogot\u00e1 110221, Colombia"}]}],"member":"1968","published-online":{"date-parts":[[2020,8,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1749","DOI":"10.1001\/jamaoncol.2019.2996","article-title":"Global, regional, and national cancer incidence, mortality, years of life lost, years lived with disability, and disability-adjusted life-years for 29 cancer groups, 1990 to 2017: A systematic analysis for the global burden of disease study","volume":"5","author":"Fitzmaurice","year":"2019","journal-title":"JAMA Oncol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"394","DOI":"10.3322\/caac.21492","article-title":"Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"68","author":"Bray","year":"2018","journal-title":"CA Cancer J. Clin."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1016\/j.molonc.2010.04.004","article-title":"Histological types of breast cancer: How special are they?","volume":"4","author":"Weigelt","year":"2010","journal-title":"Mol. Oncol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1016\/j.ejrad.2012.03.005","article-title":"Computed-aided diagnosis (CAD) in the detection of breast cancer","volume":"82","author":"Dromain","year":"2013","journal-title":"Eur. J. Radiol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wang, L. (2017). Early Diagnosis of Breast Cancer. Sensors, 17.","DOI":"10.3390\/s17071572"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1400","DOI":"10.1109\/TBME.2014.2303852","article-title":"Breast Cancer Histopathology Image Analysis: A Review","volume":"61","author":"Veta","year":"2014","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Dimitriou, N., Arandjelovi\u0107, O., and Caie, P.D. (2019). Deep Learning for Whole Slide Image Analysis: An Overview. Front. Med., 6.","DOI":"10.3389\/fmed.2019.00264"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1122","DOI":"10.1001\/jama.2015.1405","article-title":"Diagnostic Concordance among Pathologists Interpreting Breast Biopsy Specimens","volume":"313","author":"Elmore","year":"2015","journal-title":"JAMA"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.imu.2016.11.001","article-title":"Detection of breast cancer on digital histopathology images: Present status and future possibilities","volume":"8","author":"Aswathy","year":"2017","journal-title":"Inform. Med. Unlocked"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.trsl.2017.10.010","article-title":"Digital image analysis in breast pathology\u2014From image processing techniques to artificial intelligence","volume":"194","author":"Robertson","year":"2018","journal-title":"Transl. Res."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lowe, D. (1999, January 20\u201325). Object recognition from local scale-invariant features. Proceedings of the Seventh IEEE International Conference on Computer Vision, Corfu, Greece.","DOI":"10.1109\/ICCV.1999.790410"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Bay, H., Tuytelaars, T., and Gool, L.V. (2006, January 7\u201313). SURF: Speeded Up Robust Features. Proceedings of the Computer Vision\u2014ECCV 2006 Lecture Notes in Computer Science, Graz, Austria.","DOI":"10.1007\/11744023_32"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1109\/TPAMI.2002.1017623","article-title":"Multiresolution gray-scale and rotation invariant texture classification with local binary patterns","volume":"24","author":"Ojala","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"Lecun","year":"2015","journal-title":"Nature"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"101742","DOI":"10.1016\/j.artmed.2019.101742","article-title":"Pressure injury image analysis with machine learning techniques: A systematic review on previous and possible future methods","volume":"102","author":"Zahia","year":"2020","journal-title":"Artif. Intell. Med."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bour, A., Castillo-Olea, C., Garcia-Zapirain, B., and Zahia, S. (2019, January 10\u201312). Automatic colon polyp classification using Convolutional Neural Network: A Case Study at Basque Country. Proceedings of the 2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Louisville, KY, USA.","DOI":"10.1109\/ISSPIT47144.2019.9001816"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_19","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). ImageNet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_20","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very deep convolutional networks for large-scale image recognition. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1563","DOI":"10.1016\/j.compbiomed.2013.08.003","article-title":"Computer-aided diagnosis of breast cancer based on fine needle biopsy microscopic images","volume":"43","author":"Kowal","year":"2013","journal-title":"Comput. Biol. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2169","DOI":"10.1109\/TMI.2013.2275151","article-title":"Computer-Aided Breast Cancer Diagnosis Based on the Analysis of Cytological Images of Fine Needle Biopsies","volume":"32","author":"Filipczuk","year":"2013","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1405","DOI":"10.1007\/s00138-012-0459-8","article-title":"Breast cancer diagnosis from biopsy images with highly reliable random subspace classifier ensembles","volume":"24","author":"Zhang","year":"2012","journal-title":"Mach. Vis. Appl."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","article-title":"Representation Learning: A Review and New Perspectives","volume":"35","author":"Bengio","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1455","DOI":"10.1109\/TBME.2015.2496264","article-title":"A Dataset for Breast Cancer Histopathological Image Classification","volume":"63","author":"Spanhol","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Spanhol, F.A., Oliveira, L.S., Petitjean, C., and Heutte, L. (2016, January 24\u201329). Breast cancer histopathological image classification using Convolutional Neural Networks. Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada.","DOI":"10.1109\/IJCNN.2016.7727519"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Bayramoglu, N., Kannala, J., and Heikkila, J. (2016, January 4\u20138). Deep learning for magnification independent breast cancer histopathology image classification. Proceedings of the 2016 23rd International Conference on Pattern Recognition (ICPR), Canc\u00fan, Mexico.","DOI":"10.1109\/ICPR.2016.7900002"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ara\u00fajo, T., Aresta, G., Castro, E., Rouco, J., Aguiar, P., Eloy, C., Pol\u00f3nia, A., and Campilho, A. (2017). Classification of breast cancer histology images using Convolutional Neural Networks. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0177544"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.ymeth.2019.06.014","article-title":"Breast cancer histopathological image classification using a hybrid deep neural network","volume":"173","author":"Yan","year":"2020","journal-title":"Methods"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-017-17204-5","article-title":"QuPath: Open source software for digital pathology image analysis","volume":"7","author":"Bankhead","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Macenko, M., Niethammer, M., Marron, J.S., Borland, D., Woosley, J.T., Guan, X., Schmitt, C., and Thomas, N.E. (July, January 28). A method for normalizing histology slides for quantitative analysis. Proceedings of the 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Boston, MA, USA.","DOI":"10.1109\/ISBI.2009.5193250"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Bianconi, F., Kather, J.N., and Reyes-Aldasoro, C.C. (2019). Evaluation of Colour Pre-processing on Patch-Based Classification of H&E-Stained Images. European Congress on Digital Pathology, Springer.","DOI":"10.1007\/978-3-030-23937-4_7"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Yao, H., Zhang, X., Zhou, X., and Liu, S. (2019). Parallel Structure Deep Neural Network Using CNN and RNN with an Attention Mechanism for Breast Cancer Histology Image Classification. Cancers, 11.","DOI":"10.3390\/cancers11121901"},{"key":"ref_36","unstructured":"Chollet, F. (2020, January 24). Keras: Deep Learning Library. Available online: https:\/\/keras.io\/api\/preprocessing\/image\/."},{"key":"ref_37","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2017). Deep Learning, The MIT Press."},{"key":"ref_38","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"80","DOI":"10.3389\/fgene.2019.00080","article-title":"Deep Learning Based Analysis of Histopathological Images of Breast Cancer","volume":"10","author":"Xie","year":"2019","journal-title":"Front. Genet."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"24680","DOI":"10.1109\/ACCESS.2018.2831280","article-title":"Classification of Breast Cancer Based on Histology Images Using Convolutional Neural Networks","volume":"6","author":"Bardou","year":"2018","journal-title":"IEEE Access"},{"key":"ref_41","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_42","first-page":"1","article-title":"Breast Cancer Multi-classification from Histopathological Images with Structured Deep Learning Model","volume":"7","author":"Han","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2362108","DOI":"10.1155\/2018\/2362108","article-title":"Histopathological Breast Cancer Image Classification by Deep Neural Network Techniques Guided by Local Clustering","volume":"2018","author":"Nahid","year":"2018","journal-title":"BioMed Res. Int."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1007\/s13755-018-0057-x","article-title":"Transfer learning based histopathologic image classification for breast cancer detection","volume":"6","author":"Deniz","year":"2018","journal-title":"Health Inf. Sci. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.neucom.2019.09.044","article-title":"BreakHis based breast cancer automatic diagnosis using deep learning: Taxonomy, survey and insights","volume":"375","author":"Benhammou","year":"2020","journal-title":"Neurocomputing"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/16\/4373\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:54:35Z","timestamp":1760176475000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/16\/4373"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,5]]},"references-count":45,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2020,8]]}},"alternative-id":["s20164373"],"URL":"https:\/\/doi.org\/10.3390\/s20164373","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,5]]}}}