{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T07:00:56Z","timestamp":1781593256547,"version":"3.54.5"},"reference-count":49,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2023,6,28]],"date-time":"2023-06-28T00:00:00Z","timestamp":1687910400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Piano Nazionale per gli investimenti Complementari al PNRR","award":["B53C22006460001"],"award-info":[{"award-number":["B53C22006460001"]}]},{"name":"Piano Nazionale per gli investimenti Complementari al PNRR","award":["n. 1511 del 30\/09\/2022"],"award-info":[{"award-number":["n. 1511 del 30\/09\/2022"]}]},{"name":"Decreto Direttoriale (Direzione Generale Ricerca) Ministero Universit\u00e0 e Ricerca","award":["B53C22006460001"],"award-info":[{"award-number":["B53C22006460001"]}]},{"name":"Decreto Direttoriale (Direzione Generale Ricerca) Ministero Universit\u00e0 e Ricerca","award":["n. 1511 del 30\/09\/2022"],"award-info":[{"award-number":["n. 1511 del 30\/09\/2022"]}]},{"name":"University of Palermo FFR (Fondo Finalizzato alla ricerca di Ateneo)","award":["B53C22006460001"],"award-info":[{"award-number":["B53C22006460001"]}]},{"name":"University of Palermo FFR (Fondo Finalizzato alla ricerca di Ateneo)","award":["n. 1511 del 30\/09\/2022"],"award-info":[{"award-number":["n. 1511 del 30\/09\/2022"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The application of machine learning techniques to histopathology images enables advances in the field, providing valuable tools that can speed up and facilitate the diagnosis process. The classification of these images is a relevant aid for physicians who have to process a large number of images in long and repetitive tasks. This work proposes the adoption of metric learning that, beyond the task of classifying images, can provide additional information able to support the decision of the classification system. In particular, triplet networks have been employed to create a representation in the embedding space that gathers together images of the same class while tending to separate images with different labels. The obtained representation shows an evident separation of the classes with the possibility of evaluating the similarity and the dissimilarity among input images according to distance criteria. The model has been tested on the BreakHis dataset, a reference and largely used dataset that collects breast cancer images with eight pathology labels and four magnification levels. Our proposed classification model achieves relevant performance on the patient level, with the advantage of providing interpretable information for the obtained results, which represent a specific feature missed by the all the recent methodologies proposed for the same purpose.<\/jats:p>","DOI":"10.3390\/s23136003","type":"journal-article","created":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T01:43:13Z","timestamp":1688002993000},"page":"6003","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Metric Learning in Histopathological Image Classification: Opening the Black Box"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4004-9338","authenticated-orcid":false,"given":"Domenico","family":"Amato","sequence":"first","affiliation":[{"name":"Department of Mathematics and Computer Science, University of Palermo, 90123 Palermo, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0999-6345","authenticated-orcid":false,"given":"Salvatore","family":"Calderaro","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Computer Science, University of Palermo, 90123 Palermo, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1602-0693","authenticated-orcid":false,"given":"Giosu\u00e9","family":"Lo Bosco","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Computer Science, University of Palermo, 90123 Palermo, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5007-6925","authenticated-orcid":false,"given":"Riccardo","family":"Rizzo","sequence":"additional","affiliation":[{"name":"Institute for High-Performance Computing and Networking, National Research Council of Italy, 90146 Palermo, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2502-0062","authenticated-orcid":false,"given":"Filippo","family":"Vella","sequence":"additional","affiliation":[{"name":"Institute for High-Performance Computing and Networking, National Research Council of Italy, 90146 Palermo, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1038\/s41591-021-01343-4","article-title":"Deep learning in histopathology: The path to the clinic","volume":"27","author":"Litjens","year":"2021","journal-title":"Nat. Med."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"264","DOI":"10.3389\/fmed.2019.00264","article-title":"Deep Learning for Whole Slide Image Analysis: An Overview","volume":"6","author":"Dimitriou","year":"2019","journal-title":"Front. Med."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1301","DOI":"10.1038\/s41591-019-0508-1","article-title":"Clinical-Grade Computational Pathology Using Weakly Supervised Deep Learning on Whole Slide Images","volume":"25","author":"Campanella","year":"2019","journal-title":"Nat. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1455","DOI":"10.1109\/TBME.2015.2496264","article-title":"A Dataset for Breast Cancer Histopathological Image Classification","volume":"63","author":"Spanhol","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-Based Learning Applied to Document Recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"e745","DOI":"10.1016\/S2589-7500(21)00208-9","article-title":"The false hope of current approaches to explainable artificial intelligence in health care","volume":"3","author":"Ghassemi","year":"2021","journal-title":"Lancet Digit. Health"},{"key":"ref_10","first-page":"18","article-title":"Fuzzy Clustering of Histopathological Images Using Deep Learning Embeddings","volume":"3074","author":"Calderaro","year":"2021","journal-title":"CEUR Workshop Proc."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Calderaro, S., Lo Bosco, G., Rizzo, R., and Vella, F. (2022, January 5\u20137). Deep Metric Learning for Histopathological Image Classification. Proceedings of the 2022 IEEE Eighth International Conference on Multimedia Big Data (BigMM), Naples, Italy.","DOI":"10.1109\/BigMM55396.2022.00016"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Calderaro, S., Lo Bosco, G., Vella, F., and Rizzo, R. (2023, January 12\u201314). Breast Cancer Histologic Grade Identification by Graph Neural Network Embeddings. Proceedings of the 2023 10th International Work-Conference on Bioinformatics and Biomedical Engineering (IWBBIO), Gran Canaria, Spain.","DOI":"10.1007\/978-3-031-34960-7_20"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Calderaro, S., Lo Bosco, G., Rizzo, R., and Vella, F. (2022, January 19\u201321). Deep Metric Learning for Transparent Classification of COVID-19 X-Ray Images. Proceedings of the 2022 16TH International Conference on Signal Image Technology & Internet Based Systems (SITIS), Dijon, France.","DOI":"10.1109\/SITIS57111.2022.00052"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"11579","DOI":"10.1038\/s41598-021-90444-8","article-title":"A Generalized Deep Learning Framework for Whole-Slide Image Segmentation and Analysis","volume":"11","author":"Khened","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.artmed.2018.04.005","article-title":"MuDeRN: Multi-Category Classification of Breast Histopathological Image Using Deep Residual Networks","volume":"88","author":"Gandomkar","year":"2018","journal-title":"Artif. Intell. Med."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Bayramoglu, N., Kannala, J., and Heikkil\u00e4, J. (2016, January 4\u20138). Deep learning for magnification independent breast cancer histopathology image classification. Proceedings of the 2016 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico.","DOI":"10.1109\/ICPR.2016.7900002"},{"key":"ref_17","unstructured":"Sun, Y., Huang, X., Wang, Y., Zhou, H., and Zhang, Q. (2021). Magnification-independent Histopathological Image Classification with Similarity-based Multi-scale Embeddings. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Gupta, V., and Bhavsar, A. (2017, January 21\u201326). Breast Cancer Histopathological Image Classification: Is Magnification Important?. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.107"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Spanhol, F.A., Oliveira, L.S., Petitjean, C., and Heutte, L. (2016, January 24\u201329). Breast cancer histopathological image classification using Convolutional Neural Networks. Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada.","DOI":"10.1109\/IJCNN.2016.7727519"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Spanhol, F.A., Oliveira, L.S., Cavalin, P.R., Petitjean, C., and Heutte, L. (2017, January 5\u20138). Deep features for breast cancer histopathological image classification. Proceedings of the 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Banff, AL, Canada.","DOI":"10.1109\/SMC.2017.8122889"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Benhammou, Y., Tabik, S., Achchab, B., and Herrera, F. (2018, January 2\u20135). A First Study Exploring the Performance of the State-of-the Art CNN Model in the Problem of Breast Cancer. Proceedings of the International Conference on Learning and Optimization Algorithms: Theory and Applications, Rabat, Morocco.","DOI":"10.1145\/3230905.3230940"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2015). Rethinking the Inception Architecture for Computer Vision. arXiv.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Song, Y., Zou, J.J., Chang, H., and Cai, W. (2017, January 18\u201321). Adapting fisher vectors for histopathology image classification. Proceedings of the 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), Melbourne, Australia.","DOI":"10.1109\/ISBI.2017.7950592"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1007\/s11263-013-0636-x","article-title":"Image Classification with the Fisher Vector: Theory and Practice","volume":"105","author":"Sanchez","year":"2013","journal-title":"Int J. Comput. Vision"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., and Duchesne, S. (2017). Supervised Intra-Embedding of Fisher Vectors for Histopathology Image Classification, Proceedings of the Medical Image Computing and Computer Assisted Intervention-MICCAI 2017, Quebec City, QC, Canada, 10\u201314 June 2017, Springer International Publishing. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-66179-7"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.eswa.2018.09.049","article-title":"Multiple instance learning for histopathological breast cancer image classification","volume":"117","author":"Sudharshan","year":"2019","journal-title":"Expert Syst. Appl."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Kumar, K., and Rao, A.C.S. (2018, January 15\u201317). Breast cancer classification of image using convolutional neural network. Proceedings of the 2018 4th International Conference on Recent Advances in Information Technology (RAIT), Dhanbad, India.","DOI":"10.1109\/RAIT.2018.8389034"},{"key":"ref_28","unstructured":"Wei, B., Han, Z., He, X., and Yin, Y. (2017, January 28\u201330). Deep learning model based breast cancer histopathological image classification. Proceedings of the 2017 IEEE 2nd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA), Chengdu, China."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"24680","DOI":"10.1109\/ACCESS.2018.2831280","article-title":"Classification of Breast Cancer Based on Histology Images Using Convolutional Neural Networks","volume":"6","author":"Bardou","year":"2018","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Das, K., Karri, S.P.K., Guha Roy, A., Chatterjee, J., and Sheet, D. (2017, January 18\u201321). Classifying histopathology whole-slides using fusion of decisions from deep convolutional network on a collection of random multi-views at multi-magnification. Proceedings of the 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), Melbourne, Australia.","DOI":"10.1109\/ISBI.2017.7950690"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"De Pietro, G., Gallo, L., Howlett, R.J., and Jain, L.C. (2018). Intelligent Interactive Multimedia Systems and Services, Springer International Publishing. Smart Innovation, Systems and Technologies.","DOI":"10.1007\/978-3-319-92231-7"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2068","DOI":"10.4066\/biomedicalresearch.29-17-3903","article-title":"Histopathological breast-image classification with restricted Boltzmann machine along with backpropagation","volume":"29","author":"Nahid","year":"2018","journal-title":"Biomed. Res."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Cardoso, M.J., Arbel, T., Carneiro, G., Syeda-Mahmood, T., Tavares, J.M.R., Moradi, M., Bradley, A., Greenspan, H., Papa, J.P., and Madabhushi, A. (2017). Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer International Publishing. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-67558-9"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Nejad, E.M., Affendey, L.S., Latip, R.B., and Bin Ishak, I. (2017, January 26\u201328). Classification of Histopathology Images of Breast into Benign and Malignant using a Single-layer Convolutional Neural Network. Proceedings of the International Conference on Imaging, Signal Processing and Communication, Penang, Malaysia.","DOI":"10.1145\/3132300.3132331"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Halacli, B., Yildirim, M., Kaya, E.K., Ulusoydan, E., Ersoy, E.O., and Topeli, A. (2023). Chronic critical illness in critically ill COVID-19 patients. Chronic Illn.","DOI":"10.1177\/17423953231161333"},{"key":"ref_36","first-page":"2654","article-title":"Do deep nets really need to be deep?","volume":"Volume 27","author":"Ba","year":"2014","journal-title":"Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_38","unstructured":"Bromley, J., Guyon, I., LeCun, Y., S\u00e4ckinger, E., and Shah, R. (1993). Advances in Neural Information Processing Systems, Morgan Kaufmann."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Feragen, A., Pelillo, M., and Loog, M. (2015). Similarity-Based Pattern Recognition, Springer International Publishing. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-24261-3"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kaya, M., and Sakir, B.H. (2019). Deep Metric Learning: A Survey. Symmetry, 11.","DOI":"10.3390\/sym11091066"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1016\/j.cag.2021.09.002","article-title":"A survey of visual analytics for Explainable Artificial Intelligence methods","volume":"102","author":"Alicioglu","year":"2022","journal-title":"Comput. Graph."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"McInnes, L., and Healy, J. (2018). UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv.","DOI":"10.21105\/joss.00861"},{"key":"ref_43","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2022, June 01). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Available online: tensorflow.org."},{"key":"ref_44","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Tucker, A., Henriques Abreu, P., Cardoso, J., Pereira Rodrigues, P., and Ria\u00f1o, D. (2021). Artificial Intelligence in Medicine, Springer International Publishing.","DOI":"10.1007\/978-3-030-77211-6"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1186\/1746-1596-7-22","article-title":"Identification of tumor epithelium and stroma in tissue microarrays using texture analysis","volume":"7","author":"Linder","year":"2012","journal-title":"Diagn. Pathol."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"648","DOI":"10.1016\/j.ejca.2008.10.036","article-title":"Xanthine oxidoreductase\u2014Clinical significance in colorectal cancer and in vitro expression of the protein in human colon cancer cells","volume":"45","author":"Linder","year":"2009","journal-title":"Eur. J. Cancer"},{"key":"ref_48","unstructured":"DARPA (2022, June 01). Broad Agency Announcement, Explainable Artificial Intelligence (XAI).DARPA-BAA-16-53. Available online: https:\/\/www.darpa.mil\/attachments\/DARPA-BAA-16-53.pdf."},{"key":"ref_49","unstructured":"Molnar, C. (2022, June 01). Interpretable Machine Learning; Lulu. com. Available online: https:\/\/christophm.github.io\/interpretable-ml-book."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/13\/6003\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:02:53Z","timestamp":1760126573000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/13\/6003"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,28]]},"references-count":49,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2023,7]]}},"alternative-id":["s23136003"],"URL":"https:\/\/doi.org\/10.3390\/s23136003","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,28]]}}}