{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T10:08:40Z","timestamp":1792750120398,"version":"4.5.0"},"reference-count":82,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,1,6]],"date-time":"2024-01-06T00:00:00Z","timestamp":1704499200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,1,6]],"date-time":"2024-01-06T00:00:00Z","timestamp":1704499200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>There is a wide application of deep learning technique to unimodal medical image analysis with significant classification accuracy performance observed. However, real-world diagnosis of some chronic diseases such as breast cancer often require multimodal data streams with different modalities of visual and textual content. Mammography, magnetic resonance imaging (MRI) and image-guided breast biopsy represent a few of multimodal visual streams considered by physicians in isolating cases of breast cancer. Unfortunately, most studies applying deep learning techniques to solving classification problems in digital breast images have often narrowed their study to unimodal samples. This is understood considering the challenging nature of multimodal image abnormality classification where the fusion of high dimension heterogeneous features learned needs to be projected into a common representation space. This paper presents a novel deep learning approach combining a dual\/twin convolutional neural network (TwinCNN) framework to address the challenge of breast cancer image classification from multi-modalities. First, modality-based feature learning was achieved by extracting both low and high levels features using the networks embedded with TwinCNN. Secondly, to address the notorious problem of high dimensionality associated with the extracted features, binary optimization method is adapted to effectively eliminate non-discriminant features in the search space. Furthermore, a novel method for feature fusion is applied to computationally leverage the ground-truth and predicted labels for each sample to enable multimodality classification. To evaluate the proposed method, digital mammography images and digital histopathology breast biopsy samples from benchmark datasets namely MIAS and BreakHis respectively. Experimental results obtained showed that the classification accuracy and area under the curve (AUC) for the single modalities yielded 0.755 and 0.861871 for histology, and 0.791 and 0.638 for mammography. Furthermore, the study investigated classification accuracy resulting from the fused feature method, and the result obtained showed that 0.977, 0.913, and 0.667 for histology, mammography, and multimodality respectively. The findings from the study confirmed that multimodal image classification based on combination of image features and predicted label improves performance. In addition, the contribution of the study shows that feature dimensionality reduction based on binary optimizer supports the elimination of non-discriminant features capable of bottle-necking the classifier.<\/jats:p>","DOI":"10.1038\/s41598-024-51329-8","type":"journal-article","created":{"date-parts":[[2024,1,6]],"date-time":"2024-01-06T01:01:45Z","timestamp":1704502905000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":69,"title":["A twin convolutional neural network with hybrid binary optimizer for multimodal breast cancer digital image classification"],"prefix":"10.1038","volume":"14","author":[{"given":"Olaide N.","family":"Oyelade","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric Aghiomesi","family":"Irunokhai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hui","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,1,6]]},"reference":[{"key":"51329_CR1","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1186\/s43055-020-00175-5","volume":"51","author":"S Iranmakani","year":"2020","unstructured":"Iranmakani, S., Mortezazadeh, T., Sajadian, F., Ghaziani, A. G. D. K. M. F. & Musa, A. E. A review of various modalities in breast imaging: Technical aspects and clinical outcomes. Egypt. J. Radiol. Nucl. Med. 51, 57. https:\/\/doi.org\/10.1186\/s43055-020-00175-5 (2020).","journal-title":"Egypt. J. Radiol. Nucl. Med."},{"key":"51329_CR2","doi-asserted-by":"publisher","first-page":"664","DOI":"10.1126\/scitranslmed.abo4802","volume":"14","author":"J Witowski","year":"2022","unstructured":"Witowski, J. et al. Improving breast cancer diagnostics with deep learning for MRI. Sci. Transl. Med. 14, 664 (2022).","journal-title":"Sci. Transl. Med."},{"key":"51329_CR3","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1186\/s13244-022-01259-8","volume":"13","author":"Y Gu","year":"2022","unstructured":"Gu, Y. et al. Deep learning based on ultrasound images assists breast lesion diagnosis in China: A multicenter diagnostic study. Insights Imaging 13, 124 (2022).","journal-title":"Insights Imaging"},{"key":"51329_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-022-22933-3","volume":"12","author":"ON Oyelade","year":"2022","unstructured":"Oyelade, O. N. & Ezugwu, A.E.-S. Immunity-based Ebola optimization search algorithm (IEOSA) for minimization of feature extraction with reduction in digital mammography using CNN models. Sci. Rep. 12, 1 (2022).","journal-title":"Sci. Rep."},{"key":"51329_CR5","doi-asserted-by":"publisher","first-page":"102366","DOI":"10.1016\/j.bspc.2020.102366","volume":"65","author":"ON Oyelade","year":"2021","unstructured":"Oyelade, O. N. & Ezugwu, A. E. A deep learning model using data augmentation for detection of architectural distortion in whole and patches of images. Biomed. Signal Process. Control 65, 102366 (2021).","journal-title":"Biomed. Signal Process. Control"},{"issue":"13","key":"51329_CR6","doi-asserted-by":"publisher","first-page":"e6914","DOI":"10.1002\/cpe.6914","volume":"34","author":"ON Oyelade","year":"2022","unstructured":"Oyelade, O. N. & Ezugwu, A. E. A comparative performance study of random-grid model for hyperparameters selection in detection of abnormalities in digital breast images. Concurr. Comput. Pract. Exp. 34(13), e6914 (2022).","journal-title":"Concurr. Comput. Pract. Exp."},{"key":"51329_CR7","first-page":"56","volume":"2022","author":"ON Oyelade","year":"2022","unstructured":"Oyelade, O. N. & Ezugwu, A. E. A novel wavelet decomposition and wavelet transformation convolutional neural network with data augmentation for breast cancer detection using digital mammogram. Sci. Rep. 2022, 56 (2022).","journal-title":"Sci. Rep."},{"key":"51329_CR8","doi-asserted-by":"publisher","first-page":"943415","DOI":"10.3389\/fonc.2022.943415","volume":"12","author":"Y Chen","year":"2022","unstructured":"Chen, Y. et al. A deep learning model based on dynamic contrast-enhanced magnetic resonance imaging enables accurate prediction of benign and malignant breast lessons. Front. Oncol. 12, 943415 (2022).","journal-title":"Front. Oncol."},{"key":"51329_CR9","doi-asserted-by":"publisher","first-page":"2021","DOI":"10.1016\/j.media.2021.102049","volume":"71","author":"J Bai","year":"2021","unstructured":"Bai, J., Posner, R., Wang, T., Yang, C. & Nabavi, S. Applying deep learning in digital breast tomosynthesis for automatic breast cancer detection: A review. Med. Image Anal. 71, 2021 (2021).","journal-title":"Med. Image Anal."},{"issue":"2","key":"51329_CR10","doi-asserted-by":"publisher","first-page":"367","DOI":"10.3390\/cancers14020367","volume":"14","author":"Y-J Mao","year":"2022","unstructured":"Mao, Y.-J. et al. Breast tumour classification using ultrasound elastography with machine learning: A systematic scoping review. Cancers (Basel) 14(2), 367 (2022).","journal-title":"Cancers (Basel)"},{"issue":"12","key":"51329_CR11","first-page":"1055","volume":"10","author":"T Fujioka","year":"2020","unstructured":"Fujioka, T. et al. The utility of deep learning in breast ultrasonic imaging: A review. Diagn. (Basel) 10(12), 1055 (2020).","journal-title":"Diagn. (Basel)"},{"issue":"2016","key":"51329_CR12","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1016\/j.ultras.2016.08.004","volume":"72","author":"Q Zhang","year":"2016","unstructured":"Zhang, Q. et al. Deep learning based classification of breast tumors with shear-wave elastography. Ultrasonics 72(2016), 150\u2013157 (2016).","journal-title":"Ultrasonics"},{"key":"51329_CR13","doi-asserted-by":"publisher","first-page":"110608","DOI":"10.1016\/j.ejrad.2022.110608","volume":"157","author":"E Ji","year":"2022","unstructured":"Ji, E., Lee, F., Chang, Y.-W., Sung, J. K. & Thomas, B. Feasibility of deep learning k-space-to-image reconstruction for diffusion weighted imaging in patients with breast cancers: Focus on image quality and reduced scan time. Eur. J. Radiol. 157, 110608 (2022).","journal-title":"Eur. J. Radiol."},{"key":"51329_CR14","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1038\/s41523-022-00488-w","volume":"8","author":"C Mercan","year":"2022","unstructured":"Mercan, C. et al. Deep learning for fully-automated nuclear pleomorphism scoring in breast cancer. NPJ Breast Cancer 8, 120 (2022).","journal-title":"NPJ Breast Cancer"},{"key":"51329_CR15","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1186\/s40658-021-00426-y","volume":"8","author":"M Decuyper","year":"2021","unstructured":"Decuyper, M., Maebe, J., Van-Holen, R. & Vandenberghe, S. Artificial intelligence with deep learning in nuclear medicine and radiology. EJNMMI Phys. 8, 81 (2021).","journal-title":"EJNMMI Phys."},{"key":"51329_CR16","doi-asserted-by":"publisher","first-page":"105943","DOI":"10.1016\/j.compbiomed.2022.105943","volume":"149","author":"ON Oyelade","year":"2022","unstructured":"Oyelade, O. N., Ezugwu, A. E., Venter, H. S., Mirjalili, S. & Gandomi, A. H. Abnormality classification and localization using dual-branch whole-region-based CNN model with histopathological images. Comput. Biol. Med. 149, 105943 (2022).","journal-title":"Comput. Biol. Med."},{"key":"51329_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-021-98978-7","volume":"11","author":"ON Oyelade","year":"2021","unstructured":"Oyelade, O. N. & Ezugwu, A. E. A bioinspired neural architecture search based convolutional neural network for breast cancer detection using histopathology images. Sci. Rep. 11, 1 (2021).","journal-title":"Sci. Rep."},{"key":"51329_CR18","doi-asserted-by":"publisher","first-page":"8","DOI":"10.3390\/s22082988","volume":"22","author":"A Vulli","year":"2022","unstructured":"Vulli, A. et al. Fine-tuned DenseNet-169 for breast cancer metastasis prediction using FastAI and 1-Cycle policy. Sensors 22, 8 (2022).","journal-title":"Sensors"},{"issue":"4","key":"51329_CR19","doi-asserted-by":"publisher","first-page":"1636","DOI":"10.1364\/BOE.486292","volume":"14","author":"M Zhang","year":"2023","unstructured":"Zhang, M., Xue, M., Li, S., Zou, Y. & Zhu, Q. Fusion deep learning approach combining diffuse optical tomography and ultrasound for improving breast cancer classification. Biomed. Opt. Express 14(4), 1636\u20131646 (2023).","journal-title":"Biomed. Opt. Express"},{"key":"51329_CR20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s42492-019-0012-y","volume":"2","author":"XW Qiwen-Xu","year":"2019","unstructured":"Qiwen-Xu, X. W. & Jiang, H. Convolutional neural network for breast cancer diagnosis using diffuse optical tomography. Vis. Comput. Ind. Biomed. Art 2, 1 (2019).","journal-title":"Vis. Comput. Ind. Biomed. Art"},{"key":"51329_CR21","doi-asserted-by":"crossref","unstructured":"Jeremic, A. Detection of breast cancer using microwave imaging and machine learning-finite element method inverse models. In 2021 Photonics & Electromagnetics Research Symposium (PIERS), Hangzhou (2021).","DOI":"10.1109\/PIERS53385.2021.9695005"},{"issue":"1","key":"51329_CR22","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1109\/5.554205","volume":"85","author":"DL Hall","year":"1997","unstructured":"Hall, D. L. & Llinas, J. An introduction to multisensor data fusion. Proc. IEEE 85(1), 6\u201323 (1997).","journal-title":"Proc. IEEE"},{"issue":"9","key":"51329_CR23","doi-asserted-by":"publisher","first-page":"1573","DOI":"10.7863\/ultra.32.9.1573","volume":"32","author":"EY Chae","year":"2013","unstructured":"Chae, E. Y., Kim, H. H., Cha, J. H., Shin, H. J. & Kim, H. Evaluation of screening whole-breast sonography as a supplemental tool in conjunction with mammography in women with dense breasts. J. Ultrasound Med. 32(9), 1573\u20131578 (2013).","journal-title":"J. Ultrasound Med."},{"issue":"4","key":"51329_CR24","doi-asserted-by":"publisher","first-page":"171","DOI":"10.5306\/wjco.v2.i4.171","volume":"2","author":"SV Sree","year":"2011","unstructured":"Sree, S. V., Ng, E.Y.-K., Acharya, R. U. & Faust, O. Breast imaging: A survey. World J. Clin. Oncol. 2(4), 171\u2013178 (2011).","journal-title":"World J. Clin. Oncol."},{"issue":"119","key":"51329_CR25","first-page":"1","volume":"20","author":"X Jia","year":"2020","unstructured":"Jia, X., Liu, Y., Yang, Z. & Yang, D. Multi-modality self-attention aware deep network for 3D biomedical segmentation. BMC Med. Inf. Dec. Making 20(119), 1\u20137 (2020).","journal-title":"BMC Med. Inf. Dec. Making"},{"issue":"163","key":"51329_CR26","first-page":"1","volume":"11","author":"GM Dimitri","year":"2022","unstructured":"Dimitri, G. M. A short survey on deep learning for multimodal integration: Applications, future perspectives and challenges. Computers 11(163), 1\u201314 (2022).","journal-title":"Computers"},{"issue":"2022","key":"51329_CR27","doi-asserted-by":"publisher","first-page":"146","DOI":"10.1016\/j.inffus.2022.07.017","volume":"88","author":"GM Dimitri","year":"2022","unstructured":"Dimitri, G. M. et al. Multimodal image fusion via deep generative models. Inf. Fusion 88(2022), 146\u2013160 (2022).","journal-title":"Inf. Fusion"},{"issue":"2","key":"51329_CR28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1093\/bib\/bbab569","volume":"23","author":"SR Stahlschmidt","year":"2022","unstructured":"Stahlschmidt, S. R., Ulfenborg, B. & Synnergren, J. Multimodal deep learning for biomedical data fusion: A review. Brief. Bioinf. 23(2), 1\u201315 (2022).","journal-title":"Brief. Bioinf."},{"issue":"2023","key":"51329_CR29","first-page":"1","volume":"27","author":"AS Vijendran","year":"2023","unstructured":"Vijendran, A. S. & Ramasamy, K. Optimal segmentation and fusion of multi-modal brain images using clustering based deep learning algorithm. Meas. Sens. 27(2023), 1\u201312 (2023).","journal-title":"Meas. Sens."},{"issue":"5","key":"51329_CR30","doi-asserted-by":"publisher","first-page":"8708","DOI":"10.3934\/mbe.2023382","volume":"20","author":"K Yan","year":"2023","unstructured":"Yan, K., Li, T., Marques, J. A. L., Gao, J. & Fong, S. J. A review on multimodal machine learning in medical diagnostics. Math. Biosci. Eng. 20(5), 8708\u20138726 (2023).","journal-title":"Math. Biosci. Eng."},{"key":"51329_CR31","doi-asserted-by":"publisher","first-page":"2939","DOI":"10.1007\/s00371-021-02166-7","volume":"38","author":"K Bayoudh","year":"2022","unstructured":"Bayoudh, K., Knani, R., Hamdaoui, F. & Mtibaa, A. A survey on deep multimodal learning for computer vision: Advances, trends, applications, and datasets. Vis. Comput. 38, 2939\u20132970 (2022).","journal-title":"Vis. Comput."},{"key":"51329_CR32","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1371\/journal.pone.0282812","volume":"18","author":"ON Oyelade","year":"2023","unstructured":"Oyelade, O. N. & Ezugwu, O.J.A.A.A.E.-S. Evolutionary binary feature selection using adaptive ebola optimization search algorithm for high-dimensional datasets. PLOS One 18, 3 (2023).","journal-title":"PLOS One"},{"key":"51329_CR33","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1186\/s40537-023-00749-w","volume":"10","author":"S Kayikci","year":"2023","unstructured":"Kayikci, S. & Khoshgoftaar, T. M. Breast cancer prediction using gated attentive multimodal deep learning. J. Big Data 10, 62 (2023).","journal-title":"J. Big Data"},{"issue":"10","key":"51329_CR34","doi-asserted-by":"publisher","first-page":"2704","DOI":"10.3390\/cancers15102704","volume":"15","author":"M Bobowicz","year":"2023","unstructured":"Bobowicz, M. et al. Attention-based deep learning system for classification of breast lesions\u2014multimodal, weakly supervised approach. Cancers 15(10), 2704 (2023).","journal-title":"Cancers"},{"key":"51329_CR35","first-page":"6","volume":"8","author":"S Misra","year":"2022","unstructured":"Misra, S. et al. Deep learning-based multimodal fusion network for segmentation and classification of breast cancers using B-mode and elastography ultrasound images. Bioeng. Transl. Med. 8, 6 (2022).","journal-title":"Bioeng. Transl. Med."},{"key":"51329_CR36","first-page":"2022","volume":"3714422","author":"MM Althobaiti","year":"2022","unstructured":"Althobaiti, M. M. et al. Deep transfer learning-based breast cancer detection and classification model using photoacoustic multimodal images. Biomed. Res. Int. 3714422, 2022 (2022).","journal-title":"Biomed. Res. Int."},{"issue":"48","key":"51329_CR37","first-page":"1","volume":"3","author":"MD Li","year":"2020","unstructured":"Li, M. D. et al. Siamese neural networks for continuous disease severity evaluation and change detection in medical imaging. Dig. Med. 3(48), 1\u20139 (2020).","journal-title":"Dig. Med."},{"issue":"55","key":"51329_CR38","first-page":"1","volume":"23","author":"Q Li","year":"2023","unstructured":"Li, Q. & He, S. Similarity matching of medical question based on Siamese network. BMC Med. Inf. Dec. Mak. 23(55), 1\u201313 (2023).","journal-title":"BMC Med. Inf. Dec. Mak."},{"issue":"2023","key":"51329_CR39","first-page":"1","volume":"37","author":"TR Ornob","year":"2022","unstructured":"Ornob, T. R., Roy, G. & Hassan, E. CovidExpert: A Triplet Siamese Neural Network framework for the detection of COVID-19. Inf. Med. Unlocked 37(2023), 1\u201314 (2022).","journal-title":"Inf. Med. Unlocked"},{"key":"51329_CR40","first-page":"2022","volume":"1\u201326","author":"P Ranjan","year":"2022","unstructured":"Ranjan, P. & Girdhar, A. Deep siamese network with handcrafted feature extraction for hyperspectral image classifcation. Multimedia Tools Appl. 1\u201326, 2022 (2022).","journal-title":"Multimedia Tools Appl."},{"key":"51329_CR41","doi-asserted-by":"publisher","first-page":"1655","DOI":"10.1002\/ima.22543","volume":"31","author":"S Deepak","year":"2021","unstructured":"Deepak, S. & Ameer, P. M. Brain tumour classification using siamese neural network and neighbourhood analysis in embedded feature space. Int. J. Imaging Syst. Technol. 31, 1655\u20131669 (2021).","journal-title":"Int. J. Imaging Syst. Technol."},{"issue":"174","key":"51329_CR42","first-page":"1","volume":"46","author":"MN Pal","year":"2021","unstructured":"Pal, M. N., Roy, S. & Banerjee, M. Content based retrieval of retinal OCT scans using twin CNN. S\u00e5dhan\u00e5 46(174), 1\u201314 (2021).","journal-title":"S\u00e5dhan\u00e5"},{"key":"51329_CR43","doi-asserted-by":"crossref","unstructured":"Bertinetto, L., Valmadre, J., Henriques, J., Vedaldi, A. & Torr, P. Fully-convolutional siamese networks for object tracking. In ECCV 2016 Workshops. ECCV 2016. Lecture Notes in Computer Science, Cham (2016).","DOI":"10.1007\/978-3-319-48881-3_56"},{"issue":"1","key":"51329_CR44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/08839514.2022.2094885","volume":"36","author":"H Takimoto","year":"2022","unstructured":"Takimoto, H., Seki, J., Situju, S. F. & Kanagawa, A. Anomaly detection using siamese network with attention mechanism for few-shot learning. Appl. Artif. Intell. 36(1), 1\u201318 (2022).","journal-title":"Appl. Artif. Intell."},{"key":"51329_CR45","unstructured":"Chung, Y.-A. & Weng, W.-H. Learning deep representations of medical images using siamese CNNs with application to content-based image retrieval. In 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA (2017)."},{"key":"51329_CR46","first-page":"18015","volume":"34","author":"EH Houssein","year":"2022","unstructured":"Houssein, E. H., Emam, M. M. & Ali, A. A. An optimized deep learning architecture for breast cancer diagnosis based on improved marine predators algorithm. Eng. Lett. 34, 18015\u201318033 (2022).","journal-title":"Eng. Lett."},{"key":"51329_CR47","first-page":"1","volume":"30","author":"H Yinyin","year":"2020","unstructured":"Yinyin, H., Xiaoxia, Z., Jiao, Y. & Shuai, F. A hybrid convolutional neural network model based on different evolution for medical image classification. Eng. Lett. 30, 1 (2020).","journal-title":"Eng. Lett."},{"key":"51329_CR48","first-page":"9481","volume":"81","author":"JJ Winston","year":"2022","unstructured":"Winston, J. J., Hemanth, D. J., Angelopoulou, A. & Kapetanios, E. Hybrid deep convolutional neural models for iris image recognition multimedia tools and applications. J. Eng. Technol. 81, 9481\u20139503 (2022).","journal-title":"J. Eng. Technol."},{"issue":"2","key":"51329_CR49","first-page":"35","volume":"6","author":"CK Jennifer","year":"2021","unstructured":"Jennifer, C. K., Faisal, B. S. & Aliyu, N. S. Breast cancer classification using deep convolutional neural networks. J. Eng. Technol. 6(2), 35\u201338 (2021).","journal-title":"J. Eng. Technol."},{"issue":"1","key":"51329_CR50","doi-asserted-by":"publisher","first-page":"61","DOI":"10.56471\/slujst.v4i.270","volume":"4","author":"ON Oyelade","year":"2022","unstructured":"Oyelade, O. N., Sambo, A. A., Dung, P. B. & Abubakar, A. H. Data augmentation-aided convolutional neural network for detection of abnormalities in digital mammograph. SLU J. Sci. Technol. 4(1), 61\u201367 (2022).","journal-title":"SLU J. Sci. Technol."},{"issue":"3","key":"51329_CR51","first-page":"3039","volume":"12","author":"S Khan","year":"2021","unstructured":"Khan, S., Reddy, V. & Anupama, P. DL-CNN framework for medical image analysis. Diagnostics 12(3), 3039\u20133044 (2021).","journal-title":"Diagnostics"},{"key":"51329_CR52","first-page":"77905","volume":"9","author":"ON Oyelade","year":"2021","unstructured":"Oyelade, O. N., Ezugwua, A. E. & Chiroma, H. CovFrameNet: An enhanced deep learning framework for COVID-19 detection. Diagnostics 9, 77905\u201377919 (2021).","journal-title":"Diagnostics"},{"key":"51329_CR53","doi-asserted-by":"publisher","first-page":"346","DOI":"10.3390\/diagnostics13030346","volume":"13","author":"NT Trang","year":"2023","unstructured":"Trang, N. T., Long, K. Q., An, P. L. & Dang, T. N. Development of an artificial intelligence-based breast cancer detection model by combining mammograms and medical health records. Diagnostics 13, 346 (2023).","journal-title":"Diagnostics"},{"key":"51329_CR54","first-page":"145","volume":"11","author":"R Yang","year":"2021","unstructured":"Yang, R. & Yu, Y. Artificial convolutional neural network in object detection and semantic segmentation for medical imaging analysis. Front. Oncol. 11, 145 (2021).","journal-title":"Front. Oncol."},{"issue":"10","key":"51329_CR55","first-page":"535","volume":"9","author":"D Harshita","year":"2021","unstructured":"Harshita, D. & Jitendra, T. Deep leraning based Detection Model for coronavirus (COVID-19) using CT and X-ray image Data. Front. Oncol. 9(10), 535\u2013542 (2021).","journal-title":"Front. Oncol."},{"key":"51329_CR56","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1038\/s41598-022-09905-3","volume":"12","author":"ON Oyelade","year":"2022","unstructured":"Oyelade, O. N. & Ezugwu, A. E. A novel wavelet decomposition and transformation convolutional neural network with data augmentation for breast cancer detection using digital mammogram. Sci. Rep. 12, 12 (2022).","journal-title":"Sci. Rep."},{"key":"51329_CR57","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1016\/j.ecoinf.2018.10.002","volume":"48","author":"BB Traore","year":"2018","unstructured":"Traore, B. B., Bernard, K.-F. & Fana, T. Deep convolution neural network for image recognition. Ecol. Inf. 48, 257\u2013268 (2018).","journal-title":"Ecol. Inf."},{"key":"51329_CR58","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1371\/journal.pone.0253205","volume":"16","author":"CH Chiang","year":"2021","unstructured":"Chiang, C. H., Weng, C. L. & Chiu, H. W. Automatic classification of medical image modality and anatomical location using convolutional neural network. PLOS ONE 16, 6 (2021).","journal-title":"PLOS ONE"},{"issue":"20057","key":"51329_CR59","first-page":"1","volume":"12","author":"DM Anisuzzaman","year":"2022","unstructured":"Anisuzzaman, D. M. et al. Multi-modal wound classifcation using wound image and location by deep neural network. Sci. Rep. 12(20057), 1\u201320 (2022).","journal-title":"Sci. Rep."},{"issue":"1050981","key":"51329_CR60","first-page":"1","volume":"16","author":"W Kong","year":"2022","unstructured":"Kong, W., Li, C. & Lei, Y. Multimodal medical image fusion using convolutional neural network and extreme learning machine. Front. Neurorobot. 16(1050981), 1\u201315 (2022).","journal-title":"Front. Neurorobot."},{"issue":"14","key":"51329_CR61","doi-asserted-by":"publisher","first-page":"22732","DOI":"10.1364\/OE.430508","volume":"29","author":"J Song","year":"2021","unstructured":"Song, J. et al. Multicolor image classification using the multimodal information bottleneck network (MMIB-Net) for detecting diabetic retinopathy. Opt. Express 29(14), 22732\u201322748 (2021).","journal-title":"Opt. Express"},{"issue":"2","key":"51329_CR62","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1109\/TRPMS.2018.2890359","volume":"3","author":"G Zhe","year":"2019","unstructured":"Zhe, G., Xiang, L., Heng, H., Ning, G. & Quanzheng, L. Deep learning-based image segmentation on multimodal medical imaging. IEEE Trans. Radiat. Plasma Med. Sci. 3(2), 162\u2013169 (2019).","journal-title":"IEEE Trans. Radiat. Plasma Med. Sci."},{"key":"51329_CR63","first-page":"162","volume":"3","author":"FA Spanhol","year":"2016","unstructured":"Spanhol, F. A., Oliveira, L. S., Petitjean, C. & Heutte, L. Breast cancer histopathological image classification using convolutional neural networks. IEEE Trans. Radiat. Plasma Med. Sci. 3, 162 (2016).","journal-title":"IEEE Trans. Radiat. Plasma Med. Sci."},{"key":"51329_CR64","first-page":"1583","volume":"7","author":"S Xu","year":"2019","unstructured":"Xu, S. et al. An early diagnosis of oral cancer based on three-dimensional convolutional neural networks. IEEE Access 7, 1583 (2019).","journal-title":"IEEE Access"},{"issue":"6","key":"51329_CR65","first-page":"316","volume":"9","author":"M Nawaz","year":"2018","unstructured":"Nawaz, M., Sewissy, A. A. & Soliman, T. A. Multi-class breast cancer classification using deep learning convolutional neural network. Int. J. Adv. Comput. Sci. Appl. 9(6), 316\u2013322 (2018).","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"51329_CR66","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1016\/j.ymeth.2019.06.014","volume":"173","author":"R Yan","year":"2020","unstructured":"Yan, R. et al. Breast cancer histopathological image classification using a hybrid deep neural network. Methods 173, 52\u201360 (2020).","journal-title":"Methods"},{"key":"51329_CR67","doi-asserted-by":"publisher","first-page":"4101","DOI":"10.1007\/s00330-021-08519-z","volume":"32","author":"A Pfob","year":"2022","unstructured":"Pfob, A. et al. The importance of multi-modal imaging and clinical information for humans and AI-based algorithms to classify breast masses (INSPiRED 003): An international, multicenter analysis. Eur. Radiol. 32, 4101\u20134115 (2022).","journal-title":"Eur. Radiol."},{"key":"51329_CR68","doi-asserted-by":"publisher","first-page":"2023","DOI":"10.1016\/j.optlaseng.2023.107667","volume":"168","author":"J Wu","year":"2023","unstructured":"Wu, J. et al. Multimodal microscopic imaging with deep learning for highly effective diagnosis of breast cancer. Opt. Lasers Eng. 168, 2023 (2023).","journal-title":"Opt. Lasers Eng."},{"key":"51329_CR69","doi-asserted-by":"publisher","first-page":"2021","DOI":"10.1038\/s41598-021-98408-8","volume":"11","author":"S Joo","year":"2021","unstructured":"Joo, S. et al. Multimodal deep learning models for the prediction of pathologic response to neoadjuvant chemotherapy in breast cancer. Sci. Rep. 11, 2021 (2021).","journal-title":"Sci. Rep."},{"key":"51329_CR70","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1109\/38.946629","volume":"21","author":"E Reinhard","year":"2001","unstructured":"Reinhard, E., Adhikhmin, M., Gooch, B. & Shirley, P. Color transfer between images. IEEE Comput. Graph. Appl. 21, 34\u201341 (2001).","journal-title":"IEEE Comput. Graph. Appl."},{"key":"51329_CR71","first-page":"2022","volume":"1\u201338","author":"O Akinola","year":"2022","unstructured":"Akinola, O., Oyelade, O. N. & Ezugwu, A.E.-S. Binary Ebola Optimization Search Algorithm (BEOSA) using novel S-V transformation functions for solving feature selection and classfication problem. Appl. Sci. 1\u201338, 2022 (2022).","journal-title":"Appl. Sci."},{"key":"51329_CR72","first-page":"896","volume":"2021","author":"ON Oyelade","year":"2021","unstructured":"Oyelade, O. N., Ezugwu, A. E., Mohammed, T. & Abualigah, L. Ebola Optimization Search Algorithm (EOSA): A metaheuristic algorithm based on the propagation model of Ebola virus disease with application in medical image classification problem. IEEE Access 2021, 896 (2021).","journal-title":"IEEE Access"},{"key":"51329_CR73","unstructured":"Spanhol, F., Oliveira, L. S., Petitjean, C. & Heutte, L. Breast cancer histopathological database (BreakHis) (2021, accessed 20 Apr 2021). https:\/\/web.inf.ufpr.br\/vri\/databases\/breast-cancer-histopathological-database-BreakHis\/."},{"issue":"7","key":"51329_CR74","doi-asserted-by":"publisher","first-page":"1455","DOI":"10.1109\/TBME.2015.2496264","volume":"63","author":"FA Spanhol","year":"2016","unstructured":"Spanhol, F. A., Oliveira, L. S., Petitjean, C. & Heutte, L. A dataset for breast cancer histopathological image classification. IEEE Trans. Biomed. Eng. (TBME) 63(7), 1455\u20131462 (2016).","journal-title":"IEEE Trans. Biomed. Eng. (TBME)"},{"key":"51329_CR75","first-page":"563","volume":"2019","author":"A Pol\u00f3nia","year":"2019","unstructured":"Pol\u00f3nia, A., Eloy, C. & Aguiar, P. BACH dataset\u202f: Grand challenge on breast cancer histology images. Med. Image Anal. 2019, 563 (2019).","journal-title":"Med. Image Anal."},{"key":"51329_CR76","first-page":"89","volume":"2017","author":"M Marches","year":"2017","unstructured":"Marches, M. Megapixel size image creation using generative adversarial networks. ArXiv 2017, 89 (2017).","journal-title":"ArXiv"},{"key":"51329_CR77","doi-asserted-by":"crossref","unstructured":"Xi, P., Shu, C. & Goubran, R. Abnormality detection in mammography using deep convolutional neural networks. arXiv:1803.01906v1 [cs.CV] (2018).","DOI":"10.1109\/MeMeA.2018.8438639"},{"issue":"110","key":"51329_CR78","first-page":"1","volume":"24","author":"Z Yang","year":"2023","unstructured":"Yang, Z. et al. CNN-Siam: Multimodal siamese CNN-based deep learning approach for drug-drug interaction prediction. BMC Bioinf. 24(110), 1\u201312 (2023).","journal-title":"BMC Bioinf."},{"key":"51329_CR79","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-021-98408-8","volume":"11","author":"S Joo","year":"2021","unstructured":"Joo, S. et al. Multimodal deep learning models for the prediction of pathologic response to neoadjuvant chemotherapy in breast cancer. Sci. Rep. 11, 1 (2021).","journal-title":"Sci. Rep."},{"key":"51329_CR80","first-page":"1","volume":"2022","author":"M Jiang","year":"2022","unstructured":"Jiang, M. et al. Multimodal imaging of target detection algorithm under artificial intelligence in the diagnosis of early breast cancer. J. Healthc. Eng. 2022, 1\u201310 (2022).","journal-title":"J. Healthc. Eng."},{"key":"51329_CR81","first-page":"6","volume":"8","author":"S Misra","year":"2022","unstructured":"Misra, S. et al. \"Deep learning-based multimodal fusion network for segmentation and classification of breast cancers using B-mode and elastography ultrasound images. Bioeng. Transl. Med. 8, 6 (2022).","journal-title":"Bioeng. Transl. Med."},{"key":"51329_CR82","doi-asserted-by":"publisher","first-page":"108235","DOI":"10.1016\/j.bspc.2021.102825","volume":"71","author":"D Muduli","year":"2022","unstructured":"Muduli, D., Dash, R. & Majhi, B. Automated diagnosis of breast cancer using multi-modal datasets: A deep convolution neural network based approach. Biomed. Signal Process. Control 71, 108235 (2022).","journal-title":"Biomed. Signal Process. Control"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-024-51329-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-024-51329-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-024-51329-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,9]],"date-time":"2024-01-09T06:05:36Z","timestamp":1704780336000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-024-51329-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,6]]},"references-count":82,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["51329"],"URL":"https:\/\/doi.org\/10.1038\/s41598-024-51329-8","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,6]]},"assertion":[{"value":"14 October 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 January 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 January 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"692"}}