{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T21:11:30Z","timestamp":1781644290202,"version":"3.54.5"},"reference-count":53,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2020,11,30]],"date-time":"2020-11-30T00:00:00Z","timestamp":1606694400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Seed Grant Program, German Jordanian University, Amman, Jordan","award":["SEEIT 03\/2020"],"award-info":[{"award-number":["SEEIT 03\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This study aims to enable effective breast ultrasound image classification by combining deep features with conventional handcrafted features to classify the tumors. In particular, the deep features are extracted from a pre-trained convolutional neural network model, namely the VGG19 model, at six different extraction levels. The deep features extracted at each level are analyzed using a features selection algorithm to identify the deep feature combination that achieves the highest classification performance. Furthermore, the extracted deep features are combined with handcrafted texture and morphological features and processed using features selection to investigate the possibility of improving the classification performance. The cross-validation analysis, which is performed using 380 breast ultrasound images, shows that the best combination of deep features is obtained using a feature set, denoted by CONV features that include convolution features extracted from all convolution blocks of the VGG19 model. In particular, the CONV features achieved mean accuracy, sensitivity, and specificity values of 94.2%, 93.3%, and 94.9%, respectively. The analysis also shows that the performance of the CONV features degrades substantially when the features selection algorithm is not applied. The classification performance of the CONV features is improved by combining these features with handcrafted morphological features to achieve mean accuracy, sensitivity, and specificity values of 96.1%, 95.7%, and 96.3%, respectively. Furthermore, the cross-validation analysis demonstrates that the CONV features and the combined CONV and morphological features outperform the handcrafted texture and morphological features as well as the fine-tuned VGG19 model. The generalization performance of the CONV features and the combined CONV and morphological features is demonstrated by performing the training using the 380 breast ultrasound images and the testing using another dataset that includes 163 images. The results suggest that the combined CONV and morphological features can achieve effective breast ultrasound image classifications that increase the capability of detecting malignant tumors and reduce the potential of misclassifying benign tumors.<\/jats:p>","DOI":"10.3390\/s20236838","type":"journal-article","created":{"date-parts":[[2020,11,29]],"date-time":"2020-11-29T21:55:31Z","timestamp":1606686931000},"page":"6838","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":47,"title":["Breast Tumor Classification in Ultrasound Images Using Combined Deep and Handcrafted Features"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9399-5769","authenticated-orcid":false,"given":"Mohammad I.","family":"Daoud","sequence":"first","affiliation":[{"name":"Department of Computer Engineering, German Jordanian University, Amman 11180, Jordan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0224-643X","authenticated-orcid":false,"given":"Samir","family":"Abdel-Rahman","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, German Jordanian University, Amman 11180, Jordan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2049-2113","authenticated-orcid":false,"given":"Tariq M.","family":"Bdair","sequence":"additional","affiliation":[{"name":"Chair for Computer Aided Medical Procedure, Technical University of Munich, 85748 Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8661-5473","authenticated-orcid":false,"given":"Mahasen S.","family":"Al-Najar","sequence":"additional","affiliation":[{"name":"Department of Diagnostic Radiology, The University of Jordan Hospital, Queen Rania Street, Amman 11942, Jordan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6948-3336","authenticated-orcid":false,"given":"Feras H.","family":"Al-Hawari","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, German Jordanian University, Amman 11180, Jordan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1296-0231","authenticated-orcid":false,"given":"Rami","family":"Alazrai","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, German Jordanian University, Amman 11180, Jordan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,11,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"394","DOI":"10.3322\/caac.21492","article-title":"Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"68","author":"Bray","year":"2018","journal-title":"CA A Cancer J. Clin."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Nothacker, M., Duda, V., Hahn, M., Warm, M., Degenhardt, F., Madjar, H., Weinbrenner, S., and Albert, U. (2009). Early detection of breast cancer: Benefits and risks of supplemental breast ultrasound in asymptomatic women with mammographically dense breast tissue. A systematic review. BMC Cancer, 9.","DOI":"10.1186\/1471-2407-9-335"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chotai, N., and Kulkarni, S. (2020). Breast Ultrasound. Breast Imaging Essentials, Springer.","DOI":"10.1007\/978-981-15-1412-8"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1007\/s10278-012-9499-x","article-title":"Breast ultrasound image classification based on multiple-instance learning","volume":"25","author":"Ding","year":"2012","journal-title":"J. Digit. Imaging"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"990","DOI":"10.1016\/j.eswa.2014.09.020","article-title":"Benign and malignant breast tumors classification based on region growing and CNN segmentation","volume":"42","author":"Rouhi","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1186\/s13673-015-0029-y","article-title":"Assessment of combined textural and morphological features for diagnosis of breast masses in ultrasound","volume":"5","author":"Prabusankarlal","year":"2015","journal-title":"Human-Centric Comput. Inf. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"6740956","DOI":"10.1155\/2016\/6740956","article-title":"A fusion-based approach for breast ultrasound image classification using multiple-ROI texture and morphological analyses","volume":"2016","author":"Daoud","year":"2016","journal-title":"Comput. Math. Methods Med."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Daoud, M.I., Saleh, A., Hababeh, I., and Alazrai, R. (2019, January 24\u201326). Content-based image retrieval for breast ultrasound images using convolutional autoencoders: A Feasibility study. Proceedings of the 3rd International Conference on Bio-Engineering for Smart Technologies (BioSMART), Paris, France.","DOI":"10.1109\/BIOSMART.2019.8734190"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"7714","DOI":"10.1088\/1361-6560\/aa82ec","article-title":"A deep learning framework for supporting the classification of breast lesions in ultrasound images","volume":"62","author":"Han","year":"2017","journal-title":"Phys. Med. Biol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1016\/j.compmedimag.2012.07.004","article-title":"Combining support vector machine with genetic algorithm to classify ultrasound breast tumor images","volume":"36","author":"Wu","year":"2012","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"4112","DOI":"10.1002\/mp.13082","article-title":"Classification of breast lesions in ultrasonography using sparse logistic regression and morphology-based texture features","volume":"45","author":"Nemat","year":"2018","journal-title":"Med. Phys."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.cmpb.2013.03.017","article-title":"Computer-aided diagnosis of breast masses using quantified BI-RADS findings","volume":"111","author":"Moon","year":"2013","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1007\/s10549-004-2043-z","article-title":"Automatic ultrasound segmentation and morphology based diagnosis of solid breast tumors","volume":"89","author":"Chang","year":"2005","journal-title":"Breast Cancer Res. Treat."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1889","DOI":"10.1109\/TMI.2012.2206398","article-title":"Analysis of co-occurrence texture statistics as a function of gray-level quantization for classifying breast ultrasound","volume":"31","author":"Gomez","year":"2012","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"693","DOI":"10.1109\/TFUZZ.2013.2269149","article-title":"Breast nodules computer-aided diagnostic system design using fuzzy cerebellar model neural networks","volume":"22","author":"Lin","year":"2014","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Daoud, M.I., Abdel-Rahman, S., and Alazrai, R. (2019, January 26\u201329). Breast ultrasound image classification using a pre-trained convolutional neural network. Proceedings of the 15th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), Sorrento, Italy.","DOI":"10.1109\/SITIS.2019.00037"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1016\/j.eng.2018.11.020","article-title":"Deep learning in medical ultrasound analysis: A review","volume":"5","author":"Liu","year":"2019","journal-title":"Engineering"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5137904","DOI":"10.1155\/2018\/5137904","article-title":"Machine learning in ultrasound computer-aided diagnostic systems: A survey","volume":"2018","author":"Huang","year":"2018","journal-title":"BioMed Res. Int."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"466","DOI":"10.1007\/s11604-019-00831-5","article-title":"Distinction between benign and malignant breast masses at breast ultrasound using deep learning method with convolutional neural network","volume":"37","author":"Fujioka","year":"2019","journal-title":"Jpn. J. Radiol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"684","DOI":"10.1016\/j.bbe.2018.05.003","article-title":"Discriminant analysis of neural style representations for breast lesion classification in ultrasound","volume":"38","author":"Byra","year":"2018","journal-title":"Biocybern. Biomed. Eng."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1002\/mp.13361","article-title":"Breast mass classification in sonography with transfer learning using a deep convolutional neural network and color conversion","volume":"46","author":"Byra","year":"2019","journal-title":"Med. Phys."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"4605191","DOI":"10.1155\/2018\/4605191","article-title":"Comparison of transferred deep neural networks in ultrasonic breast masses discrimination","volume":"2018","author":"Xiao","year":"2018","journal-title":"BioMed Res. Int."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"235013","DOI":"10.1088\/1361-6560\/ab5093","article-title":"Computer-aided diagnosis system for breast ultrasound images using deep learning","volume":"64","author":"Tanaka","year":"2019","journal-title":"Phys. Med. Biol."},{"key":"ref_25","first-page":"5162","article-title":"A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets","volume":"44","author":"Antropova","year":"2017","journal-title":"Phys. Med. Biol."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_30","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_31","unstructured":"Duda, R., and Hart, P. (1973). Pattern Classification and Scene Analysis, Wiley. [1st ed.]."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Vapnik, V.N. (2000). The Nature of Statistical Learning Theory, Springer. [2nd ed.].","DOI":"10.1007\/978-1-4757-3264-1"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1218","DOI":"10.1109\/JBHI.2017.2731873","article-title":"Automated breast ultrasound lesions detection using convolutional neural networks","volume":"22","author":"Yap","year":"2018","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_34","first-page":"159","article-title":"ImageNet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis. Vol."},{"key":"ref_35","unstructured":"Zheng, L., Zhao, Y., Wang, S., Wang, J., and Tian, Q. (2016). Good practice in CNN feature transfer. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Arandjelovi\u0107, R., and Zisserman, A. (2012, January 16\u201321). Three things everyone should know to improve object retrieval. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012), Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248018"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/j.jbi.2018.07.014","article-title":"Relief-based feature selection: Introduction and review","volume":"85","author":"Urbanowicz","year":"2018","journal-title":"J. Biomed. Inform."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1109\/TPAMI.2005.159","article-title":"Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy","volume":"27","author":"Peng","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"442","DOI":"10.1016\/0005-2795(75)90109-9","article-title":"Comparison of the predicted and observed secondary structure of T4 phage lysozyme","volume":"405","author":"Matthews","year":"1975","journal-title":"Biochim. Biophys. Acta (BBA) Protein Struct."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13040-017-0155-3","article-title":"Ten quick tips for machine learning in computational biology","volume":"10","author":"Chicco","year":"2017","journal-title":"BioData Min."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1145\/1961189.1961199","article-title":"LIBSVM: A library for support vector machines","volume":"2","author":"Chang","year":"2011","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1021\/ci0342472","article-title":"The problem of overfitting","volume":"44","author":"Hawkins","year":"2004","journal-title":"J. Chem. Inf. Comput. Sci."},{"key":"ref_43","unstructured":"Hsu, C.W., Chang, C.C., and Lin, C.J. (2008). A Practical Guide to Support Vector Classification, Department of Computer Science and Information Engineering, National Taiwan University. Technical Report."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.eswa.2018.11.024","article-title":"Automatic superpixel-based segmentation method for breast ultrasound images","volume":"121","author":"Daoud","year":"2019","journal-title":"Expert Syst. Appl."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1109\/TSMC.1973.4309314","article-title":"Textural features for image classification","volume":"SMC-3","author":"Haralick","year":"1973","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"3024","DOI":"10.1118\/1.4921123","article-title":"Computer-aided diagnosis for distinguishing between triple-negative breast cancer and fibroadenomas based on ultrasound texture features","volume":"42","author":"Moon","year":"2015","journal-title":"Med. Phys."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"928","DOI":"10.1016\/j.acra.2007.04.016","article-title":"Breast ultrasound computer-aided diagnosis using BI-RADS features","volume":"14","author":"Shen","year":"2007","journal-title":"Acad. Radiol."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1007\/BF02345742","article-title":"Boundary modelling and shape analysis methods for classification of mammographic masses","volume":"38","author":"Rangayyan","year":"2000","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1513","DOI":"10.1016\/j.acra.2008.06.005","article-title":"Quantitative analysis of lesion morphology and texture features for diagnostic prediction in breast MRI","volume":"15","author":"Nie","year":"2008","journal-title":"Acad. Radiol."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"780","DOI":"10.1109\/36.752194","article-title":"Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices","volume":"37","author":"Soh","year":"1999","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"45","DOI":"10.5589\/m02-004","article-title":"An analysis of co-occurrence texture statistics as a function of grey level quantization","volume":"28","author":"Clausi","year":"2002","journal-title":"Can. J. Remote Sens."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1016\/j.patcog.2009.05.012","article-title":"Automated breast cancer detection and classification using ultrasound images: A survey","volume":"43","author":"Cheng","year":"2010","journal-title":"Pattern Recognit."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1016\/j.patcog.2017.10.013","article-title":"Recent advances in convolutional neural networks","volume":"77","author":"Gu","year":"2018","journal-title":"Pattern Recognit."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/23\/6838\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:39:23Z","timestamp":1760179163000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/23\/6838"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,30]]},"references-count":53,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["s20236838"],"URL":"https:\/\/doi.org\/10.3390\/s20236838","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,30]]}}}