{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T16:36:13Z","timestamp":1780763773411,"version":"3.54.1"},"reference-count":48,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,6,21]],"date-time":"2021-06-21T00:00:00Z","timestamp":1624233600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["GADgET (DSAIPA\/DS\/0022\/2018)"],"award-info":[{"award-number":["GADgET (DSAIPA\/DS\/0022\/2018)"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004329","name":"Javna Agencija za Raziskovalno Dejavnost RS","doi-asserted-by":"publisher","award":["No. P5-0410"],"award-info":[{"award-number":["No. P5-0410"]}],"id":[{"id":"10.13039\/501100004329","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Bone fractures are among the main reasons for emergency room admittance and require a rapid response from doctors. Bone fractures can be severe and can lead to permanent disability if not treated correctly and rapidly. Using X-ray imaging in the emergency room to detect fractures is a challenging task that requires an experienced radiologist, a specialist who is not always available. The availability of an automatic tool for image classification can provide a second opinion for doctors operating in the emergency room and reduce the error rate in diagnosis. This study aims to increase the existing state-of-the-art convolutional neural networks\u2019 performance by using various ensemble techniques. In this approach, different CNNs (Convolutional Neural Networks) are used to classify the images; rather than choosing the best one, a stacking ensemble provides a more reliable and robust classifier. The ensemble model outperforms the results of individual CNNs by an average of 10%.<\/jats:p>","DOI":"10.3390\/jimaging7060100","type":"journal-article","created":{"date-parts":[[2021,6,21]],"date-time":"2021-06-21T04:21:21Z","timestamp":1624249281000},"page":"100","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":34,"title":["Comparing Stacking Ensemble Techniques to Improve Musculoskeletal Fracture Image Classification"],"prefix":"10.3390","volume":"7","author":[{"given":"Ibrahem","family":"Kandel","sequence":"first","affiliation":[{"name":"Nova Information Management School (NOVA IMS), Universidade Nova de Lisboa, Campus de Campolide, 1070-312 Lisboa, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8793-1451","authenticated-orcid":false,"given":"Mauro","family":"Castelli","sequence":"additional","affiliation":[{"name":"Nova Information Management School (NOVA IMS), Universidade Nova de Lisboa, Campus de Campolide, 1070-312 Lisboa, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ale\u0161","family":"Popovi\u010d","sequence":"additional","affiliation":[{"name":"Nova Information Management School (NOVA IMS), Universidade Nova de Lisboa, Campus de Campolide, 1070-312 Lisboa, Portugal"},{"name":"School of Economics and Business, University of Ljubljana, Kardeljeva Plo\u0161\u010dad 17, 1000 Ljubljana, Slovenia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"290","DOI":"10.2106\/00004623-200402000-00011","article-title":"The Incidence of Fractures and Dislocations Referred for Orthopaedic Services in a Capitated Population","volume":"86","author":"Brinker","year":"2004","journal-title":"JBJS"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.bone.2016.03.006","article-title":"Epidemiology of fractures in the United Kingdom 1988-2012: Variation with age, sex, geography, ethnicity and socioeconomic status","volume":"87","author":"Curtis","year":"2016","journal-title":"Bone"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"NIH Consensus Development Panel on Osteoporosis Prevention and Therapy, D (2001). Osteoporosis Prevention, Diagnosis, and Therapy. JAMA, 285, 785\u2013795.","DOI":"10.1001\/jama.285.6.785"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"i1","DOI":"10.1136\/gut.46.suppl_1.I1","article-title":"Guidelines for osteoporosis in coeliac disease and inflammatory bowel disease. British Society of Gastroenterology","volume":"46","author":"Scott","year":"2000","journal-title":"Gut"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"188","DOI":"10.2106\/JBJS.H.00628","article-title":"Osteoporosis Disease Management: The Role of the Orthopaedic Surgeon","volume":"90","author":"Dell","year":"2008","journal-title":"JBJS"},{"key":"ref_6","unstructured":"Courtney, M.T. (2004). Sabiston Textbook of Surgery: The Biological Basis of Modern Surgical Practice, Elsevier Saunders. [7th ed.]."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1146\/annurev.pu.16.050195.002121","article-title":"Methodological Challenges in Injury Epidemiology and Injury Prevention Research","volume":"16","author":"Cummings","year":"1995","journal-title":"Annu. Rev. Public Health"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1016\/0266-7681(92)90118-L","article-title":"Twenty Questions about Scaphoid Fractures","volume":"17","author":"Barton","year":"1992","journal-title":"J. Hand Surg. Am."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1007\/s004230000195","article-title":"Diagnosis of occult scaphoid fractures and other wrist injuries","volume":"386","author":"Kukla","year":"2001","journal-title":"Langenbeck\u2019s Arch. Surg."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"468","DOI":"10.1080\/17453674.2018.1453714","article-title":"Automated detection and classification of the proximal humerus fracture by using deep learning algorithm","volume":"89","author":"Chung","year":"2018","journal-title":"Acta Orthop."},{"key":"ref_12","unstructured":"Rajpurkar, P., Irvin, J., Bagul, A., Ding, D.Y., Duan, T., Mehta, H., Yang, B.J., Zhu, K., Laird, D., and Ball, R.L. (2017). MURA: Large Dataset for Abnormality Detection in Musculoskeletal Radiographs. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1080\/17453674.2017.1344459","article-title":"Artificial intelligence for analyzing orthopedic trauma radiographs","volume":"88","author":"Olczak","year":"2017","journal-title":"Acta Orthop."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"11591","DOI":"10.1073\/pnas.1806905115","article-title":"Deep neural network improves fracture detection by clinicians","volume":"115","author":"Lindsey","year":"2018","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Uysal, F., Hardala\u00e7, F., Peker, O., Tolunay, T., and Tokg\u00f6z, N. (2021). Classification of Shoulder X-ray Images with Deep Learning Ensemble Models. Appl. Sci., 11.","DOI":"10.3390\/app11062723"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"106530","DOI":"10.1016\/j.compeleceng.2019.106530","article-title":"Arm fracture detection in X-rays based on improved deep convolutional neural network","volume":"81","author":"Guan","year":"2020","journal-title":"Comput. Electr. Eng."},{"key":"ref_17","unstructured":"Vinh, P.C., and Rakib, A. (2021). Abnormality Bone Detection in X-Ray Images Using Convolutional Neural Network BT\u2014Context-Aware Systems and Applications, and Nature of Computation and Communication, Springer International Publishing."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Urinbayev, K., Orazbek, Y., Nurambek, Y., Mirzakhmetov, A., and Varol, H.A. (2020, January 20\u201324). End-to-End Deep Diagnosis of X-ray Images. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montr\u00e9al, QC, Canada.","DOI":"10.1109\/EMBC44109.2020.9175208"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"672","DOI":"10.1007\/s10278-018-0167-7","article-title":"Ankle Fracture Detection Utilizing a Convolutional Neural Network Ensemble Implemented with a Small Sample, De Novo Training, and Multiview Incorporation","volume":"32","author":"Kitamura","year":"2019","journal-title":"J. Digit. Imaging"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chouhan, V., Singh, S., Khamparia, A., Gupta, D., Tiwari, P., Moreira, C., Damasevicius, R., and Albuquerque, V. (2020). A Novel Transfer Learning Based Approach for Pneumonia Detection in Chest X-ray Images. Appl. Sci., 10.","DOI":"10.3390\/app10020559"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Rajaraman, S., Candemir, S., Xue, Z., Alderson, P., Thoma, G., and Antani, S. (2019). A Novel Stacked Model Ensemble for Improved TB Detection in Chest Radiographs, CRC Press.","DOI":"10.1201\/9780429029417-1"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1016\/j.ebiom.2019.06.050","article-title":"Automated diagnosis of ear disease using ensemble deep learning with a big otoendoscopy image database","volume":"45","author":"Cha","year":"2019","journal-title":"EBioMedicine"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kandel, I., Castelli, M., and Popovi\u010d, A. (2020). Musculoskeletal Images Classification for Detection of Fractures Using Transfer Learning. J. Imaging, 6.","DOI":"10.3390\/jimaging6110127"},{"key":"ref_26","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Maaten, L.v.d., and Weinberger, K.Q. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep Learning with Depthwise Separable Convolutions. Proceedings of the IEEEConference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_30","first-page":"11","article-title":"Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning","volume":"42","author":"Karlsson","year":"2014","journal-title":"Pattern Recognit. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"4475","DOI":"10.1007\/s10462-019-09799-0","article-title":"A survey of fracture detection techniques in bone X-ray images","volume":"53","author":"Joshi","year":"2020","journal-title":"Artif. Intell. Rev."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z.B. (2016, January 27\u201330). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach. Learn."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_35","first-page":"49","article-title":"Multi-class AdaBoost","volume":"2","author":"Zhu","year":"2009","journal-title":"Stat. Interface"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/S0167-9473(01)00065-2","article-title":"Stochastic Gradient Boosting","volume":"38","author":"Friedman","year":"2002","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zhang, R., Li, W., Tan, W., and Mo, T. (2017, January 25\u201330). Deep and Shallow Model for Insurance Churn Prediction Service. Proceedings of the 2017 IEEE International Conference on Services Computing (SCC), Honolulu, HI, USA.","DOI":"10.1109\/SCC.2017.51"},{"key":"ref_38","unstructured":"Rish, I. (2001, January 4\u201310). An Empirical Study of the Na\u00efve Bayes Classifier. Proceedings of the IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, Seattle, WA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","article-title":"Stacked generalization","volume":"5","author":"Wolpert","year":"1992","journal-title":"Neural Netw."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A Coefficient of Agreement for Nominal Scales","volume":"20","author":"Cohen","year":"1960","journal-title":"Educ. Psychol. Meas."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Kandel, I., and Castelli, M. (2020). How deeply to fine-tune a convolutional neural network: A case study using a histopathology dataset. Appl. Sci., 10.","DOI":"10.3390\/app10103359"},{"key":"ref_42","unstructured":"Kingma, D., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. Int. Conf. Learn. Represent."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"115041","DOI":"10.1109\/ACCESS.2020.3003810","article-title":"Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-Rays","volume":"8","author":"Rajaraman","year":"2020","journal-title":"IEEE Access"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus PhotographsAccuracy of a Deep Learning Algorithm for Detection of Diabetic RetinopathyAccuracy of a Deep Learning Algorithm for Detection of Di","volume":"316","author":"Gulshan","year":"2016","journal-title":"JAMA"},{"key":"ref_45","unstructured":"Calin-Jageman, R., and EBSCOhost (2016). Introduction to the New Statistics, Routledge."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Ganaie, M., Hu, M., Tanveer, M., and Suganthan, P. (2021). Ensemble deep learning: A review. arXiv.","DOI":"10.1016\/j.engappai.2022.105151"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Zhou, Z.-H. (2012). Ensemble Methods: Foundations and Algorithms, Chapman & Hall\/CRC. [1st ed.].","DOI":"10.1201\/b12207"},{"key":"ref_48","unstructured":"Ting, K.M., and Witten, I.H. (1997). Stacked Generalization: When Does It Work?, Computer Science Working Papers; Department of Computer Science, University of Waik."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/7\/6\/100\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:19:49Z","timestamp":1760163589000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/7\/6\/100"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,21]]},"references-count":48,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["jimaging7060100"],"URL":"https:\/\/doi.org\/10.3390\/jimaging7060100","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,21]]}}}