{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T01:11:14Z","timestamp":1779153074207,"version":"3.51.4"},"reference-count":44,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2019,7,5]],"date-time":"2019-07-05T00:00:00Z","timestamp":1562284800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2017YFB0202601"],"award-info":[{"award-number":["2017YFB0202601"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Macular edema (ME) is a retinal condition in which central vision of a patient is affected. ME leads to accumulation of fluid in the surrounding macular region resulting in a swollen macula. Optical coherence tomography (OCT) and the fundus photography are the two widely used retinal examination techniques that can effectively detect ME. Many researchers have utilized retinal fundus and OCT imaging for detecting ME. However, to the best of our knowledge, no work is found in the literature that fuses the findings from both retinal imaging modalities for the effective and more reliable diagnosis of ME. In this paper, we proposed an automated framework for the classification of ME and healthy eyes using retinal fundus and OCT scans. The proposed framework is based on deep ensemble learning where the input fundus and OCT scans are recognized through the deep convolutional neural network (CNN) and are processed accordingly. The processed scans are further passed to the second layer of the deep CNN model, which extracts the required feature descriptors from both images. The extracted descriptors are then concatenated together and are passed to the supervised hybrid classifier made through the ensemble of the artificial neural networks, support vector machines and na\u00efve Bayes. The proposed framework has been trained on 73,791 retinal scans and is validated on 5100 scans of publicly available Zhang dataset and Rabbani dataset. The proposed framework achieved the accuracy of 94.33% for diagnosing ME and healthy subjects and achieved the mean dice coefficient of 0.9019 \u00b1 0.04 for accurately extracting the retinal fluids, 0.7069 \u00b1 0.11 for accurately extracting hard exudates and 0.8203 \u00b1 0.03 for accurately extracting retinal blood vessels against the clinical markings.<\/jats:p>","DOI":"10.3390\/s19132970","type":"journal-article","created":{"date-parts":[[2019,7,5]],"date-time":"2019-07-05T11:44:16Z","timestamp":1562327056000},"page":"2970","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["Deep Ensemble Learning Based Objective Grading of Macular Edema by Extracting Clinically Significant Findings from Fused Retinal Imaging Modalities"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3672-8100","authenticated-orcid":false,"given":"Bilal","family":"Hassan","sequence":"first","affiliation":[{"name":"School of Automation Science and Electrical Engineering, Beihang University (BUAA), Beijing 100191, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5896-8677","authenticated-orcid":false,"given":"Taimur","family":"Hassan","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Bahria University (BU), Islamabad 44000, Pakistan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bo","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Beihang University (BUAA), Beijing 100191, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9337-1921","authenticated-orcid":false,"given":"Ramsha","family":"Ahmed","sequence":"additional","affiliation":[{"name":"School of Computer and Communication Engineering, University of Science &amp; Technology Beijing (USTB), Beijing 100083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Omar","family":"Hassan","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Sir Syed CASE Institute of Technology (SSCIT), Islamabad 44000, Pakistan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,7,5]]},"reference":[{"key":"ref_1","unstructured":"GBD 2017 Disease and Injury Incidence and Prevalence Collaborators (2018). Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990\u20132017: A systematic analysis for the Global Burden of Disease Study 2017. Lancet, 392, 1789\u20131858."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Hassan, B., Ahmed, R., Li, B., Noor, A., and Hassan, Z.U. (2019). A comprehensive study capturing vision loss burden in Pakistan (1990\u20132025): Findings from the Global Burden of Disease (GBD) 2017 study. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0216492"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1383\/medc.2006.34.3.95","article-title":"Diabetic retinopathy","volume":"34","author":"Harney","year":"2006","journal-title":"Medicine"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.sjopt.2011.01.009","article-title":"Diabetic retinopathy\u2014An update","volume":"25","author":"Alghadyan","year":"2011","journal-title":"Saudi J. Ophthalmol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.knosys.2015.02.005","article-title":"Computer-aided diagnosis of diabetic subjects by heart rate variability signals using discrete wavelet transform method","volume":"81","author":"Acharya","year":"2015","journal-title":"Knowl. Based Syst."},{"key":"ref_6","first-page":"419","article-title":"Diabetic retinopathy: Time for action. No complacency please!","volume":"80","author":"Verma","year":"2002","journal-title":"Bull. World Health Organ."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"237","DOI":"10.3109\/09286586.2015.1058958","article-title":"Global Efforts to Generate Evidence for Vision 2020","volume":"22","author":"Mingguang","year":"2015","journal-title":"Ophthalmic Epidemiol."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Hassan, T., Akram, M.U., Hassan, B., Nasim, A., and Bazaz, S.A. (2015, January 16\u201318). Review of OCT and fundus images for detection of Macular Edema. Proceedings of the 2015 IEEE International Conference on Imaging Systems and Techniques (IST), Macau, China.","DOI":"10.1109\/IST.2015.7294517"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1178","DOI":"10.1126\/science.1957169","article-title":"Optical coherence tomography","volume":"254","author":"Huang","year":"1991","journal-title":"Science"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1864","DOI":"10.1364\/OL.18.001864","article-title":"In vivo retinal imaging by optical coherence tomography","volume":"18","author":"Swanson","year":"1993","journal-title":"Opt. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1186\/s40942-015-0005-8","article-title":"A review of optical coherence tomography angiography (OCTA)","volume":"1","author":"Romano","year":"2015","journal-title":"Int. J. Retin. Vitr."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1088\/0034-4885\/66\/2\/204","article-title":"Optical coherence tomography-principles and applications","volume":"66","author":"Fercher","year":"2003","journal-title":"Rep. Prog. Phys."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1097\/IAE.0b013e318164a907","article-title":"Fundus autofluorescence imaging: Review and perspectives","volume":"28","author":"Holz","year":"2008","journal-title":"Retina"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.sjopt.2014.03.008","article-title":"Fundus autofluorescence imaging: Fundamentals and clinical relevance","volume":"28","author":"Sepah","year":"2014","journal-title":"Saudi J. Ophthalmol."},{"key":"ref_15","unstructured":"Chee, K.L., Santiago, P.A., Lingam, G., Singh, M.S., Naing, T., Mangunkusumo, A.E., and Naser, M.N. (2014). Application of Ocular Fundus Photography and Angiography. Ophthalmological Imaging and Applications, CRC Press."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"4963","DOI":"10.1167\/iovs.06-1472","article-title":"Optical coherence tomography versus stereoscopic fundus photography or biomicroscopy for diagnosing diabetic macular edema: A systematic review","volume":"48","author":"Virgili","year":"2007","journal-title":"Investig. Ophthalmol. Vis. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1016\/j.ophtha.2003.06.028","article-title":"Comparison of the clinical diagnosis of diabetic macular edema with diagnosis by optical coherence tomography","volume":"111","author":"Browning","year":"2004","journal-title":"Ophthalmology"},{"key":"ref_18","first-page":"241","article-title":"Diabetic macular edema assessed with optical coherence tomography and stereo fundus photography","volume":"43","author":"Sander","year":"2002","journal-title":"Investig. Ophthalmol. Vis. Sci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1007\/s10916-008-9166-4","article-title":"Automatic tracing of optic disc and exudates from color fundus images using fixed and variable thresholds","volume":"33","author":"Reza","year":"2009","journal-title":"J. Med. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Sreejini, K.S., and Govindan, V.K. (2013, January 29\u201331). Automatic grading of severity of diabetic macular edema using color fundus images. Proceedings of the 2013 Third International Conference on Advances in Computing and Communications (ICACC), Cochin, India.","DOI":"10.1109\/ICACC.2013.42"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1109\/TMI.2002.806290","article-title":"Contribution of image processing to the diagnosis of diabetic retinopathy-detection of exudates in color fundus images of the human retina","volume":"21","author":"Walter","year":"2002","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/j.media.2011.07.004","article-title":"Exudate-based diabetic macular edema detection in fundus images using publicly available datasets","volume":"16","author":"Giancardo","year":"2012","journal-title":"Med. Image Anal."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1109\/TITB.2008.2007493","article-title":"A computational-intelligence-based approach for detection of exudates in diabetic retinopathy images","volume":"13","author":"Osareh","year":"2009","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"766","DOI":"10.1109\/TMI.2011.2178856","article-title":"Automatic assessment of macular edema from color retinal images","volume":"31","author":"Deepak","year":"2012","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1997","DOI":"10.1007\/s10916-011-9659-4","article-title":"Automated identification of exudates and optic disc based on inverse surface thresholding","volume":"36","author":"Yazid","year":"2012","journal-title":"J. Med. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hassan, B., and Raja, G. (2016, January 15\u201317). Fully Automated Assessment of Macular Edema using Optical Coherence Tomography (OCT) Images. Proceedings of the 2016 International Conference on Intelligent Systems Engineering (ICISE), Islamabad, Pakistan.","DOI":"10.1109\/INTELSE.2016.7475153"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1109","DOI":"10.1109\/TBME.2012.2184759","article-title":"Automated segmentation of intraretinal cystoid fluid in optical coherence tomography","volume":"59","author":"Wilkins","year":"2012","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Sugmk, J., Kiattisin, S., and Leelasantitham, A. (2014, January 26\u201328). Automated classification between age-related macular degeneration and diabetic macular edema in OCT image using image segmentation. Proceedings of the 7th 2014 Biomedical Engineering International Conference (BMEiCON), Fukuoka, Japan.","DOI":"10.1109\/BMEiCON.2014.7017441"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1364\/JOSAA.33.000455","article-title":"Structure tensor based automated detection of macular edema and central serous retinopathy using optical coherence tomography images","volume":"33","author":"Hassan","year":"2016","journal-title":"J. Opt. Soc. Am. A Opt. Image Sci. Vis."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1172","DOI":"10.1364\/BOE.6.001172","article-title":"Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema","volume":"6","author":"Chiu","year":"2015","journal-title":"Biomed. Opt. Express"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sahar, S., Ayaz, S., Akram, M.U., and Basit, I. (2015, January 6\u20138). A Case Study Approach: Iterative Prototyping Model Based Detection of Macular Edema in Retinal OCT Images. Proceedings of the 27th International Conference on Software Engineering and Knowledge Engineering (SEKE), Pittsburgh, PA, USA.","DOI":"10.18293\/SEKE2015-033"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3568","DOI":"10.1364\/BOE.5.003568","article-title":"Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images","volume":"5","author":"Srinivasan","year":"2014","journal-title":"Biomed. Opt. Express"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"3440","DOI":"10.1364\/BOE.8.003440","article-title":"Deep-learning based, automated segmentation of macular edema in optical coherence tomography","volume":"8","author":"Lee","year":"2017","journal-title":"Biomed. Opt. Express"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1016\/j.compbiomed.2018.12.015","article-title":"Deep structure tensor graph search framework for automated extraction and characterization of retinal layers and fluid pathology in retinal SD-OCT scans","volume":"105","author":"Hassan","year":"2018","journal-title":"Comput. Biol. Med."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1122","DOI":"10.1016\/j.cell.2018.02.010","article-title":"Identifying medical diagnoses and treatable diseases by image-based deep learning","volume":"172","author":"Kermany","year":"2018","journal-title":"Cell"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1109\/TMI.2017.2780115","article-title":"Macular OCT classification using a multi-scale convolutional neural network ensemble","volume":"37","author":"Rasti","year":"2017","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1482","DOI":"10.1167\/iovs.14-15457","article-title":"Fully automatic segmentation of fluorescein leakage in subjects with diabetic macular edema","volume":"56","author":"Rabbani","year":"2015","journal-title":"Investig. Ophthalmol. Vis. Sci."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Mahmudi, T., Kafieh, R., Rabbani, H., and Akhlagi, M. (2014, January 17\u201321). Comparison of macular OCTs in right and left eyes of normal people. Proceedings of the Medical Imaging 2014: Biomedical Applications in Molecular, Structural, and Functional Imaging, San Diego, CA, USA.","DOI":"10.1117\/12.2044046"},{"key":"ref_39","first-page":"761901","article-title":"Diabetic retinopathy grading by digital curvelet transform","volume":"2012","author":"Alipour","year":"2012","journal-title":"Comput. Math. Methods Med."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1005","DOI":"10.1049\/iet-ipr.2011.0333","article-title":"Automatic detection of exudates and optic disk in retinal images using curvelet transform","volume":"6","author":"Esmaeili","year":"2012","journal-title":"IET Image Process."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1607","DOI":"10.1007\/s00417-012-2093-6","article-title":"Analysis of foveal avascular zone for grading of diabetic retinopathy severity based on curvelet transform","volume":"250","author":"Alipour","year":"2012","journal-title":"Graefe\u2019s Arch. Clin. Exp. Ophthalmol."},{"key":"ref_42","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the 5th International Conference on Neural Information Processing Systems, Lake Tahoe, CA, USA."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Bengio, Y. (2012). Practical recommendations for gradient based training of deep architectures. Neural Networks: Tricks of the Trade, Springer.","DOI":"10.1007\/978-3-642-35289-8_26"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"K\u00f6the, U. (2003). Edge and junction detection with an improved structure tensor. Jt. Pattern Recognit. Symp., 25\u201332.","DOI":"10.1007\/978-3-540-45243-0_4"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/13\/2970\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:02:56Z","timestamp":1760187776000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/13\/2970"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,5]]},"references-count":44,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2019,7]]}},"alternative-id":["s19132970"],"URL":"https:\/\/doi.org\/10.3390\/s19132970","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,5]]}}}