{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T17:58:33Z","timestamp":1781287113704,"version":"3.54.1"},"reference-count":53,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2020,6,12]],"date-time":"2020-06-12T00:00:00Z","timestamp":1591920000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Institute of Diabetes and Digestive and Kidney Diseases","award":["K23 DK117061 \u2212 01A1"],"award-info":[{"award-number":["K23 DK117061 \u2212 01A1"]}]},{"name":"University of Virginia, Engineering in Medicine","award":["SS & DEB"],"award-info":[{"award-number":["SS & DEB"]}]},{"name":"University of Virginia Translational Health Research Institute of Virginia (THRIV)","award":["AA"],"award-info":[{"award-number":["AA"]}]},{"DOI":"10.13039\/100000865","name":"Bill and Melinda Gates Foundation","doi-asserted-by":"publisher","award":["OPP1138727; 367 SRM, OPP1144149; PK, OPP1066118"],"award-info":[{"award-number":["OPP1138727; 367 SRM, OPP1144149; PK, OPP1066118"]}],"id":[{"id":"10.13039\/100000865","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Image classification is central to the big data revolution in medicine. Improved information processing methods for diagnosis and classification of digital medical images have shown to be successful via deep learning approaches. As this field is explored, there are limitations to the performance of traditional supervised classifiers. This paper outlines an approach that is different from the current medical image classification tasks that view the issue as multi-class classification. We performed a hierarchical classification using our Hierarchical Medical Image classification (HMIC) approach. HMIC uses stacks of deep learning models to give particular comprehension at each level of the clinical picture hierarchy. For testing our performance, we use biopsy of the small bowel images that contain three categories in the parent level (Celiac Disease, Environmental Enteropathy, and histologically normal controls). For the child level, Celiac Disease Severity is classified into 4 classes (I, IIIa, IIIb, and IIIC).<\/jats:p>","DOI":"10.3390\/info11060318","type":"journal-article","created":{"date-parts":[[2020,6,15]],"date-time":"2020-06-15T03:17:32Z","timestamp":1592191052000},"page":"318","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":66,"title":["HMIC: Hierarchical Medical Image Classification, A Deep Learning Approach"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6451-4786","authenticated-orcid":false,"given":"Kamran","family":"Kowsari","sequence":"first","affiliation":[{"name":"Department of Systems and Information Engineering, University of Virginia, Charlottesville, VA 22904, USA"},{"name":"Office of Health Informatics and Analytics, University of California, Los Angeles (UCLA), CA 90095, USA"},{"name":"Sensing Systems for Health Lab, University of Virginia, Charlottesville, VA 22911, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3995-0735","authenticated-orcid":false,"given":"Rasoul","family":"Sali","sequence":"additional","affiliation":[{"name":"Department of Systems and Information Engineering, University of Virginia, Charlottesville, VA 22904, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6307-2106","authenticated-orcid":false,"given":"Lubaina","family":"Ehsan","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, School of Medicine, University of Virginia, Charlottesville, VA 22903, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William","family":"Adorno","sequence":"additional","affiliation":[{"name":"Department of Systems and Information Engineering, University of Virginia, Charlottesville, VA 22904, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Asad","family":"Ali","sequence":"additional","affiliation":[{"name":"Department of Paediatrics and Child Health, The Aga Khan University, Karachi 74800, Pakistan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1150-6632","authenticated-orcid":false,"given":"Sean","family":"Moore","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, School of Medicine, University of Virginia, Charlottesville, VA 22903, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Beatrice","family":"Amadi","sequence":"additional","affiliation":[{"name":"Tropical Gastroenterology and Nutrition Group, University of Zambia School of Medicine, 32379 Lusak, Zambia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0844-6448","authenticated-orcid":false,"given":"Paul","family":"Kelly","sequence":"additional","affiliation":[{"name":"Tropical Gastroenterology and Nutrition Group, University of Zambia School of Medicine, 32379 Lusak, Zambia"},{"name":"Blizard Institute, Barts and The London School of Medicine, Queen Mary University of London, London E1 4NS, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0954-0583","authenticated-orcid":false,"given":"Sana","family":"Syed","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, School of Medicine, University of Virginia, Charlottesville, VA 22903, USA"},{"name":"Department of Paediatrics and Child Health, The Aga Khan University, Karachi 74800, Pakistan"},{"name":"School of Data Science, University of Virginia, Charlottesville, VA 22904, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9140-2632","authenticated-orcid":false,"given":"Donald","family":"Brown","sequence":"additional","affiliation":[{"name":"Department of Systems and Information Engineering, University of Virginia, Charlottesville, VA 22904, USA"},{"name":"School of Data Science, University of Virginia, Charlottesville, VA 22904, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Sali, R., Ehsan, L., Kowsari, K., Khan, M., Moskaluk, C.A., Syed, S., and Brown, D.E. (2019). CeliacNet: Celiac Disease Severity Diagnosis on Duodenal Histopathological Images Using Deep Residual Networks. arXiv.","DOI":"10.1109\/BIBM47256.2019.8983270"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kowsari, K., Sali, R., Khan, M.N., Adorno, W., Ali, S.A., Moore, S.R., Amadi, B.C., Kelly, P., Syed, S., and Brown, D.E. (2019). Diagnosis of celiac disease and environmental enteropathy on biopsy images using color balancing on convolutional neural networks. Proceedings of the Future Technologies Conference, Springer.","DOI":"10.1007\/978-3-030-32520-6_55"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Kowsari, K. (2020). Diagnosis and Analysis of Celiac Disease and Environmental Enteropathy on Biopsy Images using Deep Learning Approaches. [Ph.D. Thesis, University of California].","DOI":"10.1007\/978-3-030-32520-6_55"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., and Brown, D. (2019). Text Classification Algorithms: A Survey. Information, 10.","DOI":"10.3390\/info10040150"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Nobles, A.L., Glenn, J.J., Kowsari, K., Teachman, B.A., and Barnes, L.E. (2018, January 21\u201326). Identification of imminent suicide risk among young adults using text messages. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/3173574.3173987"},{"key":"ref_7","unstructured":"Zhai, S., Cheng, Y., Zhang, Z.M., and Lu, W. (2016, January 5\u201310). Doubly convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1016\/j.bbe.2019.01.005","article-title":"Comparison of traditional image processing and deep learning approaches for classification of white blood cells in peripheral blood smear images","volume":"39","author":"Hegde","year":"2019","journal-title":"Biocybern. Biomed. Eng."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"65333","DOI":"10.1109\/ACCESS.2018.2875677","article-title":"Patient2Vec: A Personalized Interpretable Deep Representation of the Longitudinal Electronic Health Record","volume":"6","author":"Zhang","year":"2018","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"352","DOI":"10.1093\/ndt\/gfs460","article-title":"Secreted Klotho and FGF23 in chronic kidney disease Stage 1 to 5: A sequence suggested from a cross-sectional study","volume":"28","author":"Pavik","year":"2013","journal-title":"Nephrol. Dial. Transplant."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kowsari, K., Brown, D.E., Heidarysafa, M., Meimandi, K.J., Gerber, M.S., and Barnes, L.E. (2017, January 18\u201321). Hdltex: Hierarchical deep learning for text classification. Proceedings of the 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), Cancun, Mexico.","DOI":"10.1109\/ICMLA.2017.0-134"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dumais, S., and Chen, H. (2000, January 24\u201328). Hierarchical classification of web content. Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Athens, Greece.","DOI":"10.1145\/345508.345593"},{"key":"ref_13","unstructured":"Yan, Z., Piramuthu, R., Jagadeesh, V., Di, W., and Decoste, D. (2019). Hierarchical Deep Convolutional Neural Network for Image Classification. (10,387,773), U.S. Patent."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1016\/j.eswa.2018.09.022","article-title":"Hierarchical convolutional neural networks for fashion image classification","volume":"116","author":"Seo","year":"2019","journal-title":"Expert Syst. Appl."},{"key":"ref_15","unstructured":"Ranjan, N., Machingal, P.V., Jammalmadka, S.S.D., Thenaknidiyoor, V., and Dileep, A. (2019, January 10). Hierarchical Approach for Breast cancer Histopathology Images Classification. Available online: https:\/\/openreview.net\/forum?id=rJlGvTojG."},{"key":"ref_16","unstructured":"Zhu, X., and Bain, M. (2017). B-CNN: Branch convolutional neural network for hierarchical classification. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sali, R., Adewole, S., Ehsan, L., Denson, L.A., Kelly, P., Amadi, B.C., Holtz, L., Ali, S.A., Moore, S.R., and Syed, S. (2020). Hierarchical Deep Convolutional Neural Networks for Multi-category Diagnosis of Gastrointestinal Disorders on Histopathological Images. arXiv.","DOI":"10.1109\/ICHI48887.2020.9374332"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1097\/MPG.0000000000001147","article-title":"Environmental enteric dysfunction in children: A review","volume":"63","author":"Syed","year":"2016","journal-title":"J. Pediatr. Gastroenterol. Nutr."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1759","DOI":"10.1016\/j.ebiom.2015.09.036","article-title":"Environmental enteropathy, oral vaccine failure and growth faltering in infants in Bangladesh","volume":"2","author":"Naylor","year":"2015","journal-title":"EBioMedicine"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1097\/MPG.0b013e31821a23d0","article-title":"European Society for Pediatric Gastroenterology, Hepatology, and Nutrition guidelines for the diagnosis of coeliac disease","volume":"54","author":"Husby","year":"2012","journal-title":"J. Pediatr. Gastroenterol. Nutr."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"636","DOI":"10.1053\/gast.2001.22123","article-title":"Current approaches to diagnosis and treatment of celiac disease: An evolving spectrum","volume":"120","author":"Fasano","year":"2001","journal-title":"Gastroenterology"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., and Saltz, J.H. (2016, January 27\u201330). Patch-based convolutional neural network for whole slide tissue image classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.266"},{"key":"ref_23","unstructured":"Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep Learning, MIT Press."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, W., Huang, Y., Wang, Y., and Wang, L. (2014, January 23\u201328). Generalized autoencoder: A neural network framework for dimensionality reduction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, OH, USA.","DOI":"10.1109\/CVPRW.2014.79"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1985). Learning Internal Representations by Error Propagation, California Univ San Diego La Jolla Inst for Cognitive Science. Technical Report.","DOI":"10.21236\/ADA164453"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1186\/s13638-017-0993-1","article-title":"Text feature extraction based on deep learning: A review","volume":"2017","author":"Liang","year":"2017","journal-title":"EURASIP J. Wirel. Commun. Netw."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Masci, J., Meier, U., Cire\u015fan, D., and Schmidhuber, J. (2011). Stacked convolutional auto-encoders for hierarchical feature extraction. International Conference on Artificial Neural Networks, Springer.","DOI":"10.1007\/978-3-642-21735-7_7"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Chen, K., Seuret, M., Liwicki, M., Hennebert, J., and Ingold, R. (2015). Page segmentation of historical document images with convolutional autoencoders. Proceedings of the 2015 13th International Conference on Document Analysis and Recognition (ICDAR), IEEE.","DOI":"10.1109\/ICDAR.2015.7333914"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2351","DOI":"10.1109\/LGRS.2015.2478256","article-title":"High-resolution SAR image classification via deep convolutional autoencoders","volume":"12","author":"Geng","year":"2015","journal-title":"IEEE Geosci. Remote. Sens. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1016\/j.patrec.2009.09.011","article-title":"Data clustering: 50 years beyond K-means","volume":"31","author":"Jain","year":"2010","journal-title":"Pattern Recognit. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gao, Q., Xu, H.X., Han, H.G., and Guo, M. (2019, January 27\u201330). Soft-sensor Method for Surface Water Qualities Based on Fuzzy Neural Network. Proceedings of the 2019 Chinese Control Conference (CCC), Guangzhou, China.","DOI":"10.23919\/ChiCC.2019.8866494"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kowsari, K., Yammahi, M., Bari, N., Vichr, R., Alsaby, F., and Berkovich, S.Y. (2015). Construction of fuzzyfind dictionary using golay coding transformation for searching applications. arXiv.","DOI":"10.14569\/IJACSA.2015.060313"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kowsari, K., and Alassaf, M.H. (2016). Weighted unsupervised learning for 3d object detection. arXiv.","DOI":"10.14569\/IJACSA.2016.070180"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Alassaf, M.H., Kowsari, K., and Hahn, J.K. (2015, January 22\u201324). Automatic, real time, unsupervised spatio-temporal 3d object detection using rgb-d cameras. Proceedings of the 2015 19th International Conference on Information Visualisation, Barcelona, Spain.","DOI":"10.1109\/iV.2015.80"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Manning, C.D., Raghavan, P., and Schutze, H. (2008). Introduction to Information Retrieval, Cambridge University Press.","DOI":"10.1017\/CBO9780511809071"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Das, S., and Uehara, R. (2009). The Planar k-Means Problem is NP-Hard. WALCOM: Algorithms and Computation, Springer.","DOI":"10.1007\/978-3-642-00202-1"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"pdb","DOI":"10.1101\/pdb.prot4986","article-title":"Hematoxylin and eosin staining of tissue and cell sections","volume":"2008","author":"Fischer","year":"2008","journal-title":"Cold Spring Harb. Protoc."},{"key":"ref_38","first-page":"2014","article-title":"An introduction to Routine and special staining","volume":"18","author":"Anderson","year":"2011","journal-title":"Retrieved August"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1729","DOI":"10.1109\/TBME.2014.2303294","article-title":"A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution","volume":"61","author":"Khan","year":"2014","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Bianco, S., Cusano, C., Napoletano, P., and Schettini, R. (2017). Improving CNN-Based Texture Classification by Color Balancing. J. Imaging, 3.","DOI":"10.3390\/jimaging3030033"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1007\/s10851-014-0496-1","article-title":"Error-tolerant color rendering for digital cameras","volume":"50","author":"Bianco","year":"2014","journal-title":"J. Math. Imaging Vis."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1962","DOI":"10.1109\/TMI.2016.2529665","article-title":"Structure-preserving color normalization and sparse stain separation for histological images","volume":"35","author":"Vahadane","year":"2016","journal-title":"IEEE Trans. Med Imaging"},{"key":"ref_43","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel."},{"key":"ref_44","unstructured":"Kowsari, K., Heidarysafa, M., Brown, D.E., Meimandi, K.J., and Barnes, L.E. (2018, January 9\u201311). Rmdl: Random multimodel deep learning for classification. Proceedings of the 2nd International Conference on Information System and Data Mining, Lakeland, FL, USA."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Li, Q., Cai, W., Wang, X., Zhou, Y., Feng, D.D., and Chen, M. (2014, January 10\u201312). Medical image classification with convolutional neural network. Proceedings of the 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV), Singapore.","DOI":"10.1109\/ICARCV.2014.7064414"},{"key":"ref_46","unstructured":"Heidarysafa, M., Kowsari, K., Brown, D.E., Jafari Meimandi, K., and Barnes, L.E. (2018). An Improvement of Data Classification Using Random Multimodel Deep Learning (RMDL). arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Scherer, D., M\u00fcller, A., and Behnke, S. (2010, January 15\u201318). Evaluation of pooling operations in convolutional architectures for object recognition. Proceedings of the Artificial Neural Networks\u2013ICANN 2010, Thessaloniki, Greece.","DOI":"10.1007\/978-3-642-15825-4_10"},{"key":"ref_48","unstructured":"Kingma, D., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_49","unstructured":"Chollet, F. (2019, August 19). Keras: Deep Learning Library for Theano and Tensorflow. Available online: https:\/\/keras.io\/."},{"key":"ref_50","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1023\/A:1009982220290","article-title":"An evaluation of statistical approaches to text categorization","volume":"1","author":"Yang","year":"1999","journal-title":"Inf. Retr."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1038\/nmeth.3945","article-title":"Points of significance: Classification evaluation","volume":"13","author":"Lever","year":"2016","journal-title":"Nat. Methods"},{"key":"ref_53","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/6\/318\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:38:20Z","timestamp":1760175500000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/6\/318"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,12]]},"references-count":53,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["info11060318"],"URL":"https:\/\/doi.org\/10.3390\/info11060318","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,12]]}}}