{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T02:52:53Z","timestamp":1780887173783,"version":"3.54.1"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"21-23","license":[{"start":{"date-parts":[[2021,8,12]],"date-time":"2021-08-12T00:00:00Z","timestamp":1628726400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,8,12]],"date-time":"2021-08-12T00:00:00Z","timestamp":1628726400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004252","name":"Qatar University","doi-asserted-by":"publisher","award":["QUCG-CED-20\/21-2"],"award-info":[{"award-number":["QUCG-CED-20\/21-2"]}],"id":[{"id":"10.13039\/501100004252","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004252","name":"Qatar University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004252","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Multimed Tools Appl"],"published-print":{"date-parts":[[2021,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>People with blindness or low vision utilize mobile assistive tools for various applications such as object recognition, text recognition, etc. Most of the available applications are focused on recognizing generic objects. And they have not addressed the recognition of food dishes and fruit varieties. In this paper, we propose a smartphone-based system for recognizing the food dishes as well as fruits for children with visual impairments. The Smartphone application utilizes a trained deep CNN model for recognizing the food item from the real-time images. Furthermore, we develop a new deep convolutional neural network (CNN) model for food recognition using the fusion of two CNN architectures. The new deep CNN model is developed using the ensemble learning approach. The deep CNN food recognition model is trained on a customized food recognition dataset.The customized food recognition dataset consists of 29 varieties of food dishes and fruits. Moreover, we analyze the performance of multiple state of art deep CNN models for food recognition using the transfer learning approach. The ensemble model performed better than state of art CNN models and achieved a food recognition accuracy of 95.55 % in the customized food dataset. In addition to that, the proposed deep CNN model is evaluated in two publicly available food datasets to display its efficacy for food recognition tasks.<\/jats:p>","DOI":"10.1007\/s11042-021-11329-6","type":"journal-article","created":{"date-parts":[[2021,8,12]],"date-time":"2021-08-12T19:02:21Z","timestamp":1628794941000},"page":"33011-33032","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":63,"title":["Smartphone-based food recognition system using multiple deep CNN models"],"prefix":"10.1007","volume":"80","author":[{"given":"Abdulnaser","family":"Fakhrou","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8972-0893","authenticated-orcid":false,"given":"Jayakanth","family":"Kunhoth","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Somaya","family":"Al Maadeed","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,8,12]]},"reference":[{"key":"11329_CR1","unstructured":"(2011) Vision impairment and blindness.\u00a0https:\/\/www.who.int\/newsroom\/fact-sheets\/detail\/blindness-and-visualimpairment. Accessed 22 Jan 2021"},{"key":"11329_CR2","unstructured":"(2020) Custom food dataset. https:\/\/www.dropbox.com\/sh\/irxb953mt9od181\/AAD8QLXzuBZpdMdsKyFEWFwFa?dl=0. Accessed 22 Jan 2021"},{"key":"11329_CR3","doi-asserted-by":"publisher","first-page":"360","DOI":"10.1016\/j.jvcir.2019.03.011","volume":"60","author":"E Aguilar","year":"2019","unstructured":"Aguilar E, Bola\u00f1os M, Radeva P (2019) Regularized uncertainty-based multi-task learning model for food analysis. J Vis Commun Image Represent 60:360\u2013370","journal-title":"J Vis Commun Image Represent"},{"key":"11329_CR4","series-title":"pp","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1007\/978-3-030-31332-6_6","volume-title":"Pattern Recognition and Image Analysis","author":"E Aguilar","year":"2019","unstructured":"Aguilar E, Radeva P (2019) Food recognition by integrating local and flat classifiers. In: Morales A, Fierrez J, S\u00e1nchez JS, Ribeiro B (eds) Pattern Recognition and Image Analysis. pp. Springer International Publishing, Cham, pp 65\u201374"},{"issue":"4","key":"11329_CR5","doi-asserted-by":"publisher","first-page":"1261","DOI":"10.1109\/JBHI.2014.2308928","volume":"18","author":"MM Anthimopoulos","year":"2014","unstructured":"Anthimopoulos MM, Gianola L, Scarnato L, Diem P, Mougiakakou SG (2014) A food recognition system for diabetic patients based on an optimized bag-of-features model. IEEE J Biomed Health Inform 18(4):1261\u20131271","journal-title":"IEEE J Biomed Health Inform"},{"key":"11329_CR6","doi-asserted-by":"crossref","unstructured":"Bagwan SMR, Sankpal L (2015) Visualpal: A mobile app for object recognition for the visually impaired. In 2015 International Conference on Computer, Communication and Control (IC4). Indore, India, pp 1\u20136","DOI":"10.1109\/IC4.2015.7375665"},{"key":"11329_CR7","doi-asserted-by":"crossref","unstructured":"Bashiri FS, LaRose E, Badger JC, D\u2019Souza RM, Yu Z, Peissig P (2018) Object detection to assist visually impaired people: A deep neural network adventure. In Advances in Visual Computing. Springer International Publishing, Cham, pp 500\u2013510","DOI":"10.1007\/978-3-030-03801-4_44"},{"key":"11329_CR8","doi-asserted-by":"crossref","unstructured":"Bossard L, Guillaumin M, Van\u00a0Gool L (2014) Food-101\u2013mining discriminative components with random forests. In European conference on computer vision. Springer, Zurich, Switzerland,\u00a0pp 446\u2013461","DOI":"10.1007\/978-3-319-10599-4_29"},{"key":"11329_CR9","doi-asserted-by":"crossref","unstructured":"Chincha R, Tian Y (2011) Finding objects for blind people based on surf features. In 2011 IEEE International Conference on Bioinformatics and Biomedicine Workshops (BIBMW). Georgia, USA,\u00a0pp 526\u2013527","DOI":"10.1109\/BIBMW.2011.6112423"},{"key":"11329_CR10","doi-asserted-by":"crossref","unstructured":"Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. Florida, USA\u00a0pp 248\u2013255","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"11329_CR11","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.compbiomed.2016.07.006","volume":"77","author":"GM Farinella","year":"2016","unstructured":"Farinella GM, Allegra D, Moltisanti M, Stanco F, Battiato S (2016) Retrieval and classification of food images. Comput Biol Med 77:23\u201339","journal-title":"Comput Biol Med"},{"key":"11329_CR12","doi-asserted-by":"crossref","unstructured":"Huang G, Liu Z, Van Der\u00a0Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Hawaii, USA\u00a0pp 4700\u20134708","DOI":"10.1109\/CVPR.2017.243"},{"key":"11329_CR13","unstructured":"Hussain I, He Q, Chen Z, Xie W (2018) Fruit recognition dataset (version v 1.0)[dataset]. zenodo"},{"key":"11329_CR14","doi-asserted-by":"crossref","unstructured":"Kagaya H, Aizawa K, Ogawa M (2014) Food detection and recognition using convolutional neural network. In Proceedings of the 22nd ACM International Conference on Multimedia. Florida, USA\u00a0pp 1085\u20131088","DOI":"10.1145\/2647868.2654970"},{"key":"11329_CR15","doi-asserted-by":"crossref","unstructured":"Karkar A, Al-Maadeed S (2018) Mobile assistive technologies for visual impaired users: A survey. In 2018 International Conference on Computer and Applications (ICCA). Beirut, Lebanon,\u00a0pp 427\u2013433","DOI":"10.1109\/COMAPP.2018.8460406"},{"key":"11329_CR16","doi-asserted-by":"crossref","unstructured":"Karkar A, Kunhoth J, Al-Maadeed S (2020) A scene-to-speech mobile based application: Multiple trained models approach. In 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT).\u00a0Doha, Qatar,\u00a0pp 490\u2013497","DOI":"10.1109\/ICIoT48696.2020.9089557"},{"key":"11329_CR17","doi-asserted-by":"crossref","unstructured":"Karkar A, Puthren M, Al-Maadeed S (2018) A bilingual scene-to-speech mobile based application. In 2018 International Conference on Computer and Applications (ICCA).\u00a0Beirut, Lebanon, pp 1\u2013240","DOI":"10.1109\/COMAPP.2018.8460239"},{"key":"11329_CR18","doi-asserted-by":"crossref","unstructured":"Kawano Y, Yanai K (2014) Foodcam-256: a large-scale real-time mobile food recognitionsystem employing high-dimensional features and compression of classifier weights. In Proceedings of the 22nd ACM international conference on Multimedia. Florida, USA,\u00a0pp 761\u2013762","DOI":"10.1145\/2647868.2654869"},{"key":"11329_CR19","doi-asserted-by":"crossref","unstructured":"Kay\u0131k\u00e7\u0131 \u015e, Ba\u015fol Y, D\u00f6rter E (2019) Classification of turkish cuisine with deep learning on mobile platform. In 2019 4th International Conference on Computer Science and Engineering (UBMK).\u00a0Samsun, Turkey,\u00a0pp 1\u20135","DOI":"10.1109\/UBMK.2019.8906992"},{"issue":"1","key":"11329_CR20","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1016\/j.pmcj.2011.07.003","volume":"8","author":"F Kong","year":"2012","unstructured":"Kong F, Tan J (2012) Dietcam: Automatic dietary assessment with mobile camera phones. Pervasive Mob Comput 8(1):147\u2013163","journal-title":"Pervasive Mob Comput"},{"key":"11329_CR21","doi-asserted-by":"crossref","unstructured":"Lanigan PE, Paulos AM, Williams AW, Rossi D, Narasimhan P (2006) Trinetra: Assistive technologies for grocery shopping for the blind. In ISWC. Georgia, USA,\u00a0pp 147\u2013148","DOI":"10.1109\/ISWC.2006.286369"},{"issue":"2","key":"11329_CR22","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1109\/TSC.2017.2662008","volume":"11","author":"C Liu","year":"2017","unstructured":"Liu C, Cao Y, Luo Y, Chen G, Vokkarane V, Yunsheng M, Chen S, Hou P (2017) A new deep learning-based food recognition system for dietary assessment on an edge computing service infrastructure. IEEE Trans Serv Comput 11(2):249\u2013261","journal-title":"IEEE Trans Serv Comput"},{"key":"11329_CR23","doi-asserted-by":"crossref","unstructured":"Martinel N, Foresti GL, Micheloni C (2018) Wide-slice residual networks for food recognition. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Nevada, USA,\u00a0pp 567\u2013576","DOI":"10.1109\/WACV.2018.00068"},{"key":"11329_CR24","doi-asserted-by":"crossref","unstructured":"Matsuda Y, Hoashi H, Yanai K (2012) Recognition of multiple-food images by detecting candidate regions. In 2012 IEEE International Conference on Multimedia and Expo.\u00a0Melbourne, Australia, pp 25\u201330","DOI":"10.1109\/ICME.2012.157"},{"key":"11329_CR25","doi-asserted-by":"crossref","unstructured":"Meyers A, Johnston N, Rathod V, Korattikara A, Gorban A, Silberman N, Guadarrama S, Papandreou G, Huang J, Murphy KP (2015) Im2calories: towards an automated mobile vision food diary. In Proceedings of the IEEE International Conference on Computer Vision.\u00a0Santiago, Chile,\u00a0pp 1233\u20131241","DOI":"10.1109\/ICCV.2015.146"},{"issue":"5","key":"11329_CR26","doi-asserted-by":"publisher","first-page":"1057","DOI":"10.3390\/s17051057","volume":"17","author":"D Nakamura","year":"2017","unstructured":"Nakamura D, Takizawa H, Aoyagi M, Ezaki N, Mizuno S (2017) Smartphone-based escalator recognition for the visually impaired. Sensors 17(5):1057","journal-title":"Sensors"},{"issue":"1","key":"11329_CR27","doi-asserted-by":"publisher","first-page":"11","DOI":"10.2174\/1874943700902010011","volume":"2","author":"J Nicholson","year":"2009","unstructured":"Nicholson J, Kulyukin V, Coster D (2009) Shoptalk: independent blind shopping through verbal route directions and barcode scans. The Open Rehabilitation Journal 2(1):11\u201323","journal-title":"The Open Rehabilitation Journal"},{"issue":"12","key":"11329_CR28","doi-asserted-by":"publisher","first-page":"1758","DOI":"10.1109\/LSP.2017.2758862","volume":"24","author":"P Pandey","year":"2017","unstructured":"Pandey P, Deepthi A, Mandal B, Puhan NB (2017) Foodnet: Recognizing foods using ensemble of deep networks. IEEE Signal Process Lett 24(12):1758\u20131762","journal-title":"IEEE Signal Process Lett"},{"issue":"14","key":"11329_CR29","doi-asserted-by":"publisher","first-page":"5243","DOI":"10.1007\/s11042-014-2116-x","volume":"74","author":"P Pouladzadeh","year":"2015","unstructured":"Pouladzadeh P, Shirmohammadi S, Bakirov A, Bulut A, Yassine A (2015) Cloud-based svm for food categorization. Multimed Tools Appl 74(14):5243\u20135260","journal-title":"Multimed Tools Appl"},{"key":"11329_CR30","unstructured":"Qiu J, Lo FPW, Sun Y, Wang S, Lo B (2019) Mining discriminative food regions for accurate food recognition. In BMVC. Cardiff, UK,\u00a0pp 158\u2013168"},{"key":"11329_CR31","unstructured":"Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition"},{"key":"11329_CR32","doi-asserted-by":"crossref","unstructured":"Singh S, Choudhury S, Vishal K, Jawahar C (2014) Currency recognition on mobile phones. In 2014 22nd International Conference on Pattern Recognition.\u00a0Stockholm, Sweden,\u00a0pp 2661\u20132666","DOI":"10.1109\/ICPR.2014.460"},{"key":"11329_CR33","doi-asserted-by":"crossref","unstructured":"Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA,\u00a0pp 1\u20139","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"11329_CR34","unstructured":"Tan M, Le Q (2019) EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning. PMLR, Long Beach, USA,\u00a0pp 6105\u20136114"},{"key":"11329_CR35","doi-asserted-by":"crossref","unstructured":"Yanai K, Kawano Y (2015) Food image recognition using deep convolutional network with pre-training and fine-tuning. In 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). Turin, Italy,\u00a0pp 1\u20136","DOI":"10.1109\/ICMEW.2015.7169816"},{"key":"11329_CR36","doi-asserted-by":"crossref","unstructured":"Yu F, Wang D, Shelhamer E, Darrell T (2018) Deep layer aggregation. In Proceedings of the IEEE conference on computer vision and pattern recognition. Salt Lake City, USA,\u00a0pp 2403\u20132412","DOI":"10.1109\/CVPR.2018.00255"}],"container-title":["Multimedia Tools and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-021-11329-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11042-021-11329-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-021-11329-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,10,10]],"date-time":"2021-10-10T05:28:28Z","timestamp":1633843708000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11042-021-11329-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,12]]},"references-count":36,"journal-issue":{"issue":"21-23","published-print":{"date-parts":[[2021,9]]}},"alternative-id":["11329"],"URL":"https:\/\/doi.org\/10.1007\/s11042-021-11329-6","relation":{},"ISSN":["1380-7501","1573-7721"],"issn-type":[{"value":"1380-7501","type":"print"},{"value":"1573-7721","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,12]]},"assertion":[{"value":"21 September 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 January 2021","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 July 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 August 2021","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}