{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T00:35:56Z","timestamp":1768437356277,"version":"3.49.0"},"reference-count":46,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2022,6,24]],"date-time":"2022-06-24T00:00:00Z","timestamp":1656028800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["62163007"],"award-info":[{"award-number":["62163007"]}]},{"name":"National Natural Science Foundation of China","award":["[2020]4Y056"],"award-info":[{"award-number":["[2020]4Y056"]}]},{"name":"National Natural Science Foundation of China","award":["PTRC [2020]6007"],"award-info":[{"award-number":["PTRC [2020]6007"]}]},{"name":"National Natural Science Foundation of China","award":["[2021]439"],"award-info":[{"award-number":["[2021]439"]}]},{"name":"National Natural Science Foundation of China","award":["2016[5103]"],"award-info":[{"award-number":["2016[5103]"]}]},{"name":"Science and Technology Foundation of Guizhou Province","award":["62163007"],"award-info":[{"award-number":["62163007"]}]},{"name":"Science and Technology Foundation of Guizhou Province","award":["[2020]4Y056"],"award-info":[{"award-number":["[2020]4Y056"]}]},{"name":"Science and Technology Foundation of Guizhou Province","award":["PTRC [2020]6007"],"award-info":[{"award-number":["PTRC [2020]6007"]}]},{"name":"Science and Technology Foundation of Guizhou Province","award":["[2021]439"],"award-info":[{"award-number":["[2021]439"]}]},{"name":"Science and Technology Foundation of Guizhou Province","award":["2016[5103]"],"award-info":[{"award-number":["2016[5103]"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In order to automatically perceive the user\u2019s dietary nutritional information in the smart home environment, this paper proposes a dietary nutritional information autonomous perception method based on machine vision in smart homes. Firstly, we proposed a food-recognition algorithm based on YOLOv5 to monitor the user\u2019s dietary intake using the social robot. Secondly, in order to obtain the nutritional composition of the user\u2019s dietary intake, we calibrated the weight of food ingredients and designed the method for the calculation of food nutritional composition; then, we proposed a dietary nutritional information autonomous perception method based on machine vision (DNPM) that supports the quantitative analysis of nutritional composition. Finally, the proposed algorithm was tested on the self-expanded dataset CFNet-34 based on the Chinese food dataset ChineseFoodNet. The test results show that the average recognition accuracy of the food-recognition algorithm based on YOLOv5 is 89.7%, showing good accuracy and robustness. According to the performance test results of the dietary nutritional information autonomous perception system in smart homes, the average nutritional composition perception accuracy of the system was 90.1%, the response time was less than 6 ms, and the speed was higher than 18 fps, showing excellent robustness and nutritional composition perception performance.<\/jats:p>","DOI":"10.3390\/e24070868","type":"journal-article","created":{"date-parts":[[2022,6,25]],"date-time":"2022-06-25T10:39:13Z","timestamp":1656153553000},"page":"868","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Dietary Nutritional Information Autonomous Perception Method Based on Machine Vision in Smart Homes"],"prefix":"10.3390","volume":"24","author":[{"given":"Hongyang","family":"Li","sequence":"first","affiliation":[{"name":"Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guizhou University, Guiyang 550025, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8761-5195","authenticated-orcid":false,"given":"Guanci","family":"Yang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guizhou University, Guiyang 550025, China"},{"name":"Key Laboratory of \u201cInternet+\u201d Collaborative Intelligent Manufacturing in Guizhou Province, Guiyang 550025, China"},{"name":"State Key Laboratory of Public Big Data, Guizhou University, Guiyang 550025, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,24]]},"reference":[{"key":"ref_1","unstructured":"Wang, J., and Hou, Y.J. (2021, January 3\u20137). Research on the Development Status and the Trend of Smart Home. Proceedings of the International Conference on Electronic Business, Nanjing, China."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"8768","DOI":"10.1109\/ACCESS.2019.2963560","article-title":"Dietary composition perception algorithm using social robot audition for Mandarin Chinese","volume":"8","author":"Su","year":"2020","journal-title":"IEEE Access"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Yang, G., Chen, Z., Li, Y., and Su, Z. (2019). Rapid relocation method for mobile robot based on improved ORB-SLAM2 algorithm. Remote Sens., 11.","DOI":"10.3390\/rs11020149"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"676","DOI":"10.1002\/acs.3221","article-title":"Hierarchical recursive signal modeling for multifrequency signals based on discrete measured data","volume":"35","author":"Xu","year":"2021","journal-title":"Int. J. Adapt. Control Signal. Process."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3597","DOI":"10.1109\/TCSII.2021.3076112","article-title":"Hierarchical Estimation Approach for RBF-AR Models With Regression Weights Based on the Increasing Data Length","volume":"68","author":"Zhou","year":"2021","journal-title":"IEEE Trans. Circuits Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1109\/LSP.2021.3136504","article-title":"Optimal Adaptive Filtering Algorithm by Using the Fractional-Order Derivative","volume":"29","author":"Zhang","year":"2022","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_7","unstructured":"(2022, April 01). Overweight and Obesity. Available online: https:\/\/www.who.int\/news-room\/fact-sheets\/detail\/obesity-and-overweight."},{"key":"ref_8","unstructured":"Hales, C.M., Carroll, M.D., Fryar, C.D., and Ogden, C.L. (2016). Prevalence of Obesity among Adults and Youth: United States, 2015\u20132016, Centers Disease Control Prevention."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"w822","DOI":"10.1377\/hlthaff.28.5.w822","article-title":"Annual medical spending attributable to obesity: Payer-and service-specifific estimates","volume":"28","author":"Finkelstein","year":"2009","journal-title":"Health Aff."},{"key":"ref_10","unstructured":"(2009). National Vital Statistics System U.S. Quickstats: Age-adjusted death rates for the 10 leading causes of death. Morb. Mortal. Wkly. Rep., 58, 1303."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"47477","DOI":"10.1109\/ACCESS.2020.2973625","article-title":"DeepFood: Food Image Analysis and Dietary Assessment via Deep Model","volume":"8","author":"Jiang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_12","first-page":"3","article-title":"Several conceptual changes in the \u201cHealthy China 2030\u201d planning outline","volume":"31","author":"Zhang","year":"2017","journal-title":"Soft Sci. Health"},{"key":"ref_13","unstructured":"National Health Commission (2020). Report on Nutrition and Chronic Disease Status of Chinese Residents (2020). J. Nutr., 42, 521."},{"key":"ref_14","unstructured":"Yu, B. (2019). Research on Dietary Intervention Methods Based on Multi-Dimensional Characteristics, Xiangtan University."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1926","DOI":"10.1109\/JBHI.2020.2987943","article-title":"Image-based food classification and volume estimation for dietary assessment: A review","volume":"14","author":"Lo","year":"2020","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Gao, A., Lo, F.P.W., and Lo, B. (2018, January 4\u20137). Food volume estimation for quantifying dietary intake with a wearable camera. Proceedings of the IEEE International Conference on Wearable and Implantable Body Sensor Networks (BSN), Las Vegas, NV, USA.","DOI":"10.1109\/BSN.2018.8329671"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"641861","DOI":"10.1260\/2040-2295.6.1.1","article-title":"An exploratory study on a chest-worn computer for evaluation of diet, physical activity and lifestyle","volume":"6","author":"Sun","year":"2015","journal-title":"J. Healthc. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"756","DOI":"10.1109\/JSTSP.2010.2051471","article-title":"The use of mobile devices in aiding dietary assessment and evaluation","volume":"4","author":"Zhu","year":"2010","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Fang, S., Zhu, F., Jiang, C., Zhang, S., Boushey, C.J., and Delp, E.J. (2016, January 25\u201328). A comparison of food portion size estimation using geometric models and depth images. Proceedings of the IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7532312"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lo, F.P.W., Sun, Y., Qiu, J., and Lo, B. (2018). Food volume estimation based on deep learning view synthesis from a single depth map. Nutrients, 10.","DOI":"10.3390\/nu10122005"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Lo, F.P.W., Sun, Y., Qiu, J., and Lo, B. (2019, January 19\u201322). A Novel Vision-based Approach for Dietary Assessment using Deep Learning View Synthesis. Proceedings of the 2019 IEEE 16th International Conference on Wearable and Implantable Body Sensor Networks (BSN), Chicago, IL, USA.","DOI":"10.1109\/BSN.2019.8771089"},{"key":"ref_22","unstructured":"Yu, J. (2020). Diet Monitoring System for Diabetic Patients Based on Near-Infrared Spectral Sensor, Nanjing University of Posts and Telecommunications."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.compbiomed.2016.07.006","article-title":"Retrieval and classification of food images","volume":"77","author":"Farinella","year":"2016","journal-title":"Comput. Biol. Med."},{"key":"ref_24","unstructured":"Chen, M.Y., Yang, Y.H., Ho, C.J., Wang, S.H., Liu, S.M., Chang, E., Yeh, C.H., and Ouhyoung, M. (December, January 28). Automatic Chinese food identification and quantity estimation. Proceedings of the Siggraph Asia Technical Briefs, Singapore."},{"key":"ref_25","unstructured":"Kawano, Y., and Yanai, K. (2014). Automatic expansion of a food image dataset leveraging existing categories with domain adaptation. European Conference on Computer Vision, Springer International Publishing."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Bossard, L., Guillaumin, M., and Gool, L.V. (2014). Food-101: Mining discriminative components with random forests. European Conference on Computer Vision, Springer International Publishing.","DOI":"10.1007\/978-3-319-10599-4_29"},{"key":"ref_27","unstructured":"Chen, X., Zhu, Y., Zhou, H., Diao, L., and Wang, D.Y. (2017). Chinesefoodnet: A large-scale image dataset for chinese food recognition. arXiv."},{"key":"ref_28","unstructured":"Kaur, P., Sikka, K., Wang, W., Belongie, S., and Divakaran, A. (2019). FoodX-251: A dataset for fine-grained food classification. arXiv."},{"key":"ref_29","unstructured":"Matsuda, Y., and Yanai, K. (2012, January 11\u201315). Multiple-food recognition considering co-occurrence employing manifold ranking. Proceedings of the 2012 21st International Conference on Pattern Recognition (ICPR 2012), Tsukuba, Japan."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhu, F., Bosch, M., Schap, T., Khanna, N., Ebert, D.S., Boushey, C.J., and Delp, E.J. (2011). Segmentation Assisted Food Classification for Dietary Assessment. Computational Imaging IX., SPIE.","DOI":"10.1117\/12.877036"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Kong, F., and Tan, J. (2011, January 23\u201325). DietCam: Regular Shape Food Recognition with a Camera Phone. Proceedings of the 2011 International Conference on Body Sensor Networks, Dallas, TX, USA.","DOI":"10.1109\/BSN.2011.19"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Dehais, J., Anthimopoulos, M., and Mougiakakou, S. (2016, January 16). GoCARB: A Smartphone Application for Automatic Assessment of Carbohydrate Intake. Proceedings of the 2nd International Workshop on Multimedia Assisted Dietary Management, Amsterdam, The Netherlands.","DOI":"10.1145\/2986035.2986046"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Meyers, A., Johnston, N., Rathod, V., Korattikara, A., Gorban, A., Silberman, N., Guadarrama, S., Papandreou, G., Huang, J., and Murphy, K.P. (2015, January 7\u201313). Im2Calories: Towards an Automated Mobile Vision Food Diary. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Washington, DC, USA.","DOI":"10.1109\/ICCV.2015.146"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Nabi, J., Doddamadaiah, A.R., and Lakhotia, R. (2015, January 21\u201323). Smart Dietary Monitoring System. Proceedings of the 2015 IEEE International Symposium on Nanoelectronic and Information Systems, Indore, India.","DOI":"10.1109\/iNIS.2015.20"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zenun Franco, R. (2017, January 27\u201331). Online Recommender System for Personalized Nutrition Advice. Proceedings of the Eleventh ACM Conference on Recommender Systems-RecSys \u201817\u2019, ACM, Como, Italy.","DOI":"10.1145\/3109859.3109862"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Doulah, A., Yang, X., Parton, J., Higgins, J.A., McCrory, M.A., and Sazonov, E. (2018, January 18\u201321). The importance of field experiments in testing of sensors for dietary assessment and eating behavior monitoring. Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA.","DOI":"10.1109\/EMBC.2018.8513623"},{"key":"ref_37","first-page":"276","article-title":"Research on the use of YOLOv5 object detection algorithm in mask wearing recognition","volume":"6","author":"Liu","year":"2020","journal-title":"World Sci. Res. J."},{"key":"ref_38","first-page":"224","article-title":"Application and exploration of smart examination room scheme based on deep learning technology","volume":"12","author":"Zhou","year":"2020","journal-title":"Inf. Technol. Informatiz."},{"key":"ref_39","unstructured":"(2021, November 01). Meishij Recipes. Available online: http:\/\/www.meishij.net."},{"key":"ref_40","unstructured":"(2021, November 01). Food Nutrition Inquiry Platform. Available online: http:\/\/yycx.yybq.net\/."},{"key":"ref_41","unstructured":"(2021, November 01). Shi An Tong\u2014Food Safety Inquiry System. Available online: www.eshian.com\/sat\/yyss\/list."},{"key":"ref_42","first-page":"73","article-title":"A Robot Vision Privacy Protection Method Based on Improved Cycle-GAN","volume":"48","author":"Yang","year":"2020","journal-title":"J. Huazhong Univ. Sci. Technol. (Nat. Sci. Ed.)"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1016\/j.neunet.2020.09.001","article-title":"FPGAN: Face de-identification method with generative adversarial networks for social robots","volume":"133","author":"Lin","year":"2021","journal-title":"Neural Netw."},{"key":"ref_44","first-page":"1679","article-title":"Social Robot Vision Privacy Behavior Recognition and Protection System Based on Image Semantics","volume":"32","author":"Li","year":"2020","journal-title":"J. Comput. Aided Des. Graph."},{"key":"ref_45","unstructured":"Zhao, W. (2021). Research on Target Detection Algorithm Based on YOLOv5, Xi\u2019an University of Electronic Science and Technology."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8\u201314). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/7\/868\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:39:27Z","timestamp":1760139567000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/7\/868"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,24]]},"references-count":46,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["e24070868"],"URL":"https:\/\/doi.org\/10.3390\/e24070868","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,24]]}}}