{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T22:18:00Z","timestamp":1782512280089,"version":"3.54.5"},"reference-count":45,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2023,9,29]],"date-time":"2023-09-29T00:00:00Z","timestamp":1695945600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"JSPS KAKENHI","doi-asserted-by":"publisher","award":["JP22K12095"],"award-info":[{"award-number":["JP22K12095"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>It is important for food recognition to separate each ingredient within a food image at the pixel level. Most existing research has trained a segmentation network on datasets with pixel-level annotations to achieve food ingredient segmentation. However, preparing such datasets is exceedingly hard and time-consuming. In this paper, we propose a new framework for ingredient segmentation utilizing feature maps of the CNN-based Single-Ingredient Classification Model that is trained on the dataset with image-level annotation. To train this model, we first introduce a standardized biological-based hierarchical ingredient structure and construct a single-ingredient image dataset based on this structure. Then, we build a single-ingredient classification model on this dataset as the backbone of the proposed framework. In this framework, we extract feature maps from the single-ingredient classification model and propose two methods for processing these feature maps for segmenting ingredients in the food images. We introduce five evaluation metrics (IoU, Dice, Purity, Entirety, and Loss of GTs) to assess the performance of ingredient segmentation in terms of ingredient classification. Extensive experiments demonstrate the effectiveness of the proposed method, achieving a mIoU of 0.65, mDice of 0.77, mPurity of 0.83, mEntirety of 0.80, and mLoGTs of 0.06 for the optimal model on the FoodSeg103 dataset. We believe that our approach lays the foundation for subsequent ingredient recognition.<\/jats:p>","DOI":"10.3390\/jimaging9100205","type":"journal-article","created":{"date-parts":[[2023,9,29]],"date-time":"2023-09-29T09:39:32Z","timestamp":1695980372000},"page":"205","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["A New CNN-Based Single-Ingredient Classification Model and Its Application in Food Image Segmentation"],"prefix":"10.3390","volume":"9","author":[{"given":"Ziyi","family":"Zhu","sequence":"first","affiliation":[{"name":"Faculty of Software and Information Science, Iwate Prefectural University, Takizawa, Iwate 020-0693, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6987-8212","authenticated-orcid":false,"given":"Ying","family":"Dai","sequence":"additional","affiliation":[{"name":"Faculty of Software and Information Science, Iwate Prefectural University, Takizawa, Iwate 020-0693, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3329168","article-title":"A Survey on Food Computing","volume":"52","author":"Min","year":"2018","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kagaya, H., Aizawa, K., and Ogawa, M. (2014, January 3). Food Detection and Recognition using Convolutional Neural Network. Proceedings of the 22nd ACM international conference on Multimedia, New York, NY, USA.","DOI":"10.1145\/2647868.2654970"},{"key":"ref_3","unstructured":"Aguilar, E., Bola\u00f1os, M., and Radeva, P. (2017). Image Analysis and Processing-ICIAP 2017: 19th International Conference, Catania, Italy, 11\u201315 September 2017, Springer International Publishing. Proceedings, Part II 19."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"35370","DOI":"10.1109\/ACCESS.2019.2904519","article-title":"Vision-Based Approaches for Automatic Food Recognition and Dietary Assessment: A Survey","volume":"7","author":"Subhi","year":"2019","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1926","DOI":"10.1109\/JBHI.2020.2987943","article-title":"Image-Based Food Classification and Volume Estimation for Dietary Assessment: A Review","volume":"24","author":"Lo","year":"2020","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Martinel, N., Foresti, G.L., and Micheloni, C. (2018, January 12\u201315). Wide-slice residual networks for food recognition. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.","DOI":"10.1109\/WACV.2018.00068"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zhou, F., and Lin, Y. (2016, January 27\u201330). Fine-grained image classification by exploring bipartite-graph labels. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Lake Tahoe, NV, USA.","DOI":"10.1109\/CVPR.2016.127"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Min, W., Liu, L., Luo, Z., and Jiang, S. (2019, January 15). Ingredient-guided cascaded multi-attention network for food recognition. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.","DOI":"10.1145\/3343031.3350948"},{"key":"ref_9","unstructured":"Qiu, J., Lo, F.P., Sun, Y., Wang, S., and Lo, B.P. (2019, January 9\u201312). Mining Discriminative Food Regions for Accurate Food Recognition. Proceedings of the British Machine Vision Conference, Cardiff, UK."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Bola\u00f1os, M., Ferr\u00e0, A., and Radeva, P. (2017). Food Ingredients Recognition Through Multi-label Learning. arXiv.","DOI":"10.1007\/978-3-319-70742-6_37"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Gao, J., Chen, J., Fu, H., and Jiang, Y. (2022). Dynamic Mixup for Multi-Label Long-Tailed Food Ingredient Recognition. IEEE Trans. Multimed.","DOI":"10.1109\/TMM.2022.3181789"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1514","DOI":"10.1109\/TIP.2020.3045639","article-title":"A Study of Multi-task and Region-Wise Deep Learning for Food Ingredient Recognition","volume":"30","author":"Chen","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Xue, Y., Niu, K., and He, Z. (2021, January 22\u201324). Region-Level Attention Network for Food and Ingredient Joint Recognition. Proceedings of the 2021 4th International Conference on Algorithms, Computing and Artificial Intelligence, Sanya, China.","DOI":"10.1145\/3508546.3508577"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Chen, J., Pan, L., Wei, Z., Wang, X., Ngo, C., and Chua, T. (2020, January 7\u201312). Zero-Shot Ingredient Recognition by Multi-Relational Graph Convolutional Network. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i07.6626"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wu, X., Fu, X., Liu, Y., Lim, E., Hoi, S.C., and Sun, Q. (2021, January 20\u201324). A Large-Scale Benchmark for Food Image Segmentation. Proceedings of the 29th ACM International Conference on Multimedia, New York, NY, USA.","DOI":"10.1145\/3474085.3475201"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wang, Q., Dong, X., Wang, R., and Sun, H. (2022, January 10\u201312). Swin Transformer Based Pyramid Pooling Network for Food Segmentation. Proceedings of the 2022 IEEE 2nd International Conference on Software Engineering and Artificial Intelligence (SEAI), Xiamen, China.","DOI":"10.1109\/SEAI55746.2022.9832133"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"13152","DOI":"10.1109\/ACCESS.2023.3243243","article-title":"HSIFoodIngr-64: A Dataset for Hyperspectral Food-Related Studies and a Benchmark Method on Food Ingredient Retrieval","volume":"11","author":"Xia","year":"2023","journal-title":"IEEE Access"},{"key":"ref_18","unstructured":"Romero-Tapiador, S., Tolosana, R., Morales, A., Espinosa-Salinas, I., Freixer, G., Fierrez, J., Vera-Rodr\u00edguez, R., Ortega-Garcia, J., Pau, E.C., and Molina, A.R. (2022). AI4Food-NutritionDB: Food Image Database, Nutrition Taxonomy, and Recognition Benchmark. arXiv."},{"key":"ref_19","unstructured":"(2021, January 10). \u751f\u9bae\u98df\u54c1\u54c1\u8cea\u8868\u793a\u57fa\u6e96 (Standards for Fresh Food Quality Labeling). Available online: https:\/\/www.caa.go.jp\/policies\/policy\/food_labeling\/quality\/quality_labelling_standard\/pdf\/kijun_01.pdf."},{"key":"ref_20","unstructured":"(2021). \u65b0\u98df\u54c1\u6210\u5206\u8868 FOODS 2021, \u65b0\u98df\u54c1\u6210\u5206\u8868\u7de8\u96c6\u59d4\u54e1\u4f1a (New Food Ingredients List FOODS 2021), Tokyo Horei Publishing."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3266","DOI":"10.1109\/TMM.2018.2831627","article-title":"Grab, Pay, and Eat: Semantic Food Detection for Smart Restaurants","volume":"20","author":"Aguilar","year":"2017","journal-title":"IEEE Trans. Multimed."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sharma, U., Artacho, B., and Savakis, A. (2021). Gourmetnet: Food segmentation using multi-scale waterfall features with spatial and channel attention. Sensors, 21.","DOI":"10.3390\/s21227504"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Okamoto, K., Adachi, K., and Yanai, K. (2021, January 21). Region-Based Food Calorie Estimation for Multiple-Dish Meals. Proceedings of the 13th International Workshop on Multimedia for Cooking and Eating Activities, Barcelona, Spain.","DOI":"10.1145\/3463947.3469236"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Liang, Y., Li, J., Zhao, Q., Rao, W., Zhang, C., and Wang, C. (2022, January 16\u201319). Image Segmentation and Recognition for Multi-Class Chinese Food. Proceedings of the 2022 IEEE International Conference on Image Processing (ICIP), Bordeaux, France.","DOI":"10.1109\/ICIP46576.2022.9898001"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., and Girshick, R. (2023). Segment Anything. arXiv.","DOI":"10.1109\/ICCV51070.2023.00371"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chen, J., and Ngo, C. (2016, January 15\u201319). Deep-based Ingredient Recognition for Cooking Recipe Retrieval. Proceedings of the 24th ACM international conference on Multimedia, Amsterdam, The Netherlands.","DOI":"10.1145\/2964284.2964315"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Min, W., Liu, L., Wang, Z., Luo, Z., Wei, X., Wei, X., and Jiang, S. (2020, January 12\u201316). Isia food-500: A dataset for large-scale food recognition via stacked global-local attention network. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.","DOI":"10.1145\/3394171.3414031"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Myers, A., Johnston, N., Rathod, V., Korattikara, A., Gorban, A., Silberman, N., Guadarrama, S., Papandreou, G., Huang, J., and Murphy, K. (2015, January 7\u201313). Im2Calories: Towards an Automated Mobile Vision Food Diary. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.146"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Okamoto, K., and Yanai, K. (2020, January 10\u201315). UEC-FoodPix Complete: A Large-Scale Food Image Segmentation Dataset. Proceedings of the ICPR Workshops, Virtual Event.","DOI":"10.1007\/978-3-030-68821-9_51"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1007\/s11390-016-1642-6","article-title":"Multi-Task Learning for Food Identification and Analysis with Deep Convolutional Neural Networks","volume":"31","author":"Zhang","year":"2016","journal-title":"J. Comput. Sci. Technol."},{"key":"ref_31","unstructured":"Crawshaw, M. (2020). Multi-task learning with deep neural networks: A survey. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3551","DOI":"10.1109\/TMM.2020.3028478","article-title":"MVANet: Multi-Task Guided Multi-View Attention Network for Chinese Food Recognition","volume":"23","author":"Liang","year":"2021","journal-title":"IEEE Trans. Multimed."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Dai, J., He, K., and Sun, J. (2016, January 27\u201330). Instance-Aware Semantic Segmentation via Multi-task Network Cascades. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.343"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Cipolla, R., Gal, Y., and Kendall, A. (2018, January 18\u201323). Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00781"},{"key":"ref_35","unstructured":"Li, X., Zhou, Y., Zhou, Y., and Wang, W. (2021). International Conference on Artificial Neural Networks, Springer."},{"key":"ref_36","unstructured":"Victor, S., Thomas, W., and Sebastian, R. (2019, January 27\u201328). A hierarchical multi-task approach for learning embeddings from semantic tasks. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"764","DOI":"10.1016\/j.procs.2015.06.090","article-title":"Image segmentation using K-means clustering algorithm and subtractive clustering algorithm","volume":"54","author":"Dhanachandra","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13640-018-0309-3","article-title":"Image segmentation based on adaptive K-means algorithm","volume":"2018","author":"Zheng","year":"2018","journal-title":"EURASIP J. Image Video Process."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Caron, M., Bojanowski, P., Joulin, A., and Douze, M. (2018, January 8\u201314). Deep Clustering for Unsupervised Learning of Visual Features. Proceedings of the European Conference on Computer Vision, Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_9"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Van Gansbeke, W., Vandenhende, S., Georgoulis, S., and Van Gool, L. (2021, January 11\u201317). Unsupervised semantic segmentation by contrasting object mask proposals. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00990"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zhu, Z., and Dai, Y. (2022, January 2\u20138). CNN-based visible ingredient segmentation in food images for food ingredient recognition. Proceedings of the 2022 12th International Congress on Advanced Applied Informatics (IIAI-AAI), Kanazawa, Japan.","DOI":"10.1109\/IIAIAAI55812.2022.00077"},{"key":"ref_42","unstructured":"Tan, M., and Le, Q. (2019, January 28). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Wang, Y., Liu, C., Zhu, F., Boushey, C.J., and Delp, E.J. (2016, January 25\u201328). Efficient superpixel based segmentation for food image analysis. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7532818"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"108380","DOI":"10.1016\/j.compeleceng.2022.108380","article-title":"Bayesian deep learning for semantic segmentation of food images","volume":"103","author":"Aguilar","year":"2022","journal-title":"Comput. Electr. Eng."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/9\/10\/205\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:02:12Z","timestamp":1760130132000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/9\/10\/205"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,29]]},"references-count":45,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["jimaging9100205"],"URL":"https:\/\/doi.org\/10.3390\/jimaging9100205","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,29]]}}}