{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T05:50:04Z","timestamp":1791352204127,"version":"4.3.1"},"reference-count":47,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T00:00:00Z","timestamp":1791331200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Nutr."],"abstract":"<jats:p>Diet quality is central to preventing non-communicable diseases, yet traditional dietary assessment methods are burdensome and error-prone. Estimating nutrients directly from food images offers a scalable alternative, but most methods treat a meal as a single visual entity and estimate calories and mass more accurately than individual macronutrients. It has not yet been systematically evaluated whether decomposing a meal into its ingredients - predicting nutrients per ingredient and aggregating to the dish level - improves estimation. We compared three input representations for nutrient regression from overhead food images: direct regression from the original image, regression from binary food-region masks, and regression from fine-grained ingredient-level masks. To enable this, we annotated a 1,056-image subset of the Nutrition5K dataset with 2,922 ingredient-level segmentation masks across 40 categories, and used an identical regression model for all conditions. We also evaluated a realistic setting with automatically generated masks. Contrary to our hypothesis, finer segmentation did not improve overall performance: the unsegmented and binary food-region segmentation conditions performed comparably, while ingredient-level segmentation resulted in the highest error. Ingredient-level predictions preserved the relative balance of macronutrients well per ingredient, but errors in each ingredient's estimated nutrient quantities accumulated once aggregated to the dish level, leading to a general, though not strict, tendency toward worse performance as dish complexity increased. Ingredient-level regression stayed competitive for simpler dishes with visually separable ingredients, but struggled when ingredients were highly mixed. These findings indicate that the principal limitation of ingredient-level nutrient estimation is accurate quantity estimation rather than modeling of nutrient composition. Future work should prioritize explicit quantity cues, such as depth or volume, to complement ingredient-level decomposition. We release our annotations and code to support reproducible research.<\/jats:p>","DOI":"10.3389\/fnut.2026.1927530","type":"journal-article","created":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T04:53:03Z","timestamp":1791348783000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Effects of segmentation granularity on nutrient prediction from real-world food images"],"prefix":"10.3389","volume":"13","author":[{"given":"Liam","family":"Hand","sequence":"first","affiliation":[{"name":"Division of Human Nutrition and Health, Wageningen University & Research","place":["Wageningen, Netherlands"]},{"name":"Artificial Intelligence Group, Wageningen University & Research","place":["Wageningen, Netherlands"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guido","family":"Camps","sequence":"additional","affiliation":[{"name":"Division of Human Nutrition and Health, Wageningen University & Research","place":["Wageningen, Netherlands"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ricardo","family":"da Silva Torres","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Group, Wageningen University & Research","place":["Wageningen, Netherlands"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Albert Ali","family":"Salah","sequence":"additional","affiliation":[{"name":"Information and Computer Sciences, Utrecht University","place":["Utrecht, Netherlands"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,10,7]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"e0336036","DOI":"10.1371\/journal.pone.0336036","article-title":"Global burden and future projections of non-communicable diseases (2000-2050): progress toward SDG 3.4 and disparities across regions and risk factors","volume":"20","author":"Freihat","year":"2025","journal-title":"PLoS ONE"},{"key":"B2","doi-asserted-by":"crossref","first-page":"1958","DOI":"10.1016\/S0140-6736(19)30041-8","article-title":"Health effects of dietary risks in 195 countries, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017","volume":"393","author":"Afshin","year":"2019","journal-title":"Lancet"},{"key":"B3","doi-asserted-by":"publisher","first-page":"915","DOI":"10.1079\/PHN2002383","article-title":"Bias in dietary-report instruments and its implications for nutritional epidemiology","volume":"5","author":"Kipnis","year":"2002","journal-title":"Public Health Nutr"},{"key":"B4","doi-asserted-by":"publisher","first-page":"850","DOI":"10.3389\/fendo.2019.00850","article-title":"Validity of dietary assessment methods when compared to the method of doubly labeled water: a systematic review in adults","volume":"10","author":"Burrows","year":"2019","journal-title":"Front Endocrinol"},{"key":"B5","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1186\/s12966-021-01184-x","article-title":"Effects of differential measurement error in self-reported diet in longitudinal lifestyle intervention studies","volume":"18","author":"Aaby","year":"2021","journal-title":"Int J Behav Nutr Phys Activ"},{"key":"B6","doi-asserted-by":"publisher","first-page":"178","DOI":"10.1186\/s12911-023-02262-9","article-title":"Mitigating underreported error in food frequency questionnaire data using a supervised machine learning method and error adjustment algorithm","volume":"23","author":"Popoola","year":"2023","journal-title":"BMC Med Inform Decis Mak"},{"key":"B7","doi-asserted-by":"publisher","first-page":"99","DOI":"10.3389\/fnut.2020.00099","article-title":"Reproducibility of dietary intake measurement from diet diaries, photographic food records, and a novel sensor method","volume":"7","author":"Fontana","year":"2020","journal-title":"Front Nutr"},{"key":"B8","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1016\/j.orcp.2017.04.001","article-title":"Comparison of an electronic versus traditional food diary for assessing dietary intake\u2014a validation study","volume":"11","author":"Fuller","year":"2017","journal-title":"Obes Res Clin Pract"},{"key":"B9","author":"Ahuja","year":"2010","journal-title":"The USDA Food and Nutrient Database for Dietary Studies"},{"key":"B10","doi-asserted-by":"publisher","first-page":"025702","DOI":"10.1088\/0957-0233\/26\/2\/025702","article-title":"Saliency-aware food image segmentation for personal dietary assessment using a wearable computer","volume":"26","author":"Chen","year":"2015","journal-title":"Measur Sci Technol"},{"key":"B11","doi-asserted-by":"publisher","first-page":"506","DOI":"10.1145\/3474085.3475201","article-title":"A large-scale benchmark for food image segmentation","author":"Wu","year":"2021","journal-title":"Proceedings of the 29th ACM international conference on multimedia"},{"key":"B12","doi-asserted-by":"publisher","first-page":"110017","DOI":"10.1016\/j.patcog.2023.110017","article-title":"FoodMask: real-time food instance counting, segmentation and recognition","volume":"146","author":"Nguyen","year":"2024","journal-title":"Pattern Recognit"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/SMAP59435.2023.10255199","article-title":"Food image classification and segmentation with attention-based multiple instance learning","author":"Vlachopoulou","year":"2023","journal-title":"2023 18th international workshop on semantic and social media adaptation"},{"key":"B14","doi-asserted-by":"publisher","first-page":"1469878","DOI":"10.3389\/fnut.2024.1469878","article-title":"Visual nutrition analysis: leveraging segmentation and regression for food nutrient estimation","volume":"11","author":"Zhao","year":"2024","journal-title":"Front Nutr"},{"key":"B15","doi-asserted-by":"publisher","first-page":"482","DOI":"10.1109\/CVPRW67362.2025.00053","article-title":"Decomposing food images for better nutrition analysis: a nutritionist-inspired two-step multimodal LLM approach","author":"Khlaisamniang","year":"2025","journal-title":"Proceedings of the computer vision and pattern recognition conference"},{"key":"B16","doi-asserted-by":"publisher","first-page":"2196","DOI":"10.3390\/nu17132196","article-title":"2D prediction of the nutritional composition of dishes from food images: deep learning algorithm selection and data curation beyond the Nutrition5k project","volume":"17","author":"Bianco","year":"2025","journal-title":"Nutrients"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1454466","DOI":"10.3389\/fnut.2024.1454466","article-title":"Nutritional composition analysis in food images: an innovative Swin Transformer approach","volume":"11","author":"Wang","year":"2024","journal-title":"Front Nutr"},{"key":"B18","article-title":"Nutrition estimation for dietary management: a transformer approach with depth sensing","author":"Kwan","year":"2024","journal-title":"arXiv preprint arXiv:2406.01938"},{"key":"B19","doi-asserted-by":"publisher","first-page":"108821","DOI":"10.1016\/j.jfca.2025.108821","article-title":"RGB-D food nutrient estimation supported by FLAVA contrastive learning","volume":"150","author":"Feng","year":"2025","journal-title":"J Food Compos Anal"},{"key":"B20","doi-asserted-by":"publisher","first-page":"4293","DOI":"10.3390\/foods12234293","article-title":"DPF-nutrition: food nutrition estimation via depth prediction and fusion","volume":"12","author":"Han","year":"2023","journal-title":"Foods"},{"key":"B21","doi-asserted-by":"publisher","first-page":"136309","DOI":"10.1016\/j.foodchem.2023.136309","article-title":"Vision-based food nutrition estimation via RGB-D fusion network","volume":"424","author":"Shao","year":"2023","journal-title":"Food Chem"},{"key":"B22","doi-asserted-by":"publisher","first-page":"8903","DOI":"10.1109\/CVPR46437.2021.00879","article-title":"Nutrition5k: towards automatic nutritional understanding of generic food","author":"Thames","year":"2021","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B23","doi-asserted-by":"publisher","first-page":"2795","DOI":"10.1109\/TMM.2023.3330047","article-title":"Foodsam: any food segmentation","volume":"27","author":"Lan","year":"2023","journal-title":"IEEE Trans Multim"},{"key":"B24","doi-asserted-by":"publisher","first-page":"446","DOI":"10.1007\/978-3-319-10599-4_29","article-title":"Food-101-mining discriminative components with random forests","author":"Bossard","year":"2014","journal-title":"European conference on computer vision"},{"key":"B25","doi-asserted-by":"publisher","first-page":"588","DOI":"10.1109\/JBHI.2016.2636441","article-title":"Food recognition: a new dataset, experiments, and results","volume":"21","author":"Ciocca","year":"2016","journal-title":"IEEE J Biomed Health Inform"},{"key":"B26","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1007\/978-3-030-68821-9_51","article-title":"Uec-foodpix complete: a large-scale food image segmentation dataset","author":"Okamoto","year":"2021","journal-title":"ICPR international workshops and challenges: virtual event, January 10-15, 2021, proceedings, Part V"},{"key":"B27","doi-asserted-by":"publisher","first-page":"9932","DOI":"10.1109\/TPAMI.2023.3237871","article-title":"Large scale visual food recognition","volume":"45","author":"Min","year":"2023","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1233","DOI":"10.1109\/ICCV.2015.146","article-title":"Im2Calories: towards an automated mobile vision food diary","author":"Meyers","year":"2015","journal-title":"Proceedings of the IEEE international conference on computer vision"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/AIoTSys63104.2024.10780517","article-title":"Fine-grained food image segmentation method based on MS-Mask2Former","author":"Zhang","year":"2024","journal-title":"2024 international conference on artificial intelligence of things and systems (AIoTSys)"},{"key":"B30","doi-asserted-by":"publisher","first-page":"305","DOI":"10.3390\/jimaging10120305","article-title":"IngredSAM: open-world food ingredient segmentation via a single image prompt","volume":"10","author":"Chen","year":"2024","journal-title":"J Imag"},{"key":"B31","doi-asserted-by":"publisher","first-page":"7061","DOI":"10.1109\/CVPR52729.2023.00682","article-title":"Open-vocabulary semantic segmentation with mask-adapted clip","author":"Liang","year":"2023","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B32","article-title":"Open-vocabulary semantic segmentation with frozen vision-language models","author":"Ma","year":"2022","journal-title":"arXiv preprint arXiv:2210.15138"},{"key":"B33","doi-asserted-by":"publisher","first-page":"4144","DOI":"10.1109\/CVPR52733.2024.00397","article-title":"Ovfoodseg: Elevating open-vocabulary food image segmentation via image-informed textual representation","author":"Wu","year":"2024","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B34","doi-asserted-by":"publisher","first-page":"2818","DOI":"10.1109\/CVPR.2016.308","article-title":"Rethinking the inception architecture for computer vision","author":"Szegedy","year":"2016","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"B35","doi-asserted-by":"publisher","first-page":"3651","DOI":"10.1109\/TMM.2023.3313638","article-title":"Coarse-to-fine nutrition prediction","volume":"26","author":"Wang","year":"2023","journal-title":"IEEE Trans Multim"},{"key":"B36","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1007\/978-3-319-24574-4_28","article-title":"U-Net: convolutional networks for biomedical image segmentation","author":"Ronneberger","year":"2015","journal-title":"International conference on medical image computing and computer-assisted intervention"},{"key":"B37","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1109\/CVPR.2016.90","article-title":"Deep residual learning for image recognition","author":"He","year":"2016","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"B38","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00530-025-01732-6","article-title":"Food nutrition estimation with RGB-D fusion module and bidirectional feature pyramid network","volume":"31","author":"Ma","year":"2025","journal-title":"Multim Syst"},{"key":"B39","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1109\/TAFE.2024.3493332","article-title":"Ingredient-guided RGB-D fusion network for nutritional assessment","volume":"3","author":"Feng","year":"2024","journal-title":"IEEE Trans AgriFood Electr"},{"key":"B40","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1109\/CVPR.2009.5206848","article-title":"Imagenet: a large-scale hierarchical image database","author":"Deng","year":"2009","journal-title":"2009 IEEE conference on computer vision and pattern recognition"},{"key":"B41","doi-asserted-by":"publisher","first-page":"7132","DOI":"10.1109\/CVPR.2018.00745","article-title":"Squeeze-and-excitation networks","author":"Hu","year":"2018","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"B42","doi-asserted-by":"publisher","first-page":"6881","DOI":"10.1109\/CVPR46437.2021.00681","article-title":"Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers","author":"Zheng","year":"2021","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B43","doi-asserted-by":"publisher","first-page":"454","DOI":"10.3390\/app16010454","article-title":"RDINet: a deep learning model integrating RGB-D and ingredient features for food nutrition estimation","volume":"16","author":"Kuang","year":"2026","journal-title":"Appl Sci"},{"key":"B44","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1109\/JBHI.2020.3032060","article-title":"Measuring domain shift for deep learning in histopathology","volume":"25","author":"Stacke","year":"2020","journal-title":"IEEE J Biomed Health Inf"},{"key":"B45","doi-asserted-by":"publisher","first-page":"9650","DOI":"10.1109\/ICCV48922.2021.00951","article-title":"Emerging properties in self-supervised vision transformers","author":"Caron","year":"2021","journal-title":"Proceedings of the IEEE\/CVF international conference on computer vision"},{"key":"B46","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1016\/0377-0427(87)90125-7","article-title":"Silhouettes: a graphical aid to the interpretation and validation of cluster analysis","volume":"20","author":"Rousseeuw","year":"1987","journal-title":"J Comput Appl Math"},{"key":"B47","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Van der Maaten","year":"2008","journal-title":"J Mach Learn Res"}],"container-title":["Frontiers in Nutrition"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnut.2026.1927530\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T04:53:07Z","timestamp":1791348787000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnut.2026.1927530\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,7]]},"references-count":47,"alternative-id":["10.3389\/fnut.2026.1927530"],"URL":"https:\/\/doi.org\/10.3389\/fnut.2026.1927530","relation":{},"ISSN":["2296-861X"],"issn-type":[{"value":"2296-861X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,7]]},"article-number":"1927530"}}