{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T15:54:59Z","timestamp":1792252499050,"version":"4.4.0"},"reference-count":57,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T00:00:00Z","timestamp":1790726400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"MinCiencias","award":["BIOMARCADORES-111598"],"award-info":[{"award-number":["BIOMARCADORES-111598"]}],"id":[{"id":"https:\/\/ror.org\/03fd5ne08","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Nutrients"],"abstract":"<jats:p>Background\/Objectives: Large language model (LLM) pipelines increasingly convert free-text meal descriptions into nutrient values, yet reference-standard validation is rarely feasible in production. We asked whether the controls such pipelines conventionally pass suffice and built a reference-standard-free battery any pipeline can run on its output. Methods: We audited 13,073 meals logged by 513 adults through a WhatsApp conversational agent in rural Colombia (9 February 2026 to 6 August 2026). The pipeline is a two-stage cascade of commercial LLMs from different providers: a vision model renders photographs as text, while a text model maps these to nutrients. Six pre-specified tests were applied, including test\u2013retest ICC(1,1) over recurring strings, stratified by modality because 48.3% of records carried a photograph. Results: The pipeline passed every conventional control (Atwater-reconstructed energy r = 0.9951; median relative error 0.0000) but failed reproducibility. Across 191 recurring strings (944 meals), energy ICC point estimates were 0.7061 (95% CI 0.5419\u20130.8217) text only and 0.5258 (0.3778\u20130.6513) with photographs; none reached 0.75. Modality is a design control, not an established effect, and the coefficient depends on which inputs recur, so it is not intrinsic. Under the strictest identity available\u2014byte-identical text, no photograph\u2014ICC was 0.6944. Intermediate-text inspection exposed a failure invisible at the output: 1.44% of photograph-accompanied records carried a vision-stage artefact string versus 0.01% of text-only records (ICC 0.0911). Conclusions: Internal consistency does not validate an LLM-assisted nutrition pipeline. This design measures instability under recurring inputs without isolating its cause. Prospective test\u2013retest reproducibility is a cheap, necessary acceptance criterion, and in a cascade, the intermediate representation must also be inspected.<\/jats:p>","DOI":"10.3390\/nu18193233","type":"journal-article","created":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T09:25:42Z","timestamp":1790760342000},"page":"3233","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Internal Consistency Does Not Validate an LLM-Assisted Nutrition Pipeline: A Reference-Standard-Free Forensic Reliability Audit of 13,073 Free-Text Meal Records"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-7202-2915","authenticated-orcid":false,"given":"Jhony Fernando","family":"L\u00f3pez Torres","sequence":"first","affiliation":[{"name":"Neptura, Libano 731040, Tolima, Colombia"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7632-7526","authenticated-orcid":false,"given":"Juan Gabriel","family":"Casta\u00f1eda","sequence":"additional","affiliation":[{"name":"Facultad de Ciencias Humanas, Corporaci\u00f3n Universitaria Minuto de Dios\u2014UNIMINUTO, Bogot\u00e1 D.C. 111021, Colombia"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2372-8056","authenticated-orcid":false,"given":"Edwin","family":"Rivas","sequence":"additional","affiliation":[{"name":"Facultad de Ingenier\u00eda, Universidad Distrital Francisco Jos\u00e9 de Caldas, Bogot\u00e1 D.C. 111711, Colombia"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"project-administration","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"O\u2019Hara, C., Kent, G., Flynn, A.C., Gibney, E.R., and Timon, C.M. (2025). An Evaluation of ChatGPT for Nutrient Content Estimation from Meal Photographs. Nutrients, 17.","DOI":"10.3390\/nu17040607"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"107556","DOI":"10.1016\/j.cdnut.2025.107556","article-title":"Performance Evaluation of 3 Large Language Models for Nutritional Content Estimation from Food Images","volume":"9","author":"Fridolfsson","year":"2025","journal-title":"Curr. Dev. Nutr."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"E336","DOI":"10.1017\/S0029665124005743","article-title":"Using artificial intelligence to estimate the nutritional content of meal photos: An evaluation of ChatGPT-4","volume":"83","author":"Kent","year":"2024","journal-title":"Proc. Nutr. Soc."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"458","DOI":"10.1038\/s43856-025-01159-0","article-title":"DietAI24 as a framework for comprehensive nutrition estimation using multimodal large language models","volume":"5","author":"Yan","year":"2025","journal-title":"Commun. Med."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ase, A., Borowicz, J., Rakocy, K., and Piekarska, B. (2025). Large Language Models for Real-World Nutrition Assessment: Structured Prompts, Multi-Model Validation and Expert Oversight. Nutrients, 18.","DOI":"10.3390\/nu18010023"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","article-title":"Adapted large language models can outperform medical experts in clinical text summarization","volume":"30","author":"Blankemeier","year":"2024","journal-title":"Nat. Med."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1038\/s41591-024-03423-7","article-title":"Toward expert-level medical question answering with large language models","volume":"31","author":"Singhal","year":"2025","journal-title":"Nat. Med."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2613","DOI":"10.1038\/s41591-024-03097-1","article-title":"Evaluation and mitigation of the limitations of large language models in clinical decision-making","volume":"30","author":"Hager","year":"2024","journal-title":"Nat. Med."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","year":"2024","journal-title":"Npj Digit. Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1038\/s41746-023-00873-0","article-title":"The imperative for regulatory oversight of large language models (or generative AI) in healthcare","volume":"6","author":"Topol","year":"2023","journal-title":"Npj Digit. Med."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"e078378","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins","year":"2024","journal-title":"BMJ"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1016\/j.jclinepi.2010.02.006","article-title":"The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes","volume":"63","author":"Mokkink","year":"2010","journal-title":"J. Clin. Epidemiol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1038\/s41746-024-01029-4","article-title":"Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs","volume":"7","author":"Wang","year":"2024","journal-title":"npj Digit. Med."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ponzo, V., Rosato, R., Scigliano, M.C., Onida, M., Cossai, S., De Vecchi, M., Devecchi, A., Goitre, I., Favaro, E., and Merlo, F.D. (2024). Comparison of the Accuracy, Completeness, Reproducibility, and Consistency of Different AI Chatbots in Providing Nutritional Advice: An Exploratory Study. J. Clin. Med., 13.","DOI":"10.3390\/jcm13247810"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1506","DOI":"10.1038\/s41598-024-85003-w","article-title":"Evaluation of LLMs accuracy and consistency in the registered dietitian exam through prompt engineering","volume":"15","author":"Azimi","year":"2025","journal-title":"Sci. Rep."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1286","DOI":"10.1007\/s11606-025-09888-7","article-title":"Variation in Large Language Model Recommendations in Challenging Inpatient Management Scenarios","volume":"41","author":"Landon","year":"2025","journal-title":"J. Gen. Intern. Med."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1093\/jnci\/djr189","article-title":"Dealing with Dietary Measurement Error in Nutritional Cohort Studies","volume":"103","author":"Freedman","year":"2011","journal-title":"J. Natl. Cancer Inst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1079\/PHN2002383","article-title":"Bias in dietary-report instruments and its implications for nutritional epidemiology","volume":"5","author":"Kipnis","year":"2002","journal-title":"Public Health Nutr."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1738","DOI":"10.1093\/jnci\/88.23.1738","article-title":"Measurement Error and Results from Analytic Epidemiology: Dietary Fat and Breast Cancer","volume":"88","author":"Prentice","year":"1996","journal-title":"J. Natl. Cancer Inst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"65","DOI":"10.7326\/0003-4819-146-1-200701020-00012","article-title":"Is It Necessary to Correct for Measurement Error in Nutritional Epidemiology?","volume":"146","author":"Thiebaut","year":"2007","journal-title":"Ann. Intern. Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1545","DOI":"10.1093\/ije\/dyad082","article-title":"Exploring regression dilution bias using repeat measurements of 2858 variables in \u226449,000 UK Biobank participants","volume":"52","author":"Rutter","year":"2023","journal-title":"Int. J. Epidemiol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1136\/bmj.331.7511.267","article-title":"Are these data real? Statistical methods for the detection of data fabrication in clinical trials","volume":"331","author":"Evans","year":"2005","journal-title":"BMJ"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s41073-022-00126-w","article-title":"Investigating and preventing scientific misconduct using Benford\u2019s Law","volume":"8","author":"Eckhartt","year":"2023","journal-title":"Res. Integr. Peer Rev."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"899","DOI":"10.1055\/s-0042-108727","article-title":"Terminal digit preference biases polyp size measurements at endoscopy, computed tomographic colonography and histopathology","volume":"48","author":"Plumb","year":"2016","journal-title":"Endoscopy"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Bilukha, O., Couture, A., McCain, K., and Leidman, E. (2020). Comparison of anthropometric data quality in children aged 6\u201323 and 24\u201359 months: Lessons from population-representative surveys from humanitarian settings. BMC Nutr., 6.","DOI":"10.1186\/s40795-020-00385-0"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.jclinepi.2010.03.002","article-title":"Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed","volume":"64","author":"Kottner","year":"2011","journal-title":"J. Clin. Epidemiol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1038\/s41591-024-03425-5","article-title":"The TRIPOD-LLM reporting guideline for studies using large language models","volume":"31","author":"Gallifant","year":"2025","journal-title":"Nat. Med."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"e39243","DOI":"10.2196\/39243","article-title":"Conversational Agents in Health Care: Scoping Review of Their Behavior Change Techniques and Underpinning Theory","volume":"24","author":"Martinengo","year":"2022","journal-title":"J. Med. Internet Res."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"e43227","DOI":"10.2196\/43227","article-title":"Feasibility and Acceptability of Chatbots for Nutrition and Physical Activity Health Promotion Among Adolescents: Systematic Scoping Review with Adolescent Consultation","volume":"10","author":"Han","year":"2023","journal-title":"JMIR Hum. Factors"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"e56930","DOI":"10.2196\/56930","article-title":"Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review","volume":"26","author":"Laymouna","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1037\/0033-2909.86.2.420","article-title":"Intraclass correlations: Uses in assessing rater reliability","volume":"86","author":"Shrout","year":"1979","journal-title":"Psychol. Bull."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Liljequist, D., Elfving, B., and Skavberg Roaldsen, K. (2019). Intraclass correlation\u2014A discussion and demonstration of basic features. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0219854"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1649","DOI":"10.1093\/ajcn\/86.5.1649","article-title":"Accuracy of the Atwater factors and related food energy conversion factors with low-fat, high-fiber diets when energy intake is reduced spontaneously","volume":"86","author":"Zou","year":"2007","journal-title":"Am. J. Clin. Nutr."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1214\/ss\/1032280214","article-title":"Bootstrap confidence intervals","volume":"11","author":"DiCiccio","year":"1996","journal-title":"Stat. Sci."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research","volume":"15","author":"Koo","year":"2016","journal-title":"J. Chiropr. Med."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1149","DOI":"10.3758\/BRM.41.4.1149","article-title":"Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses","volume":"41","author":"Faul","year":"2009","journal-title":"Behav. Res. Methods"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"657","DOI":"10.3390\/ejihpe14030043","article-title":"ChatGPT\u2019s Response Consistency: A Study on Repeated Queries of Medical Examination Questions","volume":"14","author":"Funk","year":"2024","journal-title":"Eur. J. Investig. Health Psychol. Educ."},{"key":"ref_39","first-page":"42","article-title":"An Empirical Study of the Non-Determinism of ChatGPT in Code Generation","volume":"34","author":"Ouyang","year":"2024","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1038\/s41586-024-07421-0","article-title":"Detecting hallucinations in large language models using semantic entropy","volume":"630","author":"Farquhar","year":"2024","journal-title":"Nature"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1177\/0379572118783181","article-title":"Assessment of Anthropometric Data Following Investments to Ensure Quality: Kenya Demographic Health Surveys Case Study, 2008 to 2009 and 2014","volume":"39","author":"Leidman","year":"2018","journal-title":"Food Nutr. Bull."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Conkle, J., Ramakrishnan, U., Flores-Ayala, R., Suchdev, P.S., and Martorell, R. (2017). Improving the quality of child anthropometry: Manual anthropometry in the Body Imaging for Nutritional Assessment Study (BINA). PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0189332"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"100804","DOI":"10.1016\/j.patter.2023.100804","article-title":"Leakage and the reproducibility crisis in machine-learning-based science","volume":"4","author":"Kapoor","year":"2023","journal-title":"Patterns"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"980","DOI":"10.3945\/an.117.016980","article-title":"Measurement Errors in Dietary Assessment Using Self-Reported 24-Hour Recalls in Low-Income Countries and Strategies for Their Prevention","volume":"8","author":"Gibson","year":"2017","journal-title":"Adv. Nutr."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"926","DOI":"10.12688\/f1000research.10703.1","article-title":"\u039b\u03ac\u03b3\u03b9\u03bf\u03c5 \u03a0 Dietary assessment methods in epidemiological research: Current state of the art and future prospects","volume":"6","author":"Naska","year":"2017","journal-title":"F1000Research"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1093\/ajcn\/nqx002","article-title":"Comparison of self-reported dietary intakes from the Automated Self-Administered 24-h recall, 4-d food records, and food-frequency questionnaires against recovery biomarkers","volume":"107","author":"Park","year":"2018","journal-title":"Am. J. Clin. Nutr."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Ravelli, M.N., and Schoeller, D.A. (2020). Traditional Self-Reported Dietary Instruments Are Prone to Inaccuracies and New Approaches Are Needed. Front. Nutr., 7.","DOI":"10.3389\/fnut.2020.00090"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Banna, J., McCrory, M.A., Fialkowski, M.K., and Boushey, C. (2017). Examining Plausibility of Self-Reported Energy Intake Data: Considerations for Method Selection. Front. Nutr., 4.","DOI":"10.3389\/fnut.2017.00045"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1277","DOI":"10.1007\/s00421-017-3641-x","article-title":"Doubly labelled water assessment of energy expenditure: Principle, practice, and promise","volume":"117","author":"Westerterp","year":"2017","journal-title":"Eur. J. Appl. Physiol."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"e29","DOI":"10.1017\/jns.2019.20","article-title":"Validity and reliability of an online self-report 24-h dietary recall method (Intake24): A doubly labelled water study and repeated-measures analysis","volume":"8","author":"Foster","year":"2019","journal-title":"J. Nutr. Sci."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Delgado, A., Issaoui, M., Vieira, M.C., de Carvalho, I.S., and Fardet, A. (2021). Food Composition Databases: Does It Matter to Human Health?. Nutrients, 13.","DOI":"10.3390\/nu13082816"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"e026652","DOI":"10.1136\/bmjopen-2018-026652","article-title":"Nutrient composition databases in the age of big data: FoodDB, a comprehensive, real-time database infrastructure","volume":"9","author":"Harrington","year":"2019","journal-title":"BMJ Open"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"2535","DOI":"10.1017\/S1368980018001222","article-title":"Energy intake and food sources of eight Latin American countries: Results from the Latin American Study of Nutrition and Health (ELANS)","volume":"21","author":"Kovalskys","year":"2018","journal-title":"Public Health Nutr."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Eldridge, A.L., Piernas, C., Illner, A.-K., Gibney, M.J., Gurinovi\u0107, M.A., De Vries, J.H., and Cade, J.E. (2018). Evaluation of New Technology-Based Tools for Dietary Intake Assessment\u2014An ILSI Europe Dietary Intake and Exposure Task Force Evaluation. Nutrients, 11.","DOI":"10.3390\/nu11010055"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Fang, S., Shao, Z., Kerr, D.A., Boushey, C.J., and Zhu, F. (2019). An End-to-End Image-Based Automatic Food Energy Estimation Technique Based on Learned Energy Distribution Images: Protocol and Methodology. Nutrients, 11.","DOI":"10.3390\/nu11040877"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"e51432","DOI":"10.2196\/51432","article-title":"Advancements in Using AI for Dietary Assessment Based on Food Images: Scoping Review","volume":"26","author":"Chotwanvirat","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_57","first-page":"2","article-title":"How Is ChatGPT\u2019s Behavior Changing over Time?","volume":"6","author":"Chen","year":"2024","journal-title":"Harv. Data Sci. Rev."}],"container-title":["Nutrients"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-6643\/18\/19\/3233\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T04:22:29Z","timestamp":1790914949000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-6643\/18\/19\/3233"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,30]]},"references-count":57,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2026,10]]}},"alternative-id":["nu18193233"],"URL":"https:\/\/doi.org\/10.3390\/nu18193233","relation":{},"ISSN":["2072-6643"],"issn-type":[{"value":"2072-6643","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,30]]}}}