{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T07:04:25Z","timestamp":1781334265415,"version":"3.54.1"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2019,11,14]],"date-time":"2019-11-14T00:00:00Z","timestamp":1573689600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"United States National Science Foundation","award":["CNS-1566359 and CNS-1750679"],"award-info":[{"award-number":["CNS-1566359 and CNS-1750679"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2019,12,31]]},"abstract":"<jats:p>Lifestyle interventions with the focus on diet are crucial in self-management and prevention of many chronic conditions, such as obesity, cardiovascular disease, diabetes, and cancer. Such interventions require a diet monitoring approach to estimate overall dietary composition and energy intake. Although wearable sensors have been used to estimate eating context (e.g., food type and eating time), accurate monitoring of dietary intake has remained a challenging problem. In particular, because monitoring dietary intake is a self-administered task that involves the end-user to record or report their nutrition intake, current diet monitoring technologies are prone to measurement errors related to challenges of human memory, estimation, and bias. New approaches based on mobile devices have been proposed to facilitate the process of dietary intake recording. These technologies require individuals to use mobile devices such as smartphones to record nutrition intake by either entering text or taking images of the food. Such approaches, however, suffer from errors due to low adherence to technology adoption and time sensitivity to the dietary intake context.<\/jats:p>\n          <jats:p>\n            In this article, we introduce\n            <jats:italic>EZNutriPal<\/jats:italic>\n            ,\n            <jats:sup>1<\/jats:sup>\n            an interactive diet monitoring system that operates on unstructured mobile data such as speech and free-text to facilitate dietary recording, real-time prompting, and personalized nutrition monitoring. EZNutriPal features a natural language processing unit that learns incrementally to add user-specific nutrition data and rules to the system. To prevent missing data that are required for dietary monitoring (e.g., calorie intake estimation), EZNutriPal devises an interactive operating mode that prompts the end-user to complete missing data in real-time. Additionally, we propose a combinatorial optimization approach to identify the most appropriate pairs of food names and food quantities in complex input sentences. We evaluate the performance of EZNutriPal using real data collected from 23 human subjects who participated in two user studies conducted in 13 days each. The results demonstrate that EZNutriPal achieves an accuracy of 89.7% in calorie intake estimation. We also assess the impacts of the incremental training and interactive prompting technologies on the accuracy of nutrient intake estimation and show that incremental training and interactive prompting improve the performance of diet monitoring by 49.6% and 29.1%, respectively, compared to a system without such computing units.\n          <\/jats:p>","DOI":"10.1145\/3319370","type":"journal-article","created":{"date-parts":[[2019,11,14]],"date-time":"2019-11-14T22:07:36Z","timestamp":1573769256000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Human-in-the-loop Learning for Personalized Diet Monitoring from Unstructured Mobile Data"],"prefix":"10.1145","volume":"9","author":[{"given":"Niloofar","family":"Hezarjaribi","sequence":"first","affiliation":[{"name":"Washington State University, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sepideh","family":"Mazrouee","sequence":"additional","affiliation":[{"name":"University of California San Diego, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saied","family":"Hemati","sequence":"additional","affiliation":[{"name":"University of Idaho, ID, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Naomi S.","family":"Chaytor","sequence":"additional","affiliation":[{"name":"Washington State University, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martine","family":"Perrigue","sequence":"additional","affiliation":[{"name":"Washington State University, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hassan","family":"Ghasemzadeh","sequence":"additional","affiliation":[{"name":"Washington State University, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,11,14]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2016.2569472"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2016.2569472"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSEN.2015.2402652"},{"key":"e_1_2_1_4_1","volume-title":"Main findings: Teens, technology, and human potential","author":"Anderson Jana","year":"2020","unstructured":"Jana Anderson and Lee Raine . 2012. Main findings: Teens, technology, and human potential in 2020 . Pew Research Center , Washington, DC . Jana Anderson and Lee Raine. 2012. Main findings: Teens, technology, and human potential in 2020. Pew Research Center, Washington, DC."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.nut.2004.11.006"},{"key":"e_1_2_1_6_1","volume-title":"Tsitsiklis","author":"Bertsimas Dimitris","year":"1997","unstructured":"Dimitris Bertsimas and John N . Tsitsiklis . 1997 . Introduction to Linear Optimization. Vol. 6 . Athena Scientific Belmont, MA. Dimitris Bertsimas and John N. Tsitsiklis. 1997. Introduction to Linear Optimization. Vol. 6. Athena Scientific Belmont, MA."},{"key":"e_1_2_1_7_1","volume-title":"The global burden of diabetic foot disease. The Lancet 366, 9498","author":"Boulton Andrew J. M.","year":"2005","unstructured":"Andrew J. M. Boulton , Loretta Vileikyte , Gunnel Ragnarson-Tennvall , and Jan Apelqvist . 2005. The global burden of diabetic foot disease. The Lancet 366, 9498 ( 2005 ), 1719--1724. Andrew J. M. Boulton, Loretta Vileikyte, Gunnel Ragnarson-Tennvall, and Jan Apelqvist. 2005. The global burden of diabetic foot disease. The Lancet 366, 9498 (2005), 1719--1724."},{"key":"e_1_2_1_8_1","volume-title":"TusaRebecca E. Schap, Stacey Paterson, Tanisha Aflague, Marc Bosch Ruiz, Ziad Ahmad, and Edward J. Delp.","author":"Boushey Carol Jo","year":"2015","unstructured":"Carol Jo Boushey , Amelia J. Harray , Deborah Anne Kerr , TusaRebecca E. Schap, Stacey Paterson, Tanisha Aflague, Marc Bosch Ruiz, Ziad Ahmad, and Edward J. Delp. 2015 . How willing are adolescents to record their dietary intake? The mobile food record. JMIR mHealth and uHealth 3, 2 (2015). Carol Jo Boushey, Amelia J. Harray, Deborah Anne Kerr, TusaRebecca E. Schap, Stacey Paterson, Tanisha Aflague, Marc Bosch Ruiz, Ziad Ahmad, and Edward J. Delp. 2015. How willing are adolescents to record their dietary intake? The mobile food record. JMIR mHealth and uHealth 3, 2 (2015)."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1038\/ejcn.2008.65"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.appet.2005.12.002"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.2283"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/NEBC.2012.6206979"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357335"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1093\/ije\/30.2.309"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3055004.3055015"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2016.2640142"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1463160.1463174"},{"key":"e_1_2_1_18_1","volume-title":"Nutrient database for dietary studies, 4.1","author":"Food USDA","year":"2004","unstructured":"USDA Food . 2004. Nutrient database for dietary studies, 4.1 . Beltsville, MD : Agricultural research service. Food Surv. Res. Group ( 2004 ). USDA Food. 2004. Nutrient database for dietary studies, 4.1. Beltsville, MD: Agricultural research service. Food Surv. Res. Group (2004)."},{"key":"e_1_2_1_19_1","volume-title":"A hardware-assisted energy-efficient processing model for activity recognition using wearables. ACM Trans. Des. Auto. Electron. Syst. 21","author":"Ghasemzadeh H.","year":"2016","unstructured":"H. Ghasemzadeh , Ramin Fallahzadeh , and Roozbeh Jafari . 2016. A hardware-assisted energy-efficient processing model for activity recognition using wearables. ACM Trans. Des. Auto. Electron. Syst. 21 ( 2016 ). H. Ghasemzadeh, Ramin Fallahzadeh, and Roozbeh Jafari. 2016. A hardware-assisted energy-efficient processing model for activity recognition using wearables. ACM Trans. Des. Auto. Electron. Syst. 21 (2016)."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1093\/ajcn\/71.1.130"},{"key":"e_1_2_1_21_1","volume-title":"Moodie","author":"Gortmaker Steven L.","year":"2011","unstructured":"Steven L. Gortmaker , Boyd A. Swinburn , David Levy , Rob Carter , Patricia L. Mabry , Diane T. Finegood , Terry Huang , Tim Marsh , and Marjory L . Moodie . 2011 . Changing the future of obesity: Science, policy, and action. The Lancet 378, 9793 (2011), 838--847. Steven L. Gortmaker, Boyd A. Swinburn, David Levy, Rob Carter, Patricia L. Mabry, Diane T. Finegood, Terry Huang, Tim Marsh, and Marjory L. Moodie. 2011. Changing the future of obesity: Science, policy, and action. The Lancet 378, 9793 (2011), 838--847."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2015.2419251"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.3850\/9783981537079_0883"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2017.2709333"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC.2016.7591115"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC.2012.6346978"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.4865269"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/nav.3800020109"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2012.392"},{"key":"e_1_2_1_30_1","volume-title":"Bell","author":"Laing Brian Yoshio","year":"2014","unstructured":"Brian Yoshio Laing , Carol M. Mangione , Chi-Hong Tseng , Mei Leng , Ekaterina Vaisberg , Megha Mahida , Michelle Bholat , Eve Glazier , Donald E. Morisky , and Douglas S . Bell . 2014 . Effectiveness of a smartphone application for weight loss compared with usual care in overweight primary care patients: A randomized, controlled trial. Ann. Internal Med. 161, 10_Supplement (2014), S5--S12. Brian Yoshio Laing, Carol M. Mangione, Chi-Hong Tseng, Mei Leng, Ekaterina Vaisberg, Megha Mahida, Michelle Bholat, Eve Glazier, Donald E. Morisky, and Douglas S. Bell. 2014. Effectiveness of a smartphone application for weight loss compared with usual care in overweight primary care patients: A randomized, controlled trial. Ann. Internal Med. 161, 10_Supplement (2014), S5--S12."},{"key":"e_1_2_1_31_1","unstructured":"Yuchao Ma and Hassan Ghasemzadeh. 2019. LabelForest: Non-parametric semi-supervised learning for activity recognition. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI\u201919) the 31st Innovative Applications of Artificial Intelligence Conference (IAAI\u201919) the 9th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI\u201919). 4520--4527. Retrieved from: https:\/\/aaai.org\/ojs\/index.php\/AAAI\/article\/view\/4369.  Yuchao Ma and Hassan Ghasemzadeh. 2019. LabelForest: Non-parametric semi-supervised learning for activity recognition. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI\u201919) the 31st Innovative Applications of Artificial Intelligence Conference (IAAI\u201919) the 9th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI\u201919). 4520--4527. Retrieved from: https:\/\/aaai.org\/ojs\/index.php\/AAAI\/article\/view\/4369."},{"key":"e_1_2_1_32_1","first-page":"565","article-title":"The medical complications of obesity","volume":"99","author":"Malnick Stephen D. H.","year":"2006","unstructured":"Stephen D. H. Malnick and Hilla Knobler . 2006 . The medical complications of obesity . J. Assoc. Phys. 99 , 9 (2006), 565 -- 579 . Stephen D. H. Malnick and Hilla Knobler. 2006. The medical complications of obesity. J. Assoc. Phys. 99, 9 (2006), 565--579.","journal-title":"J. Assoc. Phys."},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of the International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC\u201909)","author":"Martin Corby K.","unstructured":"Corby K. Martin , Sertan Kaya , and Bahadir K. Gunturk . 2009. Quantification of food intake using food image analysis . In Proceedings of the International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC\u201909) . IEEE, 6869--6872. Corby K. Martin, Sertan Kaya, and Bahadir K. Gunturk. 2009. Quantification of food intake using food image analysis. In Proceedings of the International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC\u201909). IEEE, 6869--6872."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jada.2009.10.014"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.1993.03510180077038"},{"key":"e_1_2_1_36_1","unstructured":"LLC MyFitnessPal. 2015. Myfitnesspal.  LLC MyFitnessPal. 2015. Myfitnesspal."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.1100"},{"key":"e_1_2_1_38_1","first-page":"31","article-title":"MyFitnessPal","volume":"22","author":"Rebedew David","year":"2015","unstructured":"David Rebedew . 2015 . MyFitnessPal . Fam. Pract. Manag. 22 , 2 (2015), 31 -- 31 . David Rebedew. 2015. MyFitnessPal. Fam. Pract. Manag. 22, 2 (2015), 31--31.","journal-title":"Fam. Pract. Manag."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.3109\/17538157.2012.674586"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897937.2898066"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACSSC.2014.7094840"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.3945\/ajcn.113.062125"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2011.6130229"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2016.2612580"},{"key":"e_1_2_1_45_1","first-page":"3","article-title":"Dietary assessment methodology","volume":"2","author":"Thompson Frances E.","year":"2008","unstructured":"Frances E. Thompson , Amy F. Subar , 2008 . Dietary assessment methodology . Nutrit. Prevent. Treat. Disease 2 (2008), 3 -- 39 . Frances E. Thompson, Amy F. Subar, et al. 2008. Dietary assessment methodology. Nutrit. Prevent. Treat. Disease 2 (2008), 3--39.","journal-title":"Nutrit. Prevent. Treat. Disease"},{"key":"e_1_2_1_46_1","volume-title":"Shih Yee-Marie Tan, Jacob Behrens, Ian Mathew, Erich J. Conrad, Ladson Hinton, Peter Yellowlees, and Matcheri Keshavan.","author":"Torous John","year":"2014","unstructured":"John Torous , Steven Richard Chan , Shih Yee-Marie Tan, Jacob Behrens, Ian Mathew, Erich J. Conrad, Ladson Hinton, Peter Yellowlees, and Matcheri Keshavan. 2014 . Patient smartphone ownership and interest in mobile apps to monitor symptoms of mental health conditions: A survey in four geographically distinct psychiatric clinics. JMIR Ment. Health 1, 1 (2014). John Torous, Steven Richard Chan, Shih Yee-Marie Tan, Jacob Behrens, Ian Mathew, Erich J. Conrad, Ladson Hinton, Peter Yellowlees, and Matcheri Keshavan. 2014. Patient smartphone ownership and interest in mobile apps to monitor symptoms of mental health conditions: A survey in four geographically distinct psychiatric clinics. JMIR Ment. Health 1, 1 (2014)."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11036-007-0014-4"},{"key":"e_1_2_1_48_1","unstructured":"S. Y. Wu and A. Green. 2000. Projection of chronic illness prevalence and cost inflation. RAND Health 18 (2000).  S. Y. Wu and A. Green. 2000. Projection of chronic illness prevalence and cost inflation. RAND Health 18 (2000)."}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3319370","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3319370","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:38:21Z","timestamp":1750199901000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3319370"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,14]]},"references-count":48,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,12,31]]}},"alternative-id":["10.1145\/3319370"],"URL":"https:\/\/doi.org\/10.1145\/3319370","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,14]]},"assertion":[{"value":"2018-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-11-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}