{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,3]],"date-time":"2025-12-03T17:55:49Z","timestamp":1764784549079,"version":"3.37.3"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2020,11,21]],"date-time":"2020-11-21T00:00:00Z","timestamp":1605916800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,11,21]],"date-time":"2020-11-21T00:00:00Z","timestamp":1605916800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010016","name":"Nottingham Trent University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100010016","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Ambient Intell Human Comput"],"published-print":{"date-parts":[[2021,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Eating difficulties and the subsequent need for eating assistance are a prevalent issue within the elderly population. Besides, a poor diet is considered a confounding factor for developing chronic diseases and functional limitations. Driven by the above issues, this paper proposes a wrist-worn tri-axial accelerometer based food and drink intake recognition system. First, an adaptive segmentation technique is employed to identify potential eating and drinking gestures from the continuous accelerometer readings. A posteriori, a study upon the use of Convolutional Neural Networks for the recognition of eating and drinking gestures is carried out. This includes the employment of three time series to image encoding frameworks, namely the signal spectrogram, the Markov Transition Field and the Gramian Angular Field, as well as the development of various multi-input multi-domain networks. The recognition of the gestures is then tackled as a 3-class classification problem (\u2018Eat\u2019, \u2018Drink\u2019 and \u2018Null\u2019), where the \u2018Null\u2019 class is composed of all the irrelevant gestures included in the post-segmentation gesture set. An average per-class classification accuracy of 97.10% was achieved by the proposed system. When compared to similar work, such accurate classification performance signifies a great contribution to the field of assisted living.<\/jats:p>","DOI":"10.1007\/s12652-020-02684-7","type":"journal-article","created":{"date-parts":[[2020,11,21]],"date-time":"2020-11-21T04:55:54Z","timestamp":1605934554000},"page":"9435-9447","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["A deep learning based wearable system for food and drink intake recognition"],"prefix":"10.1007","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3571-847X","authenticated-orcid":false,"given":"Dario","family":"Ortega Anderez","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ahmad","family":"Lotfi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Amir","family":"Pourabdollah","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,11,21]]},"reference":[{"issue":"21","key":"2684_CR1","doi-asserted-by":"publisher","first-page":"7027","DOI":"10.1007\/s00500-018-3364-x","volume":"22","author":"DA Adama","year":"2018","unstructured":"Adama DA, Lotfi A, Langensiepen C, Lee K, Trindade P (2018) Human activity learning for assistive robotics using a classifier ensemble. Soft Comput 22(21):7027\u20137039. https:\/\/doi.org\/10.1007\/s00500-018-3364-x","journal-title":"Soft Comput"},{"key":"2684_CR2","doi-asserted-by":"publisher","unstructured":"Amft O, Bannach D, Pirkl G, Kreil M, Lukowicz P (2010) In: 8th IEEE international conference on pervasive computing and communications workshops, PERCOM Workshops 2010. https:\/\/doi.org\/10.1109\/PERCOMW.2010.5470653","DOI":"10.1109\/PERCOMW.2010.5470653"},{"key":"2684_CR3","doi-asserted-by":"publisher","unstructured":"Anderez DO, Lotfi A, Langensiepen C (2018a) A hierarchical approach in food and drink intake recognition using wearable inertial sensors. In: Proceedings of the 11th pervasive technologies related to assistive environments conference, ACM, pp 552\u2013557. https:\/\/doi.org\/10.1145\/3197768.3201542","DOI":"10.1145\/3197768.3201542"},{"key":"2684_CR4","doi-asserted-by":"crossref","unstructured":"Anderez DO, Lotfi A, Langensiepen C (2018b) A novel crossings-based segmentation approach for gesture recognition. In: UK workshop on computational intelligence, Springer, pp 383\u2013391","DOI":"10.1007\/978-3-319-97982-3_32"},{"key":"2684_CR5","doi-asserted-by":"publisher","unstructured":"Anderez DO, Lotfi A, Pourabdollah A (2019) Temporal convolution neural network for food and drink intake recognition. In: Proceedings of the 12th ACM international conference on pervasive technologies related to assistive environments, ACM, pp 580\u2013586. https:\/\/doi.org\/10.1145\/3316782.3322784","DOI":"10.1145\/3316782.3322784"},{"key":"2684_CR6","doi-asserted-by":"publisher","first-page":"112888","DOI":"10.1016\/j.eswa.2019.112888","volume":"140","author":"DO Anderez","year":"2020","unstructured":"Anderez DO, Lotfi A, Pourabdollah A (2020) Eating and drinking gesture spotting and recognition using a novel adaptive segmentation technique and a gesture discrepancy measure. Expert Syst Appl 140:112888. https:\/\/doi.org\/10.1016\/j.eswa.2019.112888","journal-title":"Expert Syst Appl"},{"issue":"4","key":"2684_CR7","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1007\/s00779-019-01319-9","volume":"24","author":"E Casella","year":"2020","unstructured":"Casella E, Ortolani M, Silvestri S, Das SK (2020) Hierarchical syntactic models for human activity recognition through mobility traces. Pers Ubiquit Comput 24(4):451\u2013464. https:\/\/doi.org\/10.1007\/s00779-019-01319-9","journal-title":"Pers Ubiquit Comput"},{"key":"2684_CR8","doi-asserted-by":"publisher","unstructured":"Chen LH, Liu KC, Hsieh CY, Chan CT (2017) Drinking gesture spotting and identification using single wrist-worn inertial sensor. In: Proceedings of the 2017 IEEE international conference on applied system innovation: applied system innovation for modern technology, ICASI 2017, pp 299\u2013302. https:\/\/doi.org\/10.1109\/ICASI.2017.7988411","DOI":"10.1109\/ICASI.2017.7988411"},{"issue":"4","key":"2684_CR9","doi-asserted-by":"publisher","first-page":"1253","DOI":"10.1109\/JBHI.2013.2282471","volume":"18","author":"Y Dong","year":"2014","unstructured":"Dong Y, Scisco J, Wilson M, Muth E, Hoover A (2014) Detecting periods of eating during free-living by tracking wrist motion. IEEE J Biomed Health Informat 18(4):1253\u20131260. https:\/\/doi.org\/10.1109\/JBHI.2013.2282471","journal-title":"IEEE J Biomed Health Informat"},{"key":"2684_CR10","doi-asserted-by":"publisher","unstructured":"Duffner S, Berlemont S, Lefebvre G, Garcia C (2014) 3D gesture classification with convolutional neural networks. In: ICASSP, IEEE international conference on acoustics, speech and signal processing - proceedings, pp 5432\u20135436. https:\/\/doi.org\/10.1109\/ICASSP.2014.6854641","DOI":"10.1109\/ICASSP.2014.6854641"},{"issue":"2","key":"2684_CR11","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1007\/s00779-014-0827-7","volume":"19","author":"KS Gayathri","year":"2015","unstructured":"Gayathri KS, Elias S, Ravindran B (2015) Hierarchical activity recognition for dementia care using Markov Logic Network. Pers Ubiquit Comput 19(2):271\u2013285. https:\/\/doi.org\/10.1007\/s00779-014-0827-7","journal-title":"Pers Ubiquit Comput"},{"key":"2684_CR12","doi-asserted-by":"publisher","unstructured":"Ha S, Yun JM, Choi S (2016) Multi-modal Convolutional Neural Networks for Activity Recognition. In: Proceedings - 2015 IEEE international conference on systems, man, and cybernetics, SMC 2015, IEEE, pp 3017\u20133022. https:\/\/doi.org\/10.1109\/SMC.2015.525","DOI":"10.1109\/SMC.2015.525"},{"key":"2684_CR13","doi-asserted-by":"publisher","first-page":"915","DOI":"10.1016\/j.asoc.2017.09.027","volume":"62","author":"A Ignatov","year":"2018","unstructured":"Ignatov A (2018) Real-time human activity recognition from accelerometer data using convolutional neural networks. Appl Soft Comput J 62:915\u2013922. https:\/\/doi.org\/10.1016\/j.asoc.2017.09.027","journal-title":"Appl Soft Comput J"},{"key":"2684_CR14","doi-asserted-by":"publisher","unstructured":"Jiang W, Yin Z (2015) Human activity recognition using wearable sensors by deep convolutional neural networks. In: Proceedings of the 23rd ACM international conference on Multimedia, ACM, pp 1307\u20131310. https:\/\/doi.org\/10.1145\/2733373.2806333","DOI":"10.1145\/2733373.2806333"},{"issue":"6","key":"2684_CR15","doi-asserted-by":"publisher","first-page":"2010","DOI":"10.1016\/j.patcog.2007.11.016","volume":"41","author":"H Junker","year":"2008","unstructured":"Junker H, Amft O, Lukowicz P, Tr\u00f6ster G (2008) Gesture spotting with body-worn inertial sensors to detect user activities. Pattern Recogn 41(6):2010\u20132024. https:\/\/doi.org\/10.1016\/j.patcog.2007.11.016","journal-title":"Pattern Recogn"},{"key":"2684_CR16","doi-asserted-by":"crossref","unstructured":"Keogh E, Chu S, Hart D, Pazzani M (2004) Segmenting time series: a survey and novel approach. In: Data mining in time series databases, World Scientific, pp 1\u201321","DOI":"10.1142\/9789812565402_0001"},{"key":"2684_CR17","doi-asserted-by":"publisher","unstructured":"Lawal IA, Bano S (2019) Deep human activity recognition using wearable sensors. In: Proceedings of the 12th international conference on pervasive technologies related to assistive environments, ACM, pp 45\u201348. https:\/\/doi.org\/10.1145\/3316782.3321538","DOI":"10.1145\/3316782.3321538"},{"issue":"8","key":"2684_CR18","doi-asserted-by":"publisher","first-page":"887","DOI":"10.1007\/s00779-011-0403-3","volume":"15","author":"MW Lee","year":"2011","unstructured":"Lee MW, Khan AM, Kim TS (2011) A single tri-axial accelerometer-based real-time personal life log system capable of human activity recognition and exercise information generation. Pers Ubiquit Comput 15(8):887\u2013898. https:\/\/doi.org\/10.1007\/s00779-011-0403-3","journal-title":"Pers Ubiquit Comput"},{"key":"2684_CR19","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1109\/BIGCOMP.2017.7881728","volume":"2017","author":"SM Lee","year":"2017","unstructured":"Lee SM, Yoon SM, Cho H (2017) Human activity recognition from accelerometer data using convolutional neural network. IEEE Int Conf Big Data Smart Comput BigComp 2017:131\u2013134. https:\/\/doi.org\/10.1109\/BIGCOMP.2017.7881728","journal-title":"IEEE Int Conf Big Data Smart Comput BigComp"},{"issue":"1","key":"2684_CR20","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1067\/mgn.2003.8","volume":"24","author":"C Lohrmann","year":"2003","unstructured":"Lohrmann C, Dijkstra A, Dassen T (2003) The care dependency scale: an assessment instrument for elderly patients in German hospitals. Geriatr Nurs 24(1):40\u201343. https:\/\/doi.org\/10.1067\/mgn.2003.8","journal-title":"Geriatr Nurs"},{"issue":"11","key":"2684_CR21","doi-asserted-by":"publisher","first-page":"4319","DOI":"10.1007\/s12652-018-1110-y","volume":"10","author":"D Ortega-Anderez","year":"2019","unstructured":"Ortega-Anderez D, Lotfi A, Langensiepen C, Appiah K (2019) A multi-level refinement approach towards the classification of quotidian activities using accelerometer data. J Ambient Intell Hum Comput 10(11):4319\u20134330. https:\/\/doi.org\/10.1007\/s12652-018-1110-y","journal-title":"J Ambient Intell Hum Comput"},{"key":"2684_CR22","doi-asserted-by":"publisher","first-page":"S30","DOI":"10.1007\/BF03405198","volume":"96","author":"H Payette","year":"2005","unstructured":"Payette H, Shatenstein B (2005) Determinants of healthy eating in community-dwelling elderly people. Can J Public Health 96:S30\u2013S35. https:\/\/doi.org\/10.1007\/BF03405198","journal-title":"Can J Public Health"},{"issue":"3","key":"2684_CR23","doi-asserted-by":"publisher","first-page":"825","DOI":"10.1109\/JBHI.2014.2329137","volume":"19","author":"RI Ramos-Garcia","year":"2015","unstructured":"Ramos-Garcia RI, Muth ER, Gowdy JN, Hoover AW (2015) Improving the recognition of eating gestures using intergesture sequential dependencies. IEEE J Biomed Health Informat 19(3):825\u2013831. https:\/\/doi.org\/10.1109\/JBHI.2014.2329137","journal-title":"IEEE J Biomed Health Informat"},{"key":"2684_CR24","doi-asserted-by":"crossref","unstructured":"Ronao CA, Cho SB (2015) Evaluation of deep convolutional neural network architectures for human activity recognition with smartphone sensors. In: Proc. of the KIISE Korea computer congress, pp 858\u2013860","DOI":"10.1007\/978-3-319-26561-2_6"},{"key":"2684_CR25","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1016\/j.eswa.2016.04.032","volume":"59","author":"CA Ronao","year":"2016","unstructured":"Ronao CA, Cho SB (2016) Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst Appl 59:235\u2013244. https:\/\/doi.org\/10.1016\/j.eswa.2016.04.032","journal-title":"Expert Syst Appl"},{"key":"2684_CR26","doi-asserted-by":"publisher","DOI":"10.1080\/03014460802089825","author":"EC Rush","year":"2008","unstructured":"Rush EC, Valencia ME, Plank LD (2008) Validation of a 7-day physical activity diary against doubly-labelled water. Ann Hum Biol. https:\/\/doi.org\/10.1080\/03014460802089825","journal-title":"Ann Hum Biol"},{"key":"2684_CR27","doi-asserted-by":"crossref","unstructured":"Schiboni G, Amft O (2018) Sparse natural gesture spotting in free living to monitor drinking with wrist-worn inertial sensors. In: Proceedings of the international symposium on wearable computers, ACM, pp 140\u2013147","DOI":"10.1145\/3267242.3267253"},{"key":"2684_CR28","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1016\/j.eswa.2017.04.032","volume":"83","author":"JI Serrano","year":"2017","unstructured":"Serrano JI, Lambrecht S, del Castillo MD, Romero JP, Benito-Le\u00f3n J, Rocon E (2017) Identification of activities of daily living in tremorous patients using inertial sensors. Expert Syst Appl 83:40\u201348. https:\/\/doi.org\/10.1016\/j.eswa.2017.04.032","journal-title":"Expert Syst Appl"},{"issue":"4","key":"2684_CR29","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1016\/j.amepre.2005.07.005","volume":"29","author":"BJ Smith","year":"2005","unstructured":"Smith BJ, Marshall AL, Huang N (2005) Screening for physical activity in family practice: evaluation of two brief assessment tools. Am J Prev Med 29(4):256\u2013264. https:\/\/doi.org\/10.1016\/j.amepre.2005.07.005","journal-title":"Am J Prev Med"},{"key":"2684_CR30","unstructured":"Wang Z, Oates T (2015) Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. In: Workshops at the twenty-ninth AAAI conference on artificial intelligence"},{"key":"2684_CR31","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1016\/j.patrec.2018.02.010","volume":"119","author":"J Wang","year":"2019","unstructured":"Wang J, Chen Y, Hao S, Peng X, Hu L (2019) Deep learning for sensor-based activity recognition: a survey. Pattern Recogn Lett 119:3\u201311. https:\/\/doi.org\/10.1016\/j.patrec.2018.02.010","journal-title":"Pattern Recogn Lett"},{"issue":"2","key":"2684_CR32","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1046\/j.1365-2702.2001.00479.x","volume":"10","author":"A Westergren","year":"2001","unstructured":"Westergren A (2001) Eating difficulties, need for assisted eating, nutritional status and pressure ulcers in patients admitted for stroke rehabilitation. J Clin Nurs 10(2):257\u2013269. https:\/\/doi.org\/10.1046\/j.1365-2702.2001.00479.x","journal-title":"J Clin Nurs"},{"issue":"3","key":"2684_CR33","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1016\/S0020-7489(01)00025-6","volume":"39","author":"A Westergren","year":"2002","unstructured":"Westergren A, Unosson M, Ohlsson O, Loref\u00e4lt B, Hallberg IR (2002) Eating difficulties, assisted eating and nutritional status in elderly (65 years) patients in hospital rehabilitation. Int J Nurs Stud 39(3):341\u2013351. https:\/\/doi.org\/10.1016\/S0020-7489(01)00025-6","journal-title":"Int J Nurs Stud"},{"key":"2684_CR34","unstructured":"Yang JB, Nguyen MN, San PP, Li XL, Krishnaswamy S (2015) Deep convolutional neural networks on multichannel time series for human activity recognition. In: IJCAI international joint conference on artificial intelligence, pp 3995\u20134001"},{"key":"2684_CR35","doi-asserted-by":"publisher","unstructured":"Yu G, Slotine JJ (2009) Audio classification from time-frequency texture. In: ICASSP, IEEE international conference on acoustics, speech and signal processing\u2014proceedings, pp 1677\u20131680. https:\/\/doi.org\/10.1109\/ICASSP.2009.4959924","DOI":"10.1109\/ICASSP.2009.4959924"},{"key":"2684_CR36","doi-asserted-by":"publisher","first-page":"643","DOI":"10.1016\/j.neucom.2017.08.043","volume":"273","author":"N Zeng","year":"2018","unstructured":"Zeng N, Zhang H, Song B, Liu W, Li Y, Dobaie AM (2018) Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273:643\u2013649. https:\/\/doi.org\/10.1016\/j.neucom.2017.08.043","journal-title":"Neurocomputing"},{"key":"2684_CR37","doi-asserted-by":"publisher","first-page":"819","DOI":"10.1109\/TNANO.2019.2932271","volume":"18","author":"N Zeng","year":"2019","unstructured":"Zeng N, Wang Z, Zhang H, Kim KE, Li Y, Liu X (2019) An improved particle filter with a novel hybrid proposal distribution for quantitative analysis of gold immunochromatographic strips. IEEE Trans Nanotechnol 18:819\u2013829. https:\/\/doi.org\/10.1109\/TNANO.2019.2932271","journal-title":"IEEE Trans Nanotechnol"},{"key":"2684_CR38","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.04.001","author":"N Zeng","year":"2020","unstructured":"Zeng N, Li H, Wang Z, Liu W, Liu S, Alsaadi FE, Liu X (2020) Deep-reinforcement-learning-based images segmentation for quantitative analysis of gold immunochromatographic strip. Neurocomputing. https:\/\/doi.org\/10.1016\/j.neucom.2020.04.001","journal-title":"Neurocomputing"}],"container-title":["Journal of Ambient Intelligence and Humanized Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12652-020-02684-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12652-020-02684-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12652-020-02684-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,31]],"date-time":"2021-08-31T19:42:41Z","timestamp":1630438961000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12652-020-02684-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,21]]},"references-count":38,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2021,10]]}},"alternative-id":["2684"],"URL":"https:\/\/doi.org\/10.1007\/s12652-020-02684-7","relation":{},"ISSN":["1868-5137","1868-5145"],"issn-type":[{"type":"print","value":"1868-5137"},{"type":"electronic","value":"1868-5145"}],"subject":[],"published":{"date-parts":[[2020,11,21]]},"assertion":[{"value":"1 May 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 November 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 November 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"On behalf of all authors, the corresponding author states that there is no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}