{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T17:58:40Z","timestamp":1776362320151,"version":"3.51.2"},"reference-count":26,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2017,6,29]],"date-time":"2017-06-29T00:00:00Z","timestamp":1498694400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Visual activity recognition plays a fundamental role in several research fields as a way to extract semantic meaning of images and videos. Prior work has mostly focused on classification tasks, where a label is given for a video clip. However, real life scenarios require a method to browse a continuous video flow, automatically identify relevant temporal segments and classify them accordingly to target activities. This paper proposes a knowledge-driven event recognition framework to address this problem. The novelty of the method lies in the combination of a constraint-based ontology language for event modeling with robust algorithms to detect, track and re-identify people using color-depth sensing (Kinect\u00ae sensor). This combination enables to model and recognize longer and more complex events and to incorporate domain knowledge and 3D information into the same models. Moreover, the ontology-driven approach enables human understanding of system decisions and facilitates knowledge transfer across different scenes. The proposed framework is evaluated with real-world recordings of seniors carrying out unscripted, daily activities at hospital observation rooms and nursing homes. Results demonstrated that the proposed framework outperforms state-of-the-art methods in a variety of activities and datasets, and it is robust to variable and low-frame rate recordings. Further work will investigate how to extend the proposed framework with uncertainty management techniques to handle strong occlusion and ambiguous semantics, and how to exploit it to further support medicine on the timely diagnosis of cognitive disorders, such as Alzheimer\u2019s disease.<\/jats:p>","DOI":"10.3390\/s17071528","type":"journal-article","created":{"date-parts":[[2017,6,29]],"date-time":"2017-06-29T10:40:04Z","timestamp":1498732804000},"page":"1528","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Online Recognition of Daily Activities by Color-Depth Sensing and Knowledge Models"],"prefix":"10.3390","volume":"17","author":[{"given":"Carlos","family":"Crispim-Junior","sequence":"first","affiliation":[{"name":"INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93, 06902 Sophia Antipolis, France"},{"name":"CobTek-Cognition Behaviour Technology, Universit\u00e9 Nice Sophia Antipolis, 06100 Nice, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alvaro","family":"G\u00f3mez Ur\u00eda","sequence":"additional","affiliation":[{"name":"INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93, 06902 Sophia Antipolis, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carola","family":"Strumia","sequence":"additional","affiliation":[{"name":"INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93, 06902 Sophia Antipolis, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michal","family":"Koperski","sequence":"additional","affiliation":[{"name":"INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93, 06902 Sophia Antipolis, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandra","family":"K\u00f6nig","sequence":"additional","affiliation":[{"name":"CobTek-Cognition Behaviour Technology, Universit\u00e9 Nice Sophia Antipolis, 06100 Nice, France"},{"name":"MUMC-School for Mental Health and Neuroscience, Alzheimer Center Limburg, Maastricht University, 6200 Maastricht, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Farhood","family":"Negin","sequence":"additional","affiliation":[{"name":"INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93, 06902 Sophia Antipolis, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Serhan","family":"Cosar","sequence":"additional","affiliation":[{"name":"INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93, 06902 Sophia Antipolis, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anh","family":"Nghiem","sequence":"additional","affiliation":[{"name":"INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93, 06902 Sophia Antipolis, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Duc","family":"Chau","sequence":"additional","affiliation":[{"name":"INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93, 06902 Sophia Antipolis, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guillaume","family":"Charpiat","sequence":"additional","affiliation":[{"name":"INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93, 06902 Sophia Antipolis, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Francois","family":"Bremond","sequence":"additional","affiliation":[{"name":"INRIA Sophia Antipolis, 2004 route des Lucioles, BP 93, 06902 Sophia Antipolis, France"},{"name":"CobTek-Cognition Behaviour Technology, Universit\u00e9 Nice Sophia Antipolis, 06100 Nice, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,6,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Fleury, A., Noury, N., and Vacher, M. (2010, January 1\u20133). Introducing knowledge in the process of supervised classification of activities of Daily Living in Health Smart Homes. Proceedings of the 12th IEEE International Conference on e-Health Networking Applications and Services, Lyon, France.","DOI":"10.1109\/HEALTH.2010.5556549"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Medjahed, H., Istrate, D., Boudy, J., Baldinger, J.L., and Dorizzi, B. (2011, January 27\u201330). A pervasive multi-sensor data fusion for smart home healthcare monitoring. Proceedings of the IEEE International Conference on Fuzzy Systems, Taipei, Taiwan.","DOI":"10.1109\/FUZZY.2011.6007636"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Crispim-Junior, C., Bathrinarayanan, V., Fosty, B., Konig, A., Romdhane, R., Thonnat, M., and Bremond, F. (2013, January 27\u201331). Evaluation of a Monitoring System for Event Recognition of Older People. Proceedings of the the 10th IEEE International Conference on Advanced Video and Signal-Based Surveillance 2013 ( AVSS 2013), Krakow, Poland.","DOI":"10.1109\/AVSS.2013.6636634"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.cviu.2015.04.005","article-title":"Recognizing Complex Instrumental Activities of Daily Living Using Scene Information and Fuzzy Logic","volume":"140","author":"Banerjee","year":"2015","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/j.neucom.2012.08.036","article-title":"Statistical data mining of streaming motion data for activity and fall recognition in assistive environments","volume":"107","author":"Tasoulis","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1016\/j.cmpb.2013.10.014","article-title":"A Kinect-based system for cognitive rehabilitation exercises monitoring","volume":"113","year":"2014","journal-title":"Comput. Methods Progr. Biomed."},{"key":"ref_7","unstructured":"Vu, T., Bremond, F., and Thonnat, M. (2003, January 9\u201315). Automatic Video Interpretation: A Novel Algorithm for Temporal Scenario Recognition. Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI\u201903), Acapulco, Mexico."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Cao, Y., Tao, L., and Xu, G. (2009, January 7\u20139). An event-driven context model in elderly health monitoring. Proceedings of the Symposia and Workshops on Ubiquitous, Autonomic and Trusted Computing, Brisbane, Australia.","DOI":"10.1109\/UIC-ATC.2009.47"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1109\/TSMCC.2012.2198883","article-title":"Sensor-Based Activity Recognition","volume":"42","author":"Chen","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. Part C Appl. Rev."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1007\/s00138-006-0017-3","article-title":"Ontological inference for image and video analysis","volume":"17","author":"Town","year":"2006","journal-title":"Mach. Vis. Appl."},{"key":"ref_11","unstructured":"Ceusters, W., Corso, J.J., Fu, Y., Petropoulos, M., and Krovi, V. (2010, January 26\u201329). Introducing Ontological Realism for Semi-Supervised Detection and Annotation of Operationally Significant Activity in Surveillance Videos. Proceedings of the the 5th International Conference on Semantic Technologies for Intelligence, Defense and Security (STIDS), Fairfax, VA, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1109\/THMS.2013.2293714","article-title":"An Ontology-Based Hybrid Approach to Activity Modeling for Smart Homes","volume":"44","author":"Chen","year":"2014","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"13","DOI":"10.3928\/00989134-20131126-01","article-title":"Automated Fall Detection With Quality Improvement \u201cRewind\u201d to Reduce Falls in Hospital Rooms","volume":"40","author":"Rantz","year":"2014","journal-title":"J. Gerontol. Nurs."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tran, S.D., and Davis, L.S. (2008, January 12\u201318). Event Modeling and Recognition Using Markov Logic Networks. Proceedings of the the 10th European Conference on Computer Vision (ECCV 2008), Marseille, France.","DOI":"10.1007\/978-3-540-88688-4_45"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Kitani, K.M., Ziebart, B.D., Bagnell, J.A.D., and Hebert, M. (2012, January 7\u201313). Activity Forecasting. Proceedings of the European Conference on Computer Vision, Florence, Italy.","DOI":"10.1007\/978-3-642-33765-9_15"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kwak, S., Han, B., and Han, J.H. (2011, January 20\u201325). Scenario-based video event recognition by constraint flow. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995435"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Brendel, W., Fern, A., and Todorovic, S. (2011, January 20\u201325). Probabilistic event logic for interval-based event recognition. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995491"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Nghiem, A.T., and Bremond, F. (2014, January 26\u201329). Background subtraction in people detection framework for RGB-D cameras. Proceedings of the 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Seoul, Korea.","DOI":"10.1109\/AVSS.2014.6918675"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Pramerdorfer, C. (2013, January 12\u201314). Evaluation of Kinect Sensors for Fall Detection. Proceedings of the IASTED International Conference on Signal Processing, Pattern Recognition and Applications (SPPRA 2013), Innsbruck, Austria.","DOI":"10.2316\/P.2013.798-120"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. (2011, January 20\u201325). Real-time Human Pose Recognition in Parts from Single Depth Images. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995316"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1002\/nav.3800020109","article-title":"The Hungarian Method for the assignment problem","volume":"2","author":"Kuhn","year":"1955","journal-title":"Nav. Res. Logist. Q."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Chau, D.P., Thonnat, M., and Bremond, F. (2013, January 16\u201318). Automatic parameter adaptation for multi-object tracking. Proceedings of the International Conference on Computer Vision Systems, St. Petersburg, Russia.","DOI":"10.1007\/978-3-642-39402-7_25"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1145\/182.358434","article-title":"Maintaining Knowledge About Temporal Intervals","volume":"26","author":"Allen","year":"1983","journal-title":"Commun. ACM"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, H., Klaser, A., Schmid, C., and Liu, C.L. (2011, January 20\u201325). Action Recognition by Dense Trajectories. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995407"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"812","DOI":"10.1001\/archpsyc.1983.01790060110016","article-title":"THe mini-mental state examination","volume":"40","author":"Folstein","year":"1983","journal-title":"Arch. Gen. Psychiatry"},{"key":"ref_26","unstructured":"Karakostas, A., Briassouli, A., Avgerinakis, K., Kompatsiaris, I., and Tsolaki, M. (2014). The Dem@Care Experiments and Datasets: A Technical Report, Centre for Research and Technology Hellas. Technical Report."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/7\/1528\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:40:49Z","timestamp":1760208049000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/7\/1528"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,6,29]]},"references-count":26,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2017,7]]}},"alternative-id":["s17071528"],"URL":"https:\/\/doi.org\/10.3390\/s17071528","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,6,29]]}}}