{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T04:25:12Z","timestamp":1772252712114,"version":"3.50.1"},"reference-count":35,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2017,7,23]],"date-time":"2017-07-23T00:00:00Z","timestamp":1500768000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Affective computing in general and human activity and intention analysis in particular comprise a rapidly-growing field of research. Head pose and emotion changes present serious challenges when applied to player\u2019s training and ludology experience in serious games, or analysis of customer satisfaction regarding broadcast and web services, or monitoring a driver\u2019s attention. Given the increasing prominence and utility of depth sensors, it is now feasible to perform large-scale collection of three-dimensional (3D) data for subsequent analysis. Discriminative random regression forests were selected in order to rapidly and accurately estimate head pose changes in an unconstrained environment. In order to complete the secondary process of recognising four universal dominant facial expressions (happiness, anger, sadness and surprise), emotion recognition via facial expressions (ERFE) was adopted. After that, a lightweight data exchange format (JavaScript Object Notation (JSON)) is employed, in order to manipulate the data extracted from the two aforementioned settings. Motivated by the need to generate comprehensible visual representations from different sets of data, in this paper, we introduce a system capable of monitoring human activity through head pose and emotion changes, utilising an affordable 3D sensing technology (Microsoft Kinect sensor).<\/jats:p>","DOI":"10.3390\/computers6030025","type":"journal-article","created":{"date-parts":[[2017,7,24]],"date-time":"2017-07-24T10:04:13Z","timestamp":1500890653000},"page":"25","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Conceiving Human Interaction by Visualising Depth Data of Head Pose Changes and Emotion Recognition via Facial Expressions"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2194-7709","authenticated-orcid":false,"given":"Grigorios","family":"Kalliatakis","sequence":"first","affiliation":[{"name":"School of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandros","family":"Stergiou","sequence":"additional","affiliation":[{"name":"School of Computer Science and Electronic Engineering, University of Essex, Colchester CO4 3SQ, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nikolaos","family":"Vidakis","sequence":"additional","affiliation":[{"name":"Department of Informatics Engineering, Technological Educational Institute of Crete, Stauromenos 71410, Heraklion, Crete, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,7,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kalliatakis, G., Vidakis, N., and Triantafyllidis, G. (2016, January 28\u201330). Web-based visualisation of head pose and facial expressions changes: Monitoring human activity using depth data. Proceedings of the 8th Computer Science and Electronic Engineering (CEEC), Colchester, UK.","DOI":"10.1109\/CEEC.2016.7835887"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kalliatakis, G., Triantafyllidis, G., and Vidakis, N. (2015, January 18\u201321). Head pose 3D data web-based visualization. Proceedings of the 20th International Conference on 3D Web Technology (Web3D \u201915), Heraklion, Greece.","DOI":"10.1145\/2775292.2778304"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1109\/TPAMI.2008.106","article-title":"Head Pose Estimation in Computer Vision: A Survey","volume":"31","author":"Trivedi","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Breitenstein, M.D., Kuettel, D., Weise, T., van Gool, L., and Pfister, H. (2008, January 23\u201328). Real-time face pose estimation from single range images. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.","DOI":"10.1109\/CVPR.2008.4587807"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Fanelli, G., Gall, J., and Van Gool, L. (2011, January 20\u201325). Real time head pose estimation with random regression forests. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995458"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Padeleris, P., Zabulis, X., and Argyros, A.A. (2012, January 16\u201321). Head pose estimation on depth data based on Particle Swarm Optimization. Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA.","DOI":"10.1109\/CVPRW.2012.6239236"},{"key":"ref_7","unstructured":"Wu, Y., Liu, H., and Zha, H. (2005, January 2\u20136). Modeling facial expression space for recognition. Proceedings of the 2005 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AB, Canada."},{"key":"ref_8","unstructured":"Anil, J.K., and Li, S.Z. (2011). Handbook of Face Recognition, Springer."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Fang, T., Xi, Z., Ocegueda, O., Shah, S.K., and Kakadiaris, I.A. (2011, January 21\u201325). 3D facial expression recognition: A perspective on promises and challenges. Proceedings of the 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), Santa Barbara, CA, USA.","DOI":"10.1109\/FG.2011.5771466"},{"key":"ref_10","unstructured":"Xue, M., Mian, A., Liu, W., and Li, L. (2014, January 24\u201326). Fully automatic 3D facial expression recognition using local depth features. Proceedings of the 2014 IEEE Winter Conference Applications of Computer Vision (WACV), Steamboat Springs, CO, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Azazi, A., Lutfi, S.L., and Venkat, I. (2014, January 8\u201310). Analysis and evaluation of SURF descriptors for automatic 3D facial expression recognition using different classifiers. Proceedings of the 2014 Fourth World Congress on Information and Communication Technologies (WICT), Malacca, Malaysia.","DOI":"10.1109\/WICT.2014.7077296"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1007\/s12193-015-0209-0","article-title":"Hierarchical committee of deep convolutional neural networks for robust facial expression recognition","volume":"2","author":"Kim","year":"2016","journal-title":"J. Multimodal User Interfaces"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1438","DOI":"10.1109\/TMM.2016.2557063","article-title":"Muscular Movement Model-Based Automatic 3D\/4D Facial Expression Recognition","volume":"18","author":"Zhen","year":"2016","journal-title":"IEEE Trans. Multimed."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1007\/s00530-014-0400-2","article-title":"Facial expression recognition using active contour-based face detection, facial movement-based feature extraction, and non-linear feature selection","volume":"21","author":"Siddiqi","year":"2015","journal-title":"Multimed. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"738","DOI":"10.1016\/j.imavis.2012.02.004","article-title":"3D\/4D facial expression analysis: An advanced annotated face model approach","volume":"30","author":"Fang","year":"2012","journal-title":"Image Vis. Comput."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1198\/tech.2002.s78","article-title":"The visual display of quantitative information","volume":"44","author":"Mulrow","year":"2002","journal-title":"Technometrics"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1016\/j.neucom.2013.08.008","article-title":"Human intention recognition based on eyeball movement pattern and pupil size variation","volume":"128","author":"Jang","year":"2014","journal-title":"Neurocomputing"},{"key":"ref_18","first-page":"13","article-title":"Intention recognition using a graph representation","volume":"25","author":"Youn","year":"2007","journal-title":"World Acad. Sci. Eng. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Vidakis, N., Vlasopoulos, A., Kounalakis, T., Varchalamas, P., Dimitriou, M., Kalliatakis, G., Syntychakis, E., Christofakis, J., and Triantafyllidis, G. (2013, January 1\u20133). Multimodal desktop interaction: The face-object-gesture-voice example. Proceedings of the 2013 18th IEEE International Conference on Digital Signal Processing (DSP), Fira, Greece.","DOI":"10.1109\/ICDSP.2013.6622782"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Vidakis, N., Syntychakis, E., Kalafatis, K., Christinaki, E., and Triantafyllidis, G. (2015, January 2\u20137). Ludic Educational Game Creation Tool: Teaching Schoolers Road Safety. Proceedings of the 9th International Conference on Universal Access in Human-Computer Interaction, Los Angeles, CA, USA.","DOI":"10.1007\/978-3-319-20684-4_55"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Vidakis, N., Christinaki, E., Serafimidis, I., and Triantafyllidis, G. (2014, January 22\u201327). Combining ludology and narratology in an open authorable framework for educational games for children: The scenario of teaching preschoolers with autism diagnosis. Proceedings of the International Conference on Universal Access in Human-Computer Interaction, Heraklion, Greece.","DOI":"10.1007\/978-3-319-07440-5_57"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1167\/14.13.14","article-title":"Eye movements during emotion recognition in faces","volume":"14","author":"Schurgin","year":"2014","journal-title":"J. Vis."},{"key":"ref_23","unstructured":"Salgado, A. (2005). The facial and vocal expression in singers: A cognitive feedback study for improving emotional expression in solo vocal music performance. Electr. Musicol. Rev., 9."},{"key":"ref_24","unstructured":"Neidle, C., Liu, J., Liu, B., Peng, X., Vogler, C., and Metaxas, D. (2014, January 31). Computer-based tracking, analysis, and visualization of linguistically significant nonmanual events in American Sign Language (ASL). Proceedings of the LREC Workshop on the Representation and Processing of Sign Languages: Beyond the Manual Channel, Reykjavik, Iceland."},{"key":"ref_25","first-page":"459","article-title":"Edge Based Grid Super-Imposition for Crowd Emotion Recognition","volume":"3","author":"Patwardhan","year":"2016","journal-title":"Int. Res. J. Eng. Technol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1504\/IJCVR.2015.072194","article-title":"Estimating driver head pose using steerable pyramid and probabilistic learning","volume":"5","author":"Alioua","year":"2015","journal-title":"Int. J. Comput. Vis. Robot."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Vatahska, T., Bennewitz, M., and Behnke, S. (December, January 29). Feature-based head pose estimation from images. Proceedings of the 2007 7th IEEE-RAS International Conference on Humanoid Robots, Pittsburgh, PA, USA.","DOI":"10.1109\/ICHR.2007.4813889"},{"key":"ref_28","unstructured":"Kalliatakis, G. (2015). Towards an Automatic Intelligible Monitoring of Behavioral and Physiological Metrics of User Experience: Head Pose Estimation and Facial Expression Recognition. [Master\u2019s Thesis, Department of Applied Informatics and Multimedia]."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Fanelli, G., Weise, T., Gall, J., and Van Gool, L. (2011). Real time head pose estimation from consumer depth cameras. Joint Pattern Recognition Symposium, Springer.","DOI":"10.1007\/978-3-642-23123-0_11"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1007\/s11263-012-0549-0","article-title":"Random forests for real time 3D face analysis","volume":"101","author":"Fanelli","year":"2013","journal-title":"Int. J. Comput. Vis."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1109\/34.908962","article-title":"Recognizing action units for facial expression analysis","volume":"23","author":"Tian","year":"2001","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1631\/FITEE.1400209","article-title":"Using Kinect for real-time emotion recognition via facial expressions","volume":"16","author":"Mao","year":"2015","journal-title":"Front. Inf. Technol. Electr. Eng."},{"key":"ref_33","unstructured":"(2017, May 30). Microsoft, Face Tracking SDK Documentation. Available online: https:\/\/msdn.microsoft.com\/en-us\/library\/jj130970.aspx."},{"key":"ref_34","unstructured":"Ahlberg, J. (2017, July 21). Candide-3-An Updated Parameterised Face. Available online: http:\/\/citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.33.5603rep=rep1type=pdf."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Lin, B., Chen, Y., Chen, X., and Yu, Y. (2012, January 11\u201313). Comparison between JSON and XML in Applications Based on AJAX. Proceedings of the 2012 IEEE International Conference on Computer Science and Service System (CSSS), Nanjing, China.","DOI":"10.1109\/CSSS.2012.297"}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/6\/3\/25\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:43:44Z","timestamp":1760208224000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/6\/3\/25"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,7,23]]},"references-count":35,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2017,9]]}},"alternative-id":["computers6030025"],"URL":"https:\/\/doi.org\/10.3390\/computers6030025","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints201706.0003.v1","asserted-by":"object"},{"id-type":"doi","id":"10.20944\/preprints201706.0003.v2","asserted-by":"object"}]},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,7,23]]}}}