{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T20:32:17Z","timestamp":1776889937145,"version":"3.51.2"},"reference-count":36,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2018,8,27]],"date-time":"2018-08-27T00:00:00Z","timestamp":1535328000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61401002"],"award-info":[{"award-number":["61401002"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100009558","name":"University Natural Science Research Project of Anhui Province","doi-asserted-by":"publisher","award":["KJ2018A0008"],"award-info":[{"award-number":["KJ2018A0008"]}],"id":[{"id":"10.13039\/501100009558","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Facing the adolescents and detecting their emotional state is vital for promoting rehabilitation therapy within an E-Healthcare system. Focusing on a novel approach for a sensor-based E-Healthcare system, we propose an eye movement information-based emotion perception algorithm by collecting and analyzing electrooculography (EOG) signals and eye movement video synchronously. Specifically, we extract the time-frequency eye movement features by firstly applying the short-time Fourier transform (STFT) to raw multi-channel EOG signals. Subsequently, in order to integrate time domain eye movement features (i.e., saccade duration, fixation duration, and pupil diameter), we investigate two feature fusion strategies: feature level fusion (FLF) and decision level fusion (DLF). Recognition experiments have been also performed according to three emotional states: positive, neutral, and negative. The average accuracies are 88.64% (the FLF method) and 88.35% (the DLF with maximal rule method), respectively. Experimental results reveal that eye movement information can effectively reflect the emotional state of the adolescences, which provides a promising tool to improve the performance of the E-Healthcare system.<\/jats:p>","DOI":"10.3390\/s18092826","type":"journal-article","created":{"date-parts":[[2018,8,27]],"date-time":"2018-08-27T10:56:04Z","timestamp":1535367364000},"page":"2826","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Automatic Emotion Perception Using Eye Movement Information for E-Healthcare Systems"],"prefix":"10.3390","volume":"18","author":[{"given":"Yang","family":"Wang","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Anhui University, Hefei 230601, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9727-366X","authenticated-orcid":false,"given":"Zhao","family":"Lv","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Anhui University, Hefei 230601, China"},{"name":"Institute of Physical Science and Information Technology, Anhui University, Hefei 230601, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yongjun","family":"Zheng","sequence":"additional","affiliation":[{"name":"School of Electronics, Computing and Mathematics, University of Derby, Derby DE22 3AW, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,8,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1426","DOI":"10.1108\/IMDS-08-2016-0342","article-title":"Development of an intelligent e-healthcare system for the domestic care industry","volume":"117","author":"Wong","year":"2017","journal-title":"Ind. Manag. Data Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sodhro, A.H., Sangaiah, A.K., Sodhro, G.H., Lohano, S., and Pirbhulal, S. (2018). An Energy-Efficient Algorithm for Wearable Electrocardiogram Signal Processing in Ubiquitous Healthcare Applications. Sensors, 18.","DOI":"10.3390\/s18030923"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"11770","DOI":"10.3390\/s140711770","article-title":"Physiological sensor signals classification for healthcare using sensor data fusion and case-based reasoning","volume":"14","author":"Begum","year":"2014","journal-title":"Sensors"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kart, F., Miao, G., Moser, L.E., and Melliar-Smith, P.M. (2007, January 9\u201313). A distributed e-healthcare system based on the service oriented architecture. Proceedings of the 2007 IEEE International Conference on Services Computing, Salt Lake City, UT, USA.","DOI":"10.1109\/SCC.2007.2"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1007\/s10586-016-0535-3","article-title":"Automatic facial emotion recognition using weber local descriptor for e-Healthcare system","volume":"19","author":"Alhussein","year":"2016","journal-title":"Cluster Comput."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Banos, O., Villalonga, C., Bang, J., Hur, T., Kang, D., Park, S., and Hong, C.S. (2016). Human behavior analysis by means of multimodal context mining. Sensors, 16.","DOI":"10.3390\/s16081264"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"10871","DOI":"10.1109\/ACCESS.2017.2712788","article-title":"A facial-expression monitoring system for improved healthcare in smart cities","volume":"5","author":"Muhammad","year":"2017","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"13361","DOI":"10.3390\/s140813361","article-title":"Emotion recognition from single-trial EEG based on kernel Fisher\u2019s emotion pattern and imbalanced quasiconformal kernel support vector machine","volume":"18","author":"Liu","year":"2014","journal-title":"Sensors"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"15549","DOI":"10.3390\/s131115549","article-title":"A multimodal emotion detection system during human\u2013robot interaction","volume":"13","author":"Malfaz","year":"2013","journal-title":"Sensors"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1016\/j.neucom.2016.05.113","article-title":"Discriminative extreme learning machine with supervised sparsity preserving for image classification","volume":"261","author":"Peng","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2447","DOI":"10.1109\/TC.2014.2378273","article-title":"An adaptive multilevel indexing method for disaster service discovery","volume":"64","author":"Wu","year":"2015","journal-title":"IEEE Trans. Comput."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1302","DOI":"10.1016\/j.neucom.2008.11.007","article-title":"Analysis of positive and negative emotions in natural scene using brain activity and GIST","volume":"72","author":"Zhang","year":"2009","journal-title":"Neurocomputing"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kwon, Y.H., Shin, S.B., and Kim, S.D. (2018). Electroencephalography Based Fusion Two-Dimensional(2D)-Convolution Neural Networks (CNN) Model for Emotion Recognition System. Sensors, 18.","DOI":"10.3390\/s18051383"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.1109\/TBME.2010.2048568","article-title":"EEG-based emotion recognition in music listening","volume":"57","author":"Lin","year":"2010","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Santella, A., and DeCarlo, D. (2004, January 22\u201324). Robust clustering of eye movement recordings for quantification of visual interest. Proceedings of the 2004 Symposium on Eye Tracking Research & Applications, San Antonio, TX, USA.","DOI":"10.1145\/968363.968368"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"397","DOI":"10.3758\/BF03201553","article-title":"Survey of eye movement recording methods","volume":"7","author":"Young","year":"1975","journal-title":"Behav. Res. Methods"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1007\/s10964-016-0585-0","article-title":"Emotion regulation strategies in depressive and anxiety symptoms in youth: A meta-analytic review","volume":"46","author":"Naumann","year":"2017","journal-title":"J. Youth Adolesc."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1126\/science.1260497","article-title":"Adolescent mental healthopportunity and obligation","volume":"346","author":"Lee","year":"2014","journal-title":"Science"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1196\/annals.1440.010","article-title":"The adolescent brain","volume":"1124","author":"Casey","year":"2008","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1038\/nrn2513","article-title":"Why do many psychiatric disorders emerge during adolescence?","volume":"9","author":"Paus","year":"2008","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1016\/S0149-7634(00)00014-2","article-title":"The adolescent brain and age-related behavioral manifestations","volume":"24","author":"Spear","year":"2000","journal-title":"Neurosci. Biobehav. Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S1071-5819(03)00017-X","article-title":"Pupil size variation as an indication of affective processing","volume":"59","author":"Partala","year":"2003","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1111\/j.1469-8986.2008.00654.x","article-title":"The pupil as a measure of emotional arousal and autonomic activation","volume":"45","author":"Bradley","year":"2008","journal-title":"Psychophysiology"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1016\/j.neuron.2008.03.027","article-title":"Transient induced gamma-band response in EEG as a manifestation of miniature saccades","volume":"58","author":"Tomer","year":"2008","journal-title":"Neuron"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"10076","DOI":"10.1038\/s41598-017-10683-6","article-title":"Uncovering the cognitive processes underlying mental rotation: An eye-movement study","volume":"7","author":"Xue","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"741","DOI":"10.1109\/TPAMI.2010.86","article-title":"Eye movement analysis for activity recognition using electrooculography","volume":"33","author":"Bulling","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_27","unstructured":"Duchowski, A.T. (2007). Eye Tracking Methodology, Springer. [2nd ed.]."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/S0165-0270(03)00151-1","article-title":"Defining the temporal threshold for ocular fixation in free-viewing visuocognitive tasks","volume":"128","author":"Manor","year":"2003","journal-title":"J. Neurosci. Methods"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3084","DOI":"10.1109\/78.330368","article-title":"The fractional Fourier transform and time-frequency representations","volume":"42","author":"Almeida","year":"1994","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_30","unstructured":"Lu, Y., Zheng, W.L., Li, B., and Lu, B.L. (2015, January 25\u201331). Combining Eye Movements and EEG to Enhance Emotion Recognition. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina."},{"key":"ref_31","first-page":"39","article-title":"Interest-aware content discovery in peer-to-peer social networks","volume":"18","author":"Guo","year":"2017","journal-title":"ACM Trans. Internet Technol."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1145\/2811264","article-title":"A socioecological model for advanced service discovery in machine-to-machine communication networks","volume":"15","author":"Liu","year":"2016","journal-title":"ACM Trans. Embed. Comput. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1111\/j.1469-8986.1986.tb00678.x","article-title":"EOG-Based Recording and Automated Detection of Sleep Rapid Eye Movements: A Critical Review, and Some Recommendations","volume":"23","author":"Boukadoum","year":"1986","journal-title":"Psychophysiology"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"17997","DOI":"10.1109\/ACCESS.2017.2750701","article-title":"A Robust Online Saccadic Eye Movement Recognition Method Combining Electrooculography and Video","volume":"5","author":"Ding","year":"2017","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/0262-8856(90)90059-E","article-title":"Comparative study of Hough transform methods for circle finding","volume":"8","author":"Yuen","year":"1990","journal-title":"Image Vis. Comput."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/5.554205","article-title":"An introduction to multisensor data fusion","volume":"85","author":"Hall","year":"1997","journal-title":"Proc. IEEE"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/2826\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:21:23Z","timestamp":1760196083000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/2826"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8,27]]},"references-count":36,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2018,9]]}},"alternative-id":["s18092826"],"URL":"https:\/\/doi.org\/10.3390\/s18092826","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,8,27]]}}}