{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T04:30:04Z","timestamp":1772253004559,"version":"3.50.1"},"reference-count":67,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,5,2]],"date-time":"2022-05-02T00:00:00Z","timestamp":1651449600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"MUR\u2014Italian Ministry for University and Research","award":["ARS_01_01120"],"award-info":[{"award-number":["ARS_01_01120"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Predicting change from multivariate time series has relevant applications ranging from the medical to engineering fields. Multisensory stimulation therapy in patients with dementia aims to change the patient\u2019s behavioral state. For example, patients who exhibit a baseline of agitation may be paced to change their behavioral state to relaxed. This study aimed to predict changes in one\u2019s behavioral state from the analysis of the physiological and neurovegetative parameters to support the therapist during the stimulation session. In order to extract valuable indicators for predicting changes, both handcrafted and learned features were evaluated and compared. The handcrafted features were defined starting from the CATCH22 feature collection, while the learned ones were extracted using a temporal convolutional network, and the behavioral state was predicted through bidirectional long short-term memory auto-encoder, operating jointly. From the comparison with the state of the art, the learned features-based approach exhibits superior performance with accuracy rates of up to 99.42% with a time window of 70 seconds and up to 98.44% with a time window of 10 seconds.<\/jats:p>","DOI":"10.3390\/s22093468","type":"journal-article","created":{"date-parts":[[2022,5,3]],"date-time":"2022-05-03T08:26:35Z","timestamp":1651566395000},"page":"3468","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Behavioral Change Prediction from Physiological Signals Using Deep Learned Features"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9737-3721","authenticated-orcid":false,"given":"Giovanni","family":"Diraco","sequence":"first","affiliation":[{"name":"National Research Council of Italy, IMM\u2014Institute for Microelectronics and Microsystems, 73100 Lecce, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pietro","family":"Siciliano","sequence":"additional","affiliation":[{"name":"National Research Council of Italy, IMM\u2014Institute for Microelectronics and Microsystems, 73100 Lecce, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8970-3313","authenticated-orcid":false,"given":"Alessandro","family":"Leone","sequence":"additional","affiliation":[{"name":"National Research Council of Italy, IMM\u2014Institute for Microelectronics and Microsystems, 73100 Lecce, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"115641","DOI":"10.1016\/j.eswa.2021.115641","article-title":"Embedding-based real-time change point detection with application to activity segmentation in smart home time series data","volume":"185","author":"Bermejo","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"106538","DOI":"10.1016\/j.cemconres.2021.106538","article-title":"Screening candidate supplementary cementitious materials under standard and accelerated curing through time-series surface resistivity measurements and change-point detection. detection","volume":"148","author":"Rios","year":"2021","journal-title":"Cem. Concr. Res."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"109841","DOI":"10.1016\/j.enbuild.2020.109841","article-title":"The application of quality control charts for identifying changes in time-series home energy data","volume":"215","author":"Ivancos","year":"2020","journal-title":"Energy Build."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"e3104","DOI":"10.1212\/WNL.0000000000010747","article-title":"Sequence of Alzheimer disease biomarker changes in cognitively normal adults: A cross-sectional study","volume":"95","author":"Luo","year":"2020","journal-title":"Neurology"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"180168","DOI":"10.1109\/ACCESS.2019.2956768","article-title":"Automated change-point detection of EEG signals based on structural time-series analysis","volume":"7","author":"Chen","year":"2019","journal-title":"IEEE Access"},{"key":"ref_6","unstructured":"Boccadamo, A., Luca, A.D., Palam\u00e0, L., Sancarlo, D., Leone, A., Diraco, G., Siciliano, P., Toma, A., Maffia, M., and De Riccardis, L. (2019). Multi Sensorial Stimulation Lab: A New Approach for Severe Dementia. Italian Forum of Ambient Assisted Living, Springer."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1037\/0033-295X.98.4.459","article-title":"Autonomic determinism: The modes of autonomic control, the doctrine of autonomic space, and the laws of auto-nomic constraint","volume":"98","author":"Berntson","year":"1991","journal-title":"Psychol. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1017\/S0954579401002012","article-title":"Vagal tone, development, and Gray\u2019s motivational theory: To-ward an integrated model of autonomic nervous system functioning in psychopatholo-gy","volume":"13","author":"Beauchaine","year":"2001","journal-title":"Dev. Psychopathol."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Goldstein, S., Naglieri, J.A., Princiotta, D., and Otero, T.M. (2014). Introduction: A his-tory of executive functioning as a theoretical and clinical construct. Handbook of Executive Functioning, Springer.","DOI":"10.1007\/978-1-4614-8106-5_1"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1016\/j.ejphar.2007.11.071","article-title":"Central effects of stress hormones in health and disease: Under-standing the protective and damaging effects of stress and stress mediators","volume":"583","author":"McEwen","year":"2008","journal-title":"Eur. J. Pharmacol."},{"key":"ref_11","unstructured":"Cacioppo, J.T., Tassinary, L.G., and Berntson, G.G. (2007). Handbook of Psychophysiology, Cambridge University Press."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1016\/0166-2236(92)90092-M","article-title":"Characteristics of function-specific pathways in the sympathetic nervous system","volume":"15","author":"McLachlan","year":"1992","journal-title":"Trends Neurosci."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Sakr, G.E., Elhajj, I.H., and Wejinya, U.C. (2009, January 14\u201317). Multi level SVM for subject independent agitation detection. Proceedings of the 2009 IEEE\/ASME International Conference on Advanced Intelligent Mechatronics, Singapore.","DOI":"10.1109\/AIM.2009.5229958"},{"key":"ref_14","first-page":"1672","article-title":"Using noninvasive wearable computers to recognize human emotions from physiological signals","volume":"2004","author":"Lisetti","year":"2004","journal-title":"EURASIP J. Appl. Signal Processing"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/S0167-8760(97)00087-1","article-title":"Fingertip temperature as an indicator for sympathetic responses","volume":"29","author":"Kistler","year":"1998","journal-title":"Int. J. Psychophysiol."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Fernandes, A., Helawar, R., Lokesh, R., Tari, T., and Shahapurkar, A.V. (2014, January 18\u201319). Determination of stress using blood pressure and galvanic skin response. Proceedings of the 2014 International Conference on Communication and Network Technologies, Sivakasi, India.","DOI":"10.1109\/CNT.2014.7062747"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.patrec.2014.01.008","article-title":"A review of unsupervised feature learning and deep learning for time-series modeling","volume":"42","author":"Karlsson","year":"2014","journal-title":"Pattern Recognit. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1109\/TITS.2005.848368","article-title":"Detecting stress during real-world driving tasks using physiological sensors","volume":"6","author":"Healey","year":"2005","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_19","unstructured":"Mitchell, H.B. (2007). Multi-Sensor Data Fusion: An Introduction, Springer Science & Business Media."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhang, L., Tamminedi, T., Ganguli, A., Yosiphon, G., and Yadegar, J. (2010). Hierarchical multiple sensor fusion using structurally learned Bayesian network. Wireless Health, Association for Computing Machinery.","DOI":"10.1145\/1921081.1921102"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1093\/oxfordjournals.eurheartj.a014868","article-title":"Heart rate variability: Standards of measurement, physiological interpretation, and clinical use","volume":"17","author":"Malik","year":"1996","journal-title":"Eur. Heart J."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1016\/j.neucom.2011.10.047","article-title":"A k-nearest-neighbor classifier with heart rate variability feature-based transformation algorithm for driving stress recognition","volume":"116","author":"Wang","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"833","DOI":"10.1007\/s40846-015-0095-7","article-title":"Ecg-based mental stress assessment using fuzzy computing and associative petri net","volume":"35","author":"Chiang","year":"2015","journal-title":"J. Med. Biol. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/j.eswa.2017.01.040","article-title":"Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiers","volume":"85","author":"Chen","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.bspc.2017.05.003","article-title":"Reaction time and physiological signals for stress recognition","volume":"38","author":"Zhang","year":"2017","journal-title":"Biomed. Signal Processing Control."},{"key":"ref_26","unstructured":"Dong, G., and Liu, H. (2018). Feature Engineering for Machine Learning and Data Analytics, CRC Press."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1016\/j.cels.2017.10.001","article-title":"hctsa: A computational framework for automated time-series phenotyping using massive feature extraction","volume":"5","author":"Fulcher","year":"2017","journal-title":"Cell Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1821","DOI":"10.1007\/s10618-019-00647-x","article-title":"catch22: Canonical time-series characteristics","volume":"33","author":"Lubba","year":"2019","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep Learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000006","article-title":"Learning deep architectures for AI","volume":"2","author":"Bengio","year":"2009","journal-title":"Found. Trends Mach. Learn."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Bengio, Y. (2012). Practical recommendations for gradient-based training of deep architectures. Neural Networks: Tricks of the Trade, Springer.","DOI":"10.1007\/978-3-642-35289-8_26"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3303","DOI":"10.1109\/TITS.2020.2980555","article-title":"An ensemble classification model with unsupervised representation learning for driving stress recognition using physiological signals","volume":"22","author":"Wang","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Guin, A. (2006, January 17\u201320). Travel time prediction using a seasonal autoregressive integrated moving average time series model. Proceedings of the 2006 IEEE Intelligent Transportation Systems Conference, Toronto, ON, Canada.","DOI":"10.1109\/ITSC.2006.1706789"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1109\/72.279181","article-title":"Learning long-term dependencies with gradient descent is difficult","volume":"5","author":"Bengio","year":"1994","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1186\/s40708-021-00133-5","article-title":"Evaluating deep learning EEG-based mental stress classification in adolescents with autism for breathing entrainment BCI","volume":"8","author":"Sundaresan","year":"2021","journal-title":"Brain Inform."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1544","DOI":"10.1109\/LRA.2018.2801475","article-title":"A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder","volume":"3","author":"Park","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"466","DOI":"10.1016\/j.neunet.2018.09.002","article-title":"Soft+ hardwired attention: An lstm framework for human trajectory prediction and abnormal event detection","volume":"108","author":"Fernando","year":"2018","journal-title":"Neural Netw."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"3127","DOI":"10.1109\/TNNLS.2019.2935975","article-title":"Unsupervised anomaly detection with LSTM neural networks","volume":"31","author":"Ergen","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1007\/s13748-019-00203-0","article-title":"Convolutional neural network: A review of models, methodologies and applications to object detection","volume":"9","author":"Dhillon","year":"2020","journal-title":"Prog. Artif. Intell."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Radovic, M., Adarkwa, O., and Wang, Q. (2017). Object recognition in aerial images using convolutional neural networks. J. Imaging, 3.","DOI":"10.3390\/jimaging3020021"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1016\/j.patcog.2017.10.013","article-title":"Recent advances in convolutional neural networks","volume":"77","author":"Gu","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1109\/TPAMI.2010.125","article-title":"Convolutional neural networks for P300 detection with application to brain-computer interfaces","volume":"33","author":"Cecotti","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1016\/j.image.2016.05.018","article-title":"EEG-based prediction of driver\u2019s cognitive performance by deep convolutional neural network","volume":"47","author":"Hajinoroozi","year":"2016","journal-title":"Signal Processing Image Commun."},{"key":"ref_45","unstructured":"Bai, S., Kolter, J.Z., and Koltun, V. (2018). An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"106063","DOI":"10.1016\/j.cmpb.2021.106063","article-title":"Automatic sleep stage classification using temporal convolutional neural network and new data augmentation technique from raw single-channel EEG","volume":"204","author":"Khalili","year":"2021","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"2755","DOI":"10.1109\/TNNLS.2018.2886414","article-title":"EEG-based spatio\u2013temporal convolutional neural network for driver fatigue evaluation","volume":"30","author":"Gao","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"112766","DOI":"10.1016\/j.cma.2019.112766","article-title":"Data-driven reduced order model with temporal convolutional neural network","volume":"360","author":"Wu","year":"2020","journal-title":"Comput. Methods Appl. Mech. Eng."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"114693","DOI":"10.1016\/j.eswa.2021.114693","article-title":"Driver stress detection via multimodal fusion using attention-based CNN-LSTM","volume":"173","author":"Mou","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_50","first-page":"e11","article-title":"An Improved Approach for Stress Detection Using Physiological Signals","volume":"21","author":"Nigam","year":"2021","journal-title":"EAI Trans. Scalable Inf. Syst."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"22","DOI":"10.25046\/aj050603","article-title":"Supervised learning techniques for stress detection in car drivers","volume":"5","author":"Zontone","year":"2020","journal-title":"Adv. Sci. Technol. Eng. Syst. J."},{"key":"ref_52","unstructured":"Zephyr Technology (2022, March 29). BioHarness 3.0 User Manual. Annapolis, USA. Available online: https:\/\/www.zephyranywhere.com\/media\/download\/bioharness3-user-manual.pdf."},{"key":"ref_53","unstructured":"MINDFIELD\u00ae BIOSYSTEMS (2022, March 29). MINDFIELD\u00ae eSense Skin Response\u2014User Manual Version 5.6.1|08.02.2022. Gronau, Germany. Available online: https:\/\/www.mindfield.de\/pdf\/phocadownload\/eSense\/English\/eSense_Skin_Response_Manual_EN.pdf."},{"key":"ref_54","unstructured":"GIMA, S.p.A (2022, March 29). Gima Abpm Pulse Rate Monitor with Bluetooth\u2014User Manual. Gessate, Italy. Available online: https:\/\/www.gimaitaly.com\/DocumentiGIMA\/Manuali\/EN\/M35112EN.pdf."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1080\/02699930903274322","article-title":"Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers","volume":"24","author":"Schaefer","year":"2010","journal-title":"Cogn. Emot."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"e044193","DOI":"10.1136\/bmjopen-2020-044193","article-title":"Virtual reality-based relaxation for enhancement of perioperative well-being and quality of life: Protocol for a randomised pilot trial","volume":"11","author":"Schrempf","year":"2021","journal-title":"BMJ Open"},{"key":"ref_57","unstructured":"Tripp(R), Inc. (2022, March 29). TRIPP for Oculus Quest 2. Available online: https:\/\/www.oculus.com\/experiences\/quest\/2173576192720129\/."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Diraco, G., Leone, A., and Siciliano, P. (2022). Clinical Decision Support System for Multisensory Stimulation Therapy in Dementia: A Preliminary Study. Lecture Notes in Electrical Engineering, Springer.","DOI":"10.1007\/978-3-031-08838-4_22"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Wang, X., Wirth, A., and Wang, L. (2007, January 28\u201331). Structure-based statistical features and multivariate time series clustering. Proceedings of the IEEE International Conference on Data Mining, ICDM, Omaha, NE, USA.","DOI":"10.1109\/ICDM.2007.103"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"105754","DOI":"10.1016\/j.knosys.2020.105754","article-title":"One-class support vector classifiers: A survey","volume":"196","author":"Alam","year":"2020","journal-title":"Knowl.-Based Syst."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Diraco, G., Siciliano, P., and Leone, A. (2021). Remaining Useful Life Prediction from 3D Scan Data with Genetically Optimized Convolutional Neural Networks. Sensors, 21.","DOI":"10.20944\/preprints202108.0272.v1"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Lara-Ben\u00edtez, P., Carranza-Garc\u00eda, M., Luna-Romera, J.M., and Riquelme, J.C. (2020). Temporal convolutional networks applied to energy-related time series forecasting. Appl. Sci., 10.","DOI":"10.20944\/preprints202003.0096.v1"},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_64","unstructured":"Beale, M., Hagan, M., and Demuth, H. (2022, March 04). Deep Learning Toolbox\u2122 Reference. MATLAB (r) R2021a. The MathWorks, Inc. Available online: https:\/\/it.mathworks.com\/help\/pdf_doc\/deeplearning\/nnet_ref.pdf."},{"key":"ref_65","unstructured":"(2022, March 04). Optimization Toolbox\u2122 User\u2019s Guide. MATLAB (r) R2021a. The MathWorks, Inc. Available online: https:\/\/it.mathworks.com\/help\/pdf_doc\/optim\/optim.pdf."},{"key":"ref_66","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Pouromran, F., Radhakrishnan, S., and Kamarthi, S. (2021). Exploration of physiological sensors, features, and machine learning models for pain intensity estimation. PLoS ONE, 16.","DOI":"10.1371\/journal.pone.0254108"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/9\/3468\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:05:31Z","timestamp":1760137531000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/9\/3468"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,2]]},"references-count":67,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["s22093468"],"URL":"https:\/\/doi.org\/10.3390\/s22093468","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202203.0403.v1","asserted-by":"object"}]},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,2]]}}}