{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T15:24:28Z","timestamp":1783437868327,"version":"3.54.6"},"reference-count":63,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"German Academic Exchange Service (DAAD)","award":["03ZZ04X02B"],"award-info":[{"award-number":["03ZZ04X02B"]}]},{"name":"German Academic Exchange Service (DAAD)","award":["638\/14-1"],"award-info":[{"award-number":["638\/14-1"]}]},{"name":"the Federal Ministry of Education and Research of Germany","award":["03ZZ04X02B"],"award-info":[{"award-number":["03ZZ04X02B"]}]},{"name":"the Federal Ministry of Education and Research of Germany","award":["638\/14-1"],"award-info":[{"award-number":["638\/14-1"]}]},{"name":"DFG","award":["03ZZ04X02B"],"award-info":[{"award-number":["03ZZ04X02B"]}]},{"name":"DFG","award":["638\/14-1"],"award-info":[{"award-number":["638\/14-1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Pain is a reliable indicator of health issues; it affects patients\u2019 quality of life when not well managed. The current methods in the clinical application undergo biases and errors; moreover, such methods do not facilitate continuous pain monitoring. For this purpose, the recent methodologies in automatic pain assessment were introduced, which demonstrated the possibility for objectively and robustly measuring and monitoring pain when using behavioral cues and physiological signals. This paper focuses on introducing a reliable automatic system for continuous monitoring of pain intensity by analyzing behavioral cues, such as facial expressions and audio, and physiological signals, such as electrocardiogram (ECG), electromyogram (EMG), and electrodermal activity (EDA) from the X-ITE Pain Dataset. Several experiments were conducted with 11 datasets regarding classification and regression; these datasets were obtained from the database to reduce the impact of the imbalanced database problem. With each single modality (Uni-modality) experiment, we used a Random Forest [RF] baseline method, a Long Short-Term Memory (LSTM) method, and a LSTM using a sample weighting method (called LSTM-SW). Further, LSTM and LSTM-SW were used with fused modalities (two modalities = Bi-modality and all modalities = Multi-modality) experiments. Sample weighting was used to downweight misclassified samples during training to improve the performance. The experiments\u2019 results confirmed that regression is better than classification with imbalanced datasets, EDA is the best single modality, and fused modalities improved the performance significantly over the single modality in 10 out of 11 datasets.<\/jats:p>","DOI":"10.3390\/s22134992","type":"journal-article","created":{"date-parts":[[2022,7,4]],"date-time":"2022-07-04T20:59:18Z","timestamp":1656968358000},"page":"4992","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["An Automatic System for Continuous Pain Intensity Monitoring Based on Analyzing Data from Uni-, Bi-, and Multi-Modality"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4738-1897","authenticated-orcid":false,"given":"Ehsan","family":"Othman","sequence":"first","affiliation":[{"name":"Department of Neuro-Information Technology, Institute for Information Technology and Communications, Otto-von-Guericke University Magdeburg, 39106 Magdeburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4762-2926","authenticated-orcid":false,"given":"Philipp","family":"Werner","sequence":"additional","affiliation":[{"name":"Department of Neuro-Information Technology, Institute for Information Technology and Communications, Otto-von-Guericke University Magdeburg, 39106 Magdeburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Frerk","family":"Saxen","sequence":"additional","affiliation":[{"name":"Department of Neuro-Information Technology, Institute for Information Technology and Communications, Otto-von-Guericke University Magdeburg, 39106 Magdeburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5606-4033","authenticated-orcid":false,"given":"Marc-Andr\u00e9","family":"Fiedler","sequence":"additional","affiliation":[{"name":"Department of Neuro-Information Technology, Institute for Information Technology and Communications, Otto-von-Guericke University Magdeburg, 39106 Magdeburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3632-2402","authenticated-orcid":false,"given":"Ayoub","family":"Al-Hamadi","sequence":"additional","affiliation":[{"name":"Department of Neuro-Information Technology, Institute for Information Technology and Communications, Otto-von-Guericke University Magdeburg, 39106 Magdeburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,1]]},"reference":[{"key":"ref_1","first-page":"439","article-title":"Facial Expression of Pain: An Evolutionary Account","volume":"25","author":"Williams","year":"2002","journal-title":"Behav. Brain Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"350","DOI":"10.1016\/j.pain.2011.10.027","article-title":"Are both the sensory and the affective dimensions of pain encoded in the face?","volume":"153","author":"Kunz","year":"2012","journal-title":"Pain"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Thiam, P., Kestler, H., and Schwenker, F. (2020, January 22\u201324). Multimodal Deep Denoising Convolutional Autoencoders for Pain Intensity Classification based on Physiological Signals. Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods, Valletta, Malta.","DOI":"10.5220\/0008896102890296"},{"key":"ref_4","first-page":"e59057","article-title":"Multi-Modal Signals for Analyzing Pain Responses to Thermal and Electrical Stimuli","volume":"146","author":"Gruss","year":"2019","journal-title":"J. Vis. Exp."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1037\/0003-066X.48.4.384","article-title":"Facial Expression and Emotion","volume":"48","author":"Ekman","year":"1993","journal-title":"Am. Psychol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"119","DOI":"10.2217\/nmt-2015-0006","article-title":"The Use of Facial Expressions for Pain Assessment Purposes in Dementia: A Narrative Review","volume":"6","author":"Oosterman","year":"2016","journal-title":"Neurodegener. Dis. Manag."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Corbett, A., Achterberg, W., Husebo, B., Lobbezoo, F., de Vet, H., Kunz, M., Strand, L., Constantinou, M., Tudose, C., and Kappesser, J. (2014). An International Road Map to Improve Pain Assessment in People with Impaired Cognition: The Development of the Pain Assessment in Impaired Cognition (PAIC) Meta-tool. BMC Neurol., 14.","DOI":"10.1186\/s12883-014-0229-5"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"125","DOI":"10.3109\/15360288.2015.1035830","article-title":"Pain Indicators for Persisting Pain in Hospitalized Infants in A South African Setting: An Explorative Study","volume":"29","author":"Snoek","year":"2015","journal-title":"J. Pain Palliat. Care Pharmacother."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Walter, S., Gruss, S., Ehleiter, H., Tan, J., Traue, H.C., Werner, P., Al-Hamadi, A., Crawcour, S., Andrade, A.O., and da Silva, G.M. (2013, January 13\u201315). The BioVid Heat Pain Database: Data for the Advancement and Systematic Validation of an Automated Pain Recognition System. Proceedings of the IEEE International Conference on Cybernetics (CYBCO), Lausanne, Switzerland.","DOI":"10.1109\/CYBConf.2013.6617456"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Pouromran, F., Radhakrishnan, S., and Kamarthi, S. (2021). Exploration of Physiological Sensors, Features, and Machine Learning Models for Pain Intensity Estimation. PLoS ONE, 16.","DOI":"10.1371\/journal.pone.0254108"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"279","DOI":"10.3389\/fnins.2017.00279","article-title":"Physiological Signal-Based Method for Measurement of Pain Intensity","volume":"11","author":"Chu","year":"2017","journal-title":"Front. Neurosci."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lopez-Martinez, D., and Picard, R. (2018, January 17\u201321). Continuous Pain Intensity Estimation from Autonomic Signals with Recurrent Neural Networks. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA.","DOI":"10.1109\/EMBC.2018.8513575"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1109\/TAFFC.2019.2892090","article-title":"Multi-modal Pain Intensity Recognition based on the SenseEmotion Database","volume":"12","author":"Thiam","year":"2019","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Werner, P., Al-Hamadi, A., Gruss, S., and Walter, S. (2019, January 3\u20136). Twofold-Multimodal Pain Recognition with the X-ITE Pain Database. Proceedings of the 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), Cambridge, UK.","DOI":"10.1109\/ACIIW.2019.8925061"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"363","DOI":"10.3922\/j.psns.2014.041","article-title":"Automatic Pain Quantification using Autonomic Parameters","volume":"7","author":"Walter","year":"2014","journal-title":"Psychol. Neurosci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1007\/s00482-020-00468-8","article-title":"Multimodale Erkennung von Schmerzintensit\u00e4t und-modalit\u00e4t mit maschinellen Lernverfahren","volume":"34","author":"Walter","year":"2020","journal-title":"Schmerz"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1109\/TBME.2021.3096137","article-title":"Automated Pain Assessment in Children using Electrodermal Activity and Video Data Fusion via Machine Learning","volume":"69","author":"Susam","year":"2022","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3335","DOI":"10.1109\/JSEN.2020.3023656","article-title":"Automated Nociceptive Pain Assessment using Physiological Signals and a Hybrid Deep Learning Network","volume":"21","author":"Subramaniam","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1097\/00003465-200311000-00010","article-title":"Assessing Pain Control in Nonverbal Critically ill Adults","volume":"22","author":"Odhner","year":"2003","journal-title":"Dimens. Crit. Care Nurs."},{"key":"ref_20","first-page":"3273","article-title":"Classification Networks for Continuous Automatic Pain Intensity Monitoring in Video using Facial Expression on the X-ITE Pain Database","volume":"21","author":"Othman","year":"2022","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Othman, E., Werner, P., Saxen, F., Al-Hamadi, A., and Walter, S. (2021, January 26\u201329). Regression Networks for Automatic Pain Intensity Recognition in Video using Facial Expression on the X-ITE Pain Database. Proceedings of the 25th International Conference on Image Processing, Computer Vision & Pattern Recognition (IPCV\u201921), Las Vegas, NV, USA.","DOI":"10.3390\/s21093273"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Othman, E., Werner, P., Saxen, F., Al-Hamadi, A., Gruss, S., and Walter, S. (2022). Facial Expression and Electrodermal Activity Analysis for Continuous Pain Intensity Monitoringon the X-ITE Pain Database. IEEE Access, submitted.","DOI":"10.21203\/rs.3.rs-927204\/v1"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/1058-9139(92)90001-S","article-title":"The Facial Expression of Pain Better than a Thousand Words?","volume":"1","author":"Craig","year":"1992","journal-title":"APS J."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1016\/0304-3959(92)90213-U","article-title":"The Consistency of Facial Expressions of Pain: A Comparison Across Modalities","volume":"51","author":"Prkachin","year":"1992","journal-title":"Pain"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1111\/j.1744-6155.2008.00140.x","article-title":"Facial Expression and Pain Assessment in the Pediatric Patient: The Primal Face of Pain","volume":"13","author":"Schiavenato","year":"2008","journal-title":"J. Spec. Pediatric Nurs."},{"key":"ref_26","unstructured":"Tavakolian, M. (2021). Efficient Spatiotemporal Representation Learning for Pain Intensity Estimation from Facial Expressions Doctor of Philosophy. [Ph.D. Thesis, University of Oulu]."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1053\/jpmn.2000.5831","article-title":"The Checklist of Nonverbal Pain Indicators (CNPI)","volume":"1","author":"Feldt","year":"2000","journal-title":"Pain Manag. Nurs."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.jpainsymman.2007.08.015","article-title":"Pain Behavior in Rheumatoid Arthritis Patients: Identification of Pain Behavior Subgroups","volume":"36","author":"Waters","year":"2008","journal-title":"J. Pain Symptom. Manag."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Naranjo-Hern\u00e1ndez, D., Reina-Tosina, J., and Roa, L.M. (2020). Sensor Technologies to Manage the Physiological Traits of Chronic Pain: A Review. Sensors, 20.","DOI":"10.3390\/s20020365"},{"key":"ref_30","first-page":"18","article-title":"Physiologic Responses to Cutaneous Pain in Neonates","volume":"6","author":"Brown","year":"1987","journal-title":"Neonatal Netw."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1097\/00000542-200109000-00007","article-title":"Acute Pain Induces Insulin Resistance in Humans","volume":"95","author":"Greisen","year":"2001","journal-title":"Anesthesiology"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1097\/00006199-199407000-00007","article-title":"Physiological Responses of Premature Infants to a Painful Stimulus","volume":"43","author":"Stevens","year":"1994","journal-title":"Nurs. Res."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"106682","DOI":"10.1016\/j.cmpb.2022.106682","article-title":"Physiological responses to pain in cancer patients: A systematic review","volume":"2017","author":"Moscato","year":"2022","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1111\/jch.12145","article-title":"The Relationship between Blood Pressure and Pain","volume":"15","author":"Meschi","year":"2013","journal-title":"J. Clin. Hypertens."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1797","DOI":"10.1016\/j.imavis.2008.12.010","article-title":"Automatic Coding of Facial Expressions Displayed during Posed and Genuine Pain","volume":"27","author":"Littlewort","year":"2009","journal-title":"Image Vis. Comput."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.imavis.2011.12.003","article-title":"Painful Monitoring: Automatic Pain Monitoring using the UNBC-McMaster Shoulder Pain Expression Archive Database","volume":"30","author":"Lucey","year":"2012","journal-title":"Image Vis. Comput."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/1744-8069-3-25","article-title":"Neuroimaging Revolutionizes Therapeutic Approaches to Chronic Pain","volume":"3","author":"Borsook","year":"2007","journal-title":"Mol. Pain"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1016\/j.pain.2006.10.011","article-title":"The Effect of Experimenter Gender on Autonomic and Subjective Responses to Pain Stimuli","volume":"129","author":"Aslaksen","year":"2006","journal-title":"Pain"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1002\/j.1532-2149.2013.00379.x","article-title":"Heart Rate Variability and Experimentally Induced Pain in Healthy Adults: A systematic Review","volume":"18","author":"Koenig","year":"2014","journal-title":"Eur. J. Pain"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Werner, P., Al-Hamadi, A., Niese, R., Walter, S., Gruss, S., and Traue, H.C. (2014, January 24\u201328). Automatic Pain Recognition from Video and Biomedical Signals. Proceedings of the 22nd International Conference on Pattern Recognition, Stockholm, Sweden.","DOI":"10.1109\/ICPR.2014.784"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1097\/ACO.0b013e3283183fe4","article-title":"Changes in Skin Conductance as a Tool to Monitor Nociceptive Stimulation and Pain","volume":"12","author":"Storm","year":"2008","journal-title":"Curr. Opin. Anaesthesiol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1093\/bja\/ael280","article-title":"Monitoring of Skin Conductance to Assess Postoperative Pain Intensity","volume":"97","author":"Ledowski","year":"2006","journal-title":"Br. J. Anaesth."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"592","DOI":"10.1016\/j.pain.2010.11.032","article-title":"Autonomic Responses to Heat Pain: Heart Rate, Skin Conductance, and their Relation to Verbal Ratings and Stimulus Intensity","volume":"152","author":"Loggia","year":"2011","journal-title":"Pain"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1109\/TAFFC.2016.2537327","article-title":"Automatic Pain Assessment with Facial Activity Descriptors","volume":"8","author":"Werner","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"K\u00e4chele, M., Thiam, P., Amirian, M., Werner, P., Walter, S., Schwenker, F., and Palm, G. (2015, January 25\u201328). Multimodal Data Fusion for Person-Independent, Continuous Estimation of Pain Intensity. Proceedings of the Engineering Applications of Neural Networks: 16th International Conference, Rhodes, Greece.","DOI":"10.1007\/978-3-319-23983-5_26"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Erekat, D., Hammal, Z., Siddiqui, M., and Dibeklio\u011flu, H. (2020, January 25\u201329). Enforcing Multilabel Consistency for Automatic Spatio-Temporal Assessment of Shoulder Pain Intensity. Proceedings of the International Conference on Multimodal Interaction (ICMI\u201920 Companion), Utrecht, The Netherlands.","DOI":"10.1145\/3395035.3425190"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Othman, E., Werner, P., Saxen, F., Al-Hamadi, A., Gruss, S., and Walter, S. (2021). Automatic vs. Human Recognition of Pain Intensity from Facial Expression on the X-ITE Pain Database. Sensors, 21.","DOI":"10.3390\/s21093273"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Othman, E., Werner, P., Saxen, F., Al-Hamadi, A., and Walter, S. (2019, January 23\u201325). Cross-Database Evaluation of Pain Recognition from Facial Video. Proceedings of the 11th International Symposium on Image and Signal Processing and Analysis (ISPA), Dubrovnik, Croatia.","DOI":"10.1109\/ISPA.2019.8868562"},{"key":"ref_50","unstructured":"Thiam, P., Kessler, V., and Schwenker, F. (2017, January 26\u201328). Hierarchical Combination of Video Features for Personalised Pain Level Recognition. Proceedings of the European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Thiam, P., and Schwenker, F. (2018, January 20). Combining Deep and Hand-Crafted Features for Audio-Based Pain Intensity Classification. Proceedings of the IAPR Workshop on Multimodal Pattern Recognition of Social Signals in Human-Computer Interaction, Bejing, China.","DOI":"10.1007\/978-3-030-20984-1_5"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Tsai, F.-S., Hsu, Y.-L., Chen, W.-C., Weng, Y.-M., Ng, C.-J., and Lee, C.-C. (2016, January 8\u201312). Toward Development and Evaluation of Pain Level-Rating Scale for Emergency Triage based on Vocal Characteristics and Facial Expressions. Proceedings of the 17th Annual Conference of the International Speech Communication Association, San Francisco, CA, USA.","DOI":"10.21437\/Interspeech.2016-408"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"106796","DOI":"10.1016\/j.dib.2021.106796","article-title":"Multimodal Neonatal Procedural and Postoperative Pain Assessment Dataset","volume":"35","author":"Salekin","year":"2021","journal-title":"Data Brief"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"720464","DOI":"10.3389\/fphys.2021.720464","article-title":"Multi-Modal Pain Intensity Assessment Based on Physiological Signals: A Deep Learning Perspective","volume":"1","author":"Thiam","year":"2021","journal-title":"Front. Physiol."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Hinduja, S., Canavan, S., and Kaur, G. (2020, January 16\u201320). Multimodal Fusion of Physiological Signals and Facial Action Units for Pain Recognition. Proceedings of the 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), Buenos Aires, Argentina.","DOI":"10.1109\/FG47880.2020.00060"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Baltru\u0161aitis, T., Robinson, P., and Morency, L.-P. (2016, January 7\u201310). OpenFace: An Open Source Facial Behavior Analysis Toolkit. Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Placid, NY, USA.","DOI":"10.1109\/WACV.2016.7477553"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Eyben, F., W\u00f6llmer, M., and Schuller, B. (2010, January 25\u201329). OpenSMILE\u2014The Munich Versatile and Fast Open-Source Audio Feature Extractor. Proceedings of the 18th ACM International Conference on Multimedia (MM \u201810), New York, NY, USA.","DOI":"10.1145\/1873951.1874246"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"1157","DOI":"10.1109\/TBME.1986.325695","article-title":"Quantitative Investigation of QRS Detection Rules Using the MIT\/BIH Arrhythmia Database","volume":"33","author":"Hamilton","year":"1986","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Othman, E., Saxen, F., Bershadskyy, D., Werner, P., Al-Hamadi, A., and Weimann, J. (2019). Predicting Group Contribution Behaviour in a Public Goods Game from Face-to-Face Communication. Sensors, 19.","DOI":"10.3390\/s19122786"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1037\/0033-2909.86.2.420","article-title":"Intraclass Correlations: Uses in Assessing Rater Reliability","volume":"86","author":"Shrout","year":"1979","journal-title":"Psychol. Bull."},{"key":"ref_62","unstructured":"Thiam, P., and Schwenker, F. (December, January 28). Multi-modal Data Fusion for Pain Intensity Assessment and Classification. Proceedings of the 7th International Conference on Image Processing Theory, Tools and Applications (IPTA), Montreal, QC, Canada."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1007\/s12530-016-9158-4","article-title":"Adaptive Confidence Learning for The Personalization of Pain Intensity Estimation Systems","volume":"8","author":"Amirian","year":"2017","journal-title":"Evol. Syst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/13\/4992\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:41:57Z","timestamp":1760139717000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/13\/4992"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,1]]},"references-count":63,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["s22134992"],"URL":"https:\/\/doi.org\/10.3390\/s22134992","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,1]]}}}