{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T23:18:51Z","timestamp":1776381531042,"version":"3.51.2"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2021,12,29]],"date-time":"2021-12-29T00:00:00Z","timestamp":1640736000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,12,29]],"date-time":"2021-12-29T00:00:00Z","timestamp":1640736000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100006196","name":"Oulun Yliopisto","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100006196","id-type":"DOI","asserted-by":"publisher"}]},{"name":"University of Oulu including Oulu University Hospital"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Multimed Tools Appl"],"published-print":{"date-parts":[[2022,2]]},"abstract":"<jats:title>Abstract\n<\/jats:title><jats:p>Pain is a complex phenomenon, the experience of which varies widely across individuals. At worst, chronic pain can lead to anxiety and depression. Cost-effective strategies are urgently needed to improve the treatment of pain, and thus we propose a novel home-based pain measurement system for the longitudinal monitoring of pain experience and variation in different patients with chronic low back pain. The autonomous nervous system and audio-visual features are analyzed from heart rate signals, voice characteristics and facial expressions using a unique measurement protocol. Self-reporting is utilized for the follow-up of changes in pain intensity, induced by well-designed physical maneuvers, and for studying the consecutive trends in pain. We describe the study protocol, including hospital measurements and questionnaires and the implementation of the home measurement devices. We also present different methods for analyzing the multimodal data: electroencephalography, audio, video and heart rate. Our intention is to provide new insights using technical methodologies that will be beneficial in the future not only for patients with low back pain but also patients suffering from any chronic pain.<\/jats:p>","DOI":"10.1007\/s11042-021-11761-8","type":"journal-article","created":{"date-parts":[[2021,12,29]],"date-time":"2021-12-29T11:03:17Z","timestamp":1640775797000},"page":"5717-5742","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Pain fingerprinting using multimodal sensing: pilot study"],"prefix":"10.1007","volume":"81","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1304-1413","authenticated-orcid":false,"given":"Anja","family":"Keskinarkaus","sequence":"first","affiliation":[]},{"given":"Ruijing","family":"Yang","sequence":"additional","affiliation":[]},{"given":"Angelos","family":"Fylakis","sequence":"additional","affiliation":[]},{"given":"Md.","family":"Surat-E-Mostafa","sequence":"additional","affiliation":[]},{"given":"Arto","family":"Hautala","sequence":"additional","affiliation":[]},{"given":"Yong","family":"Hu","sequence":"additional","affiliation":[]},{"given":"Jinye","family":"Peng","sequence":"additional","affiliation":[]},{"given":"Guoying","family":"Zhao","sequence":"additional","affiliation":[]},{"given":"Tapio","family":"Sepp\u00e4nen","sequence":"additional","affiliation":[]},{"given":"Jaro","family":"Karppinen","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,12,29]]},"reference":[{"key":"11761_CR1","doi-asserted-by":"publisher","first-page":"463","DOI":"10.1016\/j.ejpain.2004.11.001","volume":"9","author":"AV Apkarian","year":"2005","unstructured":"Apkarian AV, Bushnell MC, Treede RD, Zubieta JK (2005) Human brain mechanisms of pain perception and regulation in health and disease. Eur J Pain 9:463\u2013484","journal-title":"Eur J Pain"},{"key":"11761_CR2","doi-asserted-by":"publisher","first-page":"435","DOI":"10.1109\/TAFFC.2015.2462830","volume":"7","author":"MS Aung","year":"2015","unstructured":"Aung MS, Kaltwang S, Romera-Paredes B, Martinez B, Singh A, Cella M, Valstar M, Meng H, Kemp A, Shafizadeh M (2015) The automatic detection of chronic pain-related expression: requirements, challenges and the multimodal EmoPain dataset. IEEE Trans Affect Comput 7:435\u2013451","journal-title":"IEEE Trans Affect Comput"},{"key":"11761_CR3","doi-asserted-by":"publisher","first-page":"1153","DOI":"10.1111\/j.1553-2712.2001.tb01132.x","volume":"8","author":"PE Bijur","year":"2001","unstructured":"Bijur PE, Silver W, Gallagher EJ (2001) Reliability of the visual analog scale for measurement of acute pain. Acad Emerg Med 8:1153\u20131157","journal-title":"Acad Emerg Med"},{"key":"11761_CR4","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1016\/S0885-3924(01)00409-2","volume":"23","author":"A Caraceni","year":"2002","unstructured":"Caraceni A, Cherny N, Fainsinger R, Kaasa S, Poulain P, Radbruch L, De Conno F (2002) Pain measurement tools and methods in clinical research in palliative care: recommendations of an Expert Working Group of the European Association of Palliative Care. J Pain Symptom Manage 23:239\u2013255","journal-title":"J Pain Symptom Manage"},{"key":"11761_CR5","first-page":"27","volume":"2","author":"CC Chang","year":"2011","unstructured":"Chang CC, Lin CJ (2011) LIBSVM: A library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2:27","journal-title":"ACM Trans Intell Syst Technol (TIST)"},{"key":"11761_CR6","first-page":"1","volume":"6","author":"Y Chang","year":"2015","unstructured":"Chang Y, Lee Y, Liang K, Chen I (2015) Tsai C and Hsieh S (2015) Experiencing affective music in eyes-closed and eyes-open states: an electroencephalography study. Front Psychol 6:1\u20139","journal-title":"Front Psychol"},{"key":"11761_CR7","doi-asserted-by":"publisher","first-page":"2981","DOI":"10.3182\/20140824-6-ZA-1003.01420","volume":"47","author":"Y Chu","year":"2014","unstructured":"Chu Y, Zhao X, Yao J, Zhao Y, Wu Z (2014) Physiological signals based quantitative evaluation method of the pain. IFAC Proceedings Volumes 47:2981\u20132986","journal-title":"IFAC Proceedings Volumes"},{"key":"11761_CR8","doi-asserted-by":"publisher","first-page":"2940","DOI":"10.1097\/00007632-200011150-00017","volume":"25","author":"J Fairbank","year":"2000","unstructured":"Fairbank J, Pynsent P (2000) The Oswestry disability index. Spine 25:2940\u20132953","journal-title":"Spine"},{"key":"11761_CR9","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1016\/j.pop.2012.06.013","volume":"39","author":"EL Garland","year":"2012","unstructured":"Garland EL (2012) Pain processing in the human nervous system: a selective review of nociceptive and biobehavioral pathways. Prim Care 39:561\u2013571","journal-title":"Prim Care"},{"key":"11761_CR10","unstructured":"Gawande A (2010) Checklist manifesto, the (HB). Penguin Books IndiaA. Gawande, Checklist manifesto, the (HB)., Penguin Books India., 2010"},{"key":"11761_CR11","unstructured":"Gruss S, Geiger M, Werner P, Wilhelm O, Traue HC, Al-Hamadi A, Walter S (2016) Multi-modal signals for analyzing pain responses to thermal and electrical stimuli. J Vis ExpS 146:e59057"},{"key":"11761_CR12","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1007\/s00586-005-1044-x","volume":"15","author":"M Haefeli","year":"2006","unstructured":"Haefeli M, Elfering A (2006) Pain assessment. Eur Spine J 15:17\u201324","journal-title":"Eur Spine J"},{"key":"11761_CR13","doi-asserted-by":"crossref","unstructured":"Haque MA, Bautista RB, Noroozi F, Kulkarni K, Laursen CB, Irani R, Bellantonio M, Escalera S, Anbarjafari G, Nasrollahi K (2018) Deep multimodal pain recognition: a database and comparison of spatio-temporal visual modalities. 13th IEEE Int Conf Autom Face Gesture Recognit: 250\u2013257","DOI":"10.1109\/FG.2018.00044"},{"key":"11761_CR14","doi-asserted-by":"publisher","first-page":"1258","DOI":"10.1038\/nm.2231","volume":"16","author":"R Kuner","year":"2010","unstructured":"Kuner R (2010) Central mechanisms of pathological pain. Nat Med 16:1258\u20131266","journal-title":"Nat Med"},{"key":"11761_CR15","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1097\/j.pain.0000000000001424","volume":"160","author":"M Kunz","year":"2019","unstructured":"Kunz M, Meixner D, Lautenbacher S (2019) Facial muscle movements encoding pain-a systematic review. Pain 160:535\u2013549","journal-title":"Pain"},{"key":"11761_CR16","doi-asserted-by":"publisher","first-page":"550","DOI":"10.1097\/j.pain.0000000000001417","volume":"160","author":"J Lee","year":"2019","unstructured":"Lee J, Mawla I, Kim J, Loggia ML, Ortiz A, Jung C, Chan ST, Gerber J, Schmithorst VJ, Edwards RR et al (2019) Machine learning-based prediction of clinical pain using multimodal neuroimaging and autonomic metrics. PAIN 160:550\u2013560","journal-title":"PAIN"},{"key":"11761_CR17","doi-asserted-by":"publisher","unstructured":"Li J, Weng Y, Ng C, Lee C (2018) Learning conditional acoustic latent representation with gender and age attributes for automatic pain level recognition. Proc Interspeech 3438\u20133442. https:\/\/doi.org\/10.21437\/Interspeech.2018-1298","DOI":"10.21437\/Interspeech.2018-1298"},{"key":"11761_CR18","doi-asserted-by":"crossref","unstructured":"Lucey P, Cohn J, Prkachin K, Solomon P, Iain M (2011) Painful data: the UNBC-McMaster shoulder pain expression archive database. IEEE International Conference on Automatic Face and Gesture Recognition and Workshops 57\u201364","DOI":"10.1109\/FG.2011.5771462"},{"key":"11761_CR19","doi-asserted-by":"publisher","first-page":"5984","DOI":"10.3390\/app10175984","volume":"10","author":"R M Al-Eidan","year":"2020","unstructured":"M Al-Eidan R, Al-Khalifa H, Al-Salman A (2020) Deep-Learning-Based Models for Pain Recognition: A Systematic Review. Appl Sci 10:5984","journal-title":"Appl Sci"},{"key":"11761_CR20","doi-asserted-by":"crossref","unstructured":"Mozziconacci SJ, Hermes DJ (1997) A study of intonation patterns in speech expressing emotion or attitude: production and perception. IPO Annual Progress Report 32:154\u2013160S. J","DOI":"10.1037\/e494262004-001"},{"key":"11761_CR21","doi-asserted-by":"crossref","unstructured":"Oshrat Y, Bloch A, Lerner A, Cohen A, Avigal M, Zeilig G (2016) Speech prosody as a biosignal for physical pain detection. Conferece Proceedings of 8th Speech Prosody 420\u2013424","DOI":"10.21437\/SpeechProsody.2016-86"},{"key":"11761_CR22","volume-title":"Pain and disability: clinical, behavioral, and public policy perspectives","author":"M Osterweis","year":"1987","unstructured":"Osterweis M, Kleinman A, Mechanic D (1987) Pain and disability: clinical, behavioral, and public policy perspectives. National Academy Press, Washington, DC"},{"key":"11761_CR23","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1016\/j.neulet.2009.05.068","volume":"460","author":"B Schmidt","year":"2009","unstructured":"Schmidt B, Hanslmayr S (2009) Resting frontal EEG alpha-asymmetry predicts the evaluation of affective musical stimuli. Neurosci Lett 460:237\u2013240","journal-title":"Neurosci Lett"},{"key":"11761_CR24","doi-asserted-by":"publisher","first-page":"2074","DOI":"10.3390\/s18072074","volume":"18","author":"L Shu","year":"2018","unstructured":"Shu L, Xie J, Yang M (2018) A review of emotion recognition using physiological signals. Sensors (Basel) 18:2074","journal-title":"Sensors (Basel)"},{"issue":"14","key":"11761_CR25","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1186\/1471-2474-14-149","volume":"26","author":"J S\u00f6dervall","year":"2013","unstructured":"S\u00f6dervall J, Karppinen J, Puolitaival J, Kyll\u00f6nen E, Siltala J, Kiviniemi AM, Tulppo MP (2013) Hautala AJ (2013) Heart rate variability in sciatica patients referred to spine surgery. BMC Musculoskelet Disord 26(14):149. https:\/\/doi.org\/10.1186\/1471-2474-14-149","journal-title":"BMC Musculoskelet Disord"},{"key":"11761_CR26","unstructured":"Tarvainen MP, Niskanen JP (2008) User's Guide of Kubios HRV version 2. Biosignal Analysis and Medical Imaging Group, BSAMIG"},{"issue":"3","key":"11761_CR27","doi-asserted-by":"publisher","first-page":"743","DOI":"10.1109\/TAFFC.2019.2892090","volume":"12","author":"P Thiam","year":"2021","unstructured":"Thiam P, Kessler V, Amirian M, Bellmann P, Layher G, Zhang Y, Velana M, Gruss S, Walter S, Traue HC (2021) Multi-modal pain intensity recognition based on the senseemotion database. IEEE Trans Affect Comput 12(3):743\u2013760","journal-title":"IEEE Trans Affect Comput"},{"key":"11761_CR28","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1177\/00238309040470040301","volume":"47","author":"J Toivanen","year":"2004","unstructured":"Toivanen J, V\u00e4yrynen E, Sepp\u00e4nen T (2004) Automatic Discrimination of Emotion from Spoken Finnish. Lang Speech 47:383\u2013412","journal-title":"Lang Speech"},{"key":"11761_CR29","doi-asserted-by":"crossref","unstructured":"Tsai F, Weng Y, Ng C, Lee C (2017) Embedding stacked bottleneck vocal features in a LSTM architecture for automatic pain level classification during emergency triage. International Conference on Affective Computing and Intelligent Interaction: 313\u2013318","DOI":"10.1109\/ACII.2017.8273618"},{"key":"11761_CR30","doi-asserted-by":"crossref","unstructured":"Velana M, Gruss S, Layher G, Thiam P, Zhang Y, Schork D, Kessler V, Meudt S, Neumann H, Kim J (2016) The senseemotion database: A multimodal database for the development and systematic validation of an automatic pain-and emotion-recognition system. IAPR Workshop on Multimodal Pattern Recognition of Social Signals in Human-Computer Interaction:127\u2013139","DOI":"10.1007\/978-3-319-59259-6_11"},{"key":"11761_CR31","doi-asserted-by":"crossref","unstructured":"Walter S, Gruss S, Ehleiter H, Tan J, Traue HC, Werner P, Al-Hamadi A, Crawcour S, Andrade AO, Silva GMd (2013) The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system. IEEE International Conference on Cybernetics: 128\u2013131","DOI":"10.1109\/CYBConf.2013.6617456"},{"key":"11761_CR32","doi-asserted-by":"publisher","unstructured":"Werner P, Lopez-Martinez D, Walter S, Al-Hamadi A, Gruss S, Picard R (2019) Automatic recognition methods supporting pain assessment: a survey. IEEE Trans Affect Comput P. https:\/\/doi.org\/10.1109\/TAFFC.2019.2946774","DOI":"10.1109\/TAFFC.2019.2946774"},{"key":"11761_CR33","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1109\/TAFFC.2016.2537327","volume":"8","author":"P Werner","year":"2016","unstructured":"Werner P, Al-Hamadi A, Limbrecht-Ecklundt K, Walter S, Gruss S, Traue HC (2016) Automatic pain assessment with facial activity descriptors. IEEE Trans Affect Comput 8:286\u2013299","journal-title":"IEEE Trans Affect Comput"},{"key":"11761_CR34","doi-asserted-by":"crossref","unstructured":"Zhang Z, Girard JM, Wu Y, Zhang X, Liu P, Ciftci U, Canavan S, Reale M, Horowitz A, Yang H (2016) Multimodal spontaneous emotion corpus for human behavior analysis. Conference on Computer Vision and Pattern Recognition (CVPR): 3438\u20133446","DOI":"10.1109\/CVPR.2016.374"},{"key":"11761_CR35","doi-asserted-by":"publisher","first-page":"915","DOI":"10.1109\/TPAMI.2007.1110","volume":"6","author":"G Zhao","year":"2007","unstructured":"Zhao G, Pietik\u00e4inen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans Pattern Anal Mach Intell 6:915\u2013928","journal-title":"IEEE Trans Pattern Anal Mach Intell"}],"container-title":["Multimedia Tools and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-021-11761-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11042-021-11761-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-021-11761-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,2,22]],"date-time":"2022-02-22T06:24:38Z","timestamp":1645511078000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11042-021-11761-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,29]]},"references-count":35,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,2]]}},"alternative-id":["11761"],"URL":"https:\/\/doi.org\/10.1007\/s11042-021-11761-8","relation":{},"ISSN":["1380-7501","1573-7721"],"issn-type":[{"value":"1380-7501","type":"print"},{"value":"1573-7721","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,29]]},"assertion":[{"value":"18 May 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 August 2021","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 November 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 December 2021","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The Ethical Committee of the Northern Ostrobothnia. Hospital District, Oulu, Finland, approved the PASE study protocol.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"All voluntary patients and healthy participants gave their informed consent.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could appear to influence the work reported in this paper.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of interest\/Competing interests"}}]}}