{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T12:44:07Z","timestamp":1774529047040,"version":"3.50.1"},"reference-count":53,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2019,10,14]],"date-time":"2019-10-14T00:00:00Z","timestamp":1571011200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002428","name":"Austrian Science Fund","doi-asserted-by":"publisher","award":["FWF I-3022"],"award-info":[{"award-number":["FWF I-3022"]}],"id":[{"id":"10.13039\/501100002428","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004955","name":"\u00d6sterreichische Forschungsf\u00f6rderungsgesellschaft","doi-asserted-by":"publisher","award":["FFG 865208"],"award-info":[{"award-number":["FFG 865208"]}],"id":[{"id":"10.13039\/501100004955","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Wearable sensors are increasingly used in research, as well as for personal and private purposes. A variety of scientific studies are based on physiological measurements from such rather low-cost wearables. That said, how accurate are such measurements compared to measurements from well-calibrated, high-quality laboratory equipment used in psychological and medical research? The answer to this question, undoubtedly impacts the reliability of a study\u2019s results. In this paper, we demonstrate an approach to quantify the accuracy of low-cost wearables in comparison to high-quality laboratory sensors. We therefore developed a benchmark framework for physiological sensors that covers the entire workflow from sensor data acquisition to the computation and interpretation of diverse correlation and similarity metrics. We evaluated this framework based on a study with 18 participants. Each participant was equipped with one high-quality laboratory sensor and two wearables. These three sensors simultaneously measured the physiological parameters such as heart rate and galvanic skin response, while the participant was cycling on an ergometer following a predefined routine. The results of our benchmarking show that cardiovascular parameters (heart rate, inter-beat interval, heart rate variability) yield very high correlations and similarities. Measurement of galvanic skin response, which is a more delicate undertaking, resulted in lower, but still reasonable correlations and similarities. We conclude that the benchmarked wearables provide physiological measurements such as heart rate and inter-beat interval with an accuracy close to that of the professional high-end sensor, but the accuracy varies more for other parameters, such as galvanic skin response.<\/jats:p>","DOI":"10.3390\/s19204448","type":"journal-article","created":{"date-parts":[[2019,10,14]],"date-time":"2019-10-14T12:14:05Z","timestamp":1571055245000},"page":"4448","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Wearables and the Quantified Self: Systematic Benchmarking of Physiological Sensors"],"prefix":"10.3390","volume":"19","author":[{"given":"G\u00fcnther","family":"Sagl","sequence":"first","affiliation":[{"name":"Department of Geoinformatics\u2014Z_GIS, University of Salzburg, 5020 Salzburg, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2233-6926","authenticated-orcid":false,"given":"Bernd","family":"Resch","sequence":"additional","affiliation":[{"name":"Department of Geoinformatics\u2014Z_GIS, University of Salzburg, 5020 Salzburg, Austria"},{"name":"Center for Geographic Analysis, Harvard University, Cambridge,  MA 02138, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5029-2425","authenticated-orcid":false,"given":"Andreas","family":"Petutschnig","sequence":"additional","affiliation":[{"name":"Department of Geoinformatics\u2014Z_GIS, University of Salzburg, 5020 Salzburg, Austria"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9387-852X","authenticated-orcid":false,"given":"Kalliopi","family":"Kyriakou","sequence":"additional","affiliation":[{"name":"Department of Geoinformatics\u2014Z_GIS, University of Salzburg, 5020 Salzburg, Austria"}]},{"given":"Michael","family":"Liedlgruber","sequence":"additional","affiliation":[{"name":"Department of Psychology, University of Salzburg, 5020 Salzburg, Austria"}]},{"given":"Frank H.","family":"Wilhelm","sequence":"additional","affiliation":[{"name":"Department of Psychology, University of Salzburg, 5020 Salzburg, Austria"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"217","DOI":"10.3390\/jsan1030217","article-title":"Sensor mania! The internet of things, wearable computing, objective metrics, and the quantified self 2.0","volume":"1","author":"Swan","year":"2012","journal-title":"J. Sens. Actuator Netw."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1177\/1367549415584857","article-title":"Our metrics, ourselves: A hundred years of self-tracking from the weight scale to the wrist wearable device","volume":"18","author":"Crawford","year":"2015","journal-title":"Eur. J. Cult. Stud."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Piwek, L., Ellis, D.A., Andrews, S., and Joinson, A. (2016). The rise of consumer health wearables: Promises and barriers. PLoS Med., 13.","DOI":"10.1371\/journal.pmed.1001953"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Werner, C., Resch, B., and Loidl, M. (2019). Evaluating urban bicycle infrastructures through intersubjectivity of stress sensations derived from physiological measurements. ISPRS Int. J. Geo-Inf., 8.","DOI":"10.3390\/ijgi8060265"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Basu, S., Jana, N., Bag, A., Mahadevappa, M., Mukherjee, J., Kumar, S., and Guha, R. (2015, January 21\u201324). Emotion recognition based on physiological signals using valence-arousal model. Proceedings of the 2015 Third International Conference on Image Information Processing (ICIIP), Waknaghat, India.","DOI":"10.1109\/ICIIP.2015.7414739"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Gartner, G., and Huang, H. (2015). Urban emotions\u2014Geo-semantic emotion extraction from technical sensors, human sensors and crowdsourced data. Progress in Location-Based Services 2014, Springer International Publishing.","DOI":"10.1007\/978-3-319-11879-6"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Taj-Eldin, M., Ryan, C., O'Flynn, B., and Galvin, P. (2018). A review of wearable solutions for physiological and emotional monitoring for use by people with autism spectrum disorder and their caregivers. Sensors, 18.","DOI":"10.3390\/s18124271"},{"key":"ref_8","unstructured":"Healey, J. (2014). Physiological sensing of emotion. The Oxford Handbook of Affective Computing, Oxford University Press."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"743","DOI":"10.3389\/fphys.2018.00743","article-title":"A critical review of consumer wearables, mobile applications, and equipment for providing biofeedback, monitoring stress, and sleep in physically active populations","volume":"9","author":"Peake","year":"2018","journal-title":"Front. Physiol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1080\/00330124.2018.1547978","article-title":"Wearables and location tracking technologies for mental-state sensing in outdoor environments","volume":"71","author":"Birenboim","year":"2019","journal-title":"Prof. Geogr."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kyriakou, K., Resch, B., Sagl, G., Petutschnig, A., Werner, C., Niederseer, D., Liedlgruber, M., Wilhelm, F.H., Osborne, T., and Pykett, J. (2019). Detecting moments of stress from measurements of wearable physiological sensors. Sensors, 19.","DOI":"10.3390\/s19173805"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1553\/giscience2018_01_s344","article-title":"Combining biosensing technology and virtual environments for improved urban planning","volume":"1","author":"Zeile","year":"2018","journal-title":"GI_Forum"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"5","DOI":"10.2478\/udi-2019-0008","article-title":"Defining and assessing walkability: A concept for an integrated approach using surveys, biosensors and geospatial analysis","volume":"62","author":"Resch","year":"2019","journal-title":"Urban Dev. Issues"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Guo, R., Li, S., He, L., Gao, W., Qi, H., and Owens, G. (2013, January 5\u20138). Pervasive and unobtrusive emotion sensing for human mental health. Proceedings of the 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops, Venice, Italy.","DOI":"10.4108\/icst.pervasivehealth.2013.252133"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Majumder, S., Mondal, T., and Deen, M. (2017). Wearable sensors for remote health monitoring. Sensors, 17.","DOI":"10.3390\/s17010130"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"16043","DOI":"10.1038\/micronano.2016.43","article-title":"Emerging flexible and wearable physical sensing platforms for healthcare and biomedical applications","volume":"2","author":"Kenry","year":"2016","journal-title":"Microsyst. Nanoeng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1016\/j.ijhcs.2012.10.016","article-title":"Subject-dependent biosignal features for increased accuracy in psychological stress detection","volume":"71","author":"Giakoumis","year":"2013","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Gradl, S., Wirth, M., Richer, R., Rohleder, N., and Eskofier, B.M. (2019, January 20\u201323). An overview of the feasibility of permanent, real-time, unobtrusive stress measurement with current wearables. Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare, ACM, Trento, Italy.","DOI":"10.1145\/3329189.3329233"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.is.2015.04.007","article-title":"Time-series clustering\u2014A decade review","volume":"53","author":"Aghabozorgi","year":"2015","journal-title":"Inf. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1016\/j.knosys.2014.04.035","article-title":"An empirical evaluation of similarity measures for time series classification","volume":"67","author":"Arcos","year":"2014","journal-title":"Knowl. Based Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1016\/j.jsv.2016.02.037","article-title":"An alternative approach to measure similarity between two deterministic transient signals","volume":"371","author":"Shin","year":"2016","journal-title":"J. Sound Vib."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1145\/2782759.2782767","article-title":"Trajectory similarity measures","volume":"7","author":"Toohey","year":"2015","journal-title":"Sigspatial Spec."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1007\/s10618-012-0250-5","article-title":"Experimental comparison of representation methods and distance measures for time series data","volume":"26","author":"Wang","year":"2013","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, L., \u00d6zsu, M.T., and Oria, V. (2005, January 14\u201316). Robust and fast similarity search for moving object trajectories. Proceedings of the 2005 ACM SIGMOD International Conference on Management of Data, Baltimore, MD, USA.","DOI":"10.1145\/1066157.1066213"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1007\/s10115-004-0154-9","article-title":"Exact indexing of dynamic time warping","volume":"7","author":"Keogh","year":"2005","journal-title":"Knowl. Inf. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1016\/j.patcog.2018.04.003","article-title":"Time alignment measurement for time series","volume":"81","author":"Folgado","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.patrec.2018.11.013","article-title":"A novel distance measure for time series: Maximum shifting correlation distance","volume":"117","author":"Jiang","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1007\/s10618-015-0418-x","article-title":"Using dynamic time warping distances as features for improved time series classification","volume":"30","author":"Kate","year":"2016","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/BF03018603","article-title":"Sur quelques points du calcul fonctionnel","volume":"22","year":"1906","journal-title":"Rendiconti del Circolo Matematico di Palermo (1884\u20131940)"},{"key":"ref_30","unstructured":"Shahbaz, K. (2013). Applied similarity problems using fr\u00e9chet distance. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1016\/j.comgeo.2014.01.002","article-title":"Similarity of polygonal curves in the presence of outliers","volume":"47","author":"Gheibi","year":"2014","journal-title":"Comput. Geom."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Aronov, B., Har-Peled, S., Knauer, C., Wang, Y., and Wenk, C. (2006). Fr\u00e9chet distance for curves, revisited. European Symposium on Algorithms, Springer.","DOI":"10.1007\/11841036_8"},{"key":"ref_33","unstructured":"Wylie, T.R. (2013). The Discrete Fr\u00e9chet Distance with Applications. [Ph.D. Thesis, Montana State University-Bozeman, College of Engineering]."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"11005","DOI":"10.1016\/j.ifacol.2017.08.2479","article-title":"Comparison of different methods of measuring similarity in physiologic time series","volume":"50","author":"Kianimajd","year":"2017","journal-title":"IFAC-PapersOnLine"},{"key":"ref_35","first-page":"87","article-title":"Comparison of values of pearson\u2019s and spearman\u2019s correlation coefficients on the same sets of data","volume":"30","author":"Hauke","year":"2011","journal-title":"Quaest. Geogr."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1518","DOI":"10.1126\/science.1205438","article-title":"Detecting novel associations in large data sets","volume":"334","author":"Reshef","year":"2011","journal-title":"Science"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1502","DOI":"10.1126\/science.1215894","article-title":"A correlation for the 21st century","volume":"334","author":"Speed","year":"2011","journal-title":"Science"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"4492","DOI":"10.1038\/s41598-018-22573-6","article-title":"Analysis of generic coupling between EEG activity and PETCO2 in free breathing and breath-hold tasks using maximal information coefficient (MIC)","volume":"8","author":"Morelli","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Brillinger, D.R. (2001). Time Series: Data Analysis and Theory, Siam.","DOI":"10.1137\/1.9780898719246"},{"key":"ref_40","first-page":"427","article-title":"Distribution of the estimators for autoregressive time series with a unit root","volume":"74","author":"Dickey","year":"1979","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_41","first-page":"409","article-title":"Bioharness (\u2122) multivariable monitoring device: Part. II: Reliability","volume":"11","author":"Johnstone","year":"2012","journal-title":"J. Sports Sci. Med."},{"key":"ref_42","first-page":"400","article-title":"Bioharness (\u2122) multivariable monitoring device: Part. I: Validity","volume":"11","author":"Johnstone","year":"2012","journal-title":"J. Sports Sci. Med."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1528","DOI":"10.3758\/s13428-015-0665-1","article-title":"Anslab: Integrated multichannel peripheral biosignal processing in psychophysiological science","volume":"48","author":"Blechert","year":"2016","journal-title":"Behav. Res. Methods"},{"key":"ref_44","first-page":"307","article-title":"Integrating geographic information into survey research: Current applications, challenges and future avenues","volume":"11","author":"Bluemke","year":"2017","journal-title":"Surv. Res. Methods"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1089\/10665270360688057","article-title":"Continuous representations of time-series gene expression data","volume":"10","author":"Gerber","year":"2003","journal-title":"J. Comput. Biol."},{"key":"ref_46","first-page":"346","article-title":"Assessment of heart rate variability during alterations in stress: Complex demodulation vs. Spectral analysis","volume":"41","author":"Wilhelm","year":"2005","journal-title":"Biomed. Sci. Instrum."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1016\/S1050-6411(99)00012-7","article-title":"Coefficient of cross correlation and the time domain correspondence","volume":"9","author":"Li","year":"1999","journal-title":"J. Electromyogr. Kinesiol."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1214\/17-AOAS1093","article-title":"An empirical study of the maximal and total information coefficients and leading measures of dependence","volume":"12","author":"Reshef","year":"2018","journal-title":"Ann. Appl. Stat."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Albers, S., Alt, H., and N\u00e4her, S. (2009). The computational geometry of comparing shapes. Efficient Algorithms: Essays Dedicated to Kurt Mehlhorn on the Occasion of His 60th Birthday, Springer.","DOI":"10.1007\/978-3-642-03456-5"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Zhu, Q., Batista, G., Rakthanmanon, T., and Keogh, E. (2012, January 26\u201328). A novel approximation to dynamic time warping allows anytime clustering of massive time series datasets. Proceedings of the 2012 SIAM International Conference on Data Mining, Davis, FL, USA.","DOI":"10.1137\/1.9781611972825.86"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.artmed.2008.11.007","article-title":"Matching incomplete time series with dynamic time warping: An algorithm and an application to post-stroke rehabilitation","volume":"45","author":"Tormene","year":"2009","journal-title":"Artif. Intell. Med."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1111\/j.1469-8986.1981.tb03024.x","article-title":"Publication recommendations for electrodermal measurements","volume":"18","author":"Fowles","year":"1981","journal-title":"Psychophysiology"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1109\/TPAMI.2008.76","article-title":"Time warp edit distance with stiffness adjustment for time series matching","volume":"31","author":"Marteau","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/20\/4448\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:26:06Z","timestamp":1760189166000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/20\/4448"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,14]]},"references-count":53,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2019,10]]}},"alternative-id":["s19204448"],"URL":"https:\/\/doi.org\/10.3390\/s19204448","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,14]]}}}