{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T04:29:59Z","timestamp":1772252999392,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2021,8,9]],"date-time":"2021-08-09T00:00:00Z","timestamp":1628467200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Wearable cardiac sensors pave the way for advanced cardiac monitoring applications based on heart rate variability (HRV). In real-life settings, heart rate (HR) measurements are subject to motion artifacts that may lead to frequent data loss (missing samples in the HR signal), especially for commercial devices based on photoplethysmography (PPG). The current study had two main goals: (i) to provide a white-box quality index that estimates the amount of missing samples in any piece of HR signal; and (ii) to quantify the impact of data loss on feature extraction in a PPG-based HR signal. This was done by comparing real-life recordings from commercial sensors featuring both PPG (Empatica E4) and ECG (Zephyr BioHarness 3). After an outlier rejection process, our quality index was used to isolate portions of ECG-based HR signals that could be used as benchmark, to validate the output of Empatica E4 at the signal level and at the feature level. Our results showed high accuracy in estimating the mean HR (median error: 3.2%), poor accuracy for short-term HRV features (e.g., median error: 64% for high-frequency power), and mild accuracy for longer-term HRV features (e.g., median error: 25% for low-frequency power). These levels of errors could be reduced by using our quality index to identify time windows with few or no data loss (median errors: 0.0%, 27%, and 6.4% respectively, when no sample was missing). This quality index should be useful in future work to extract reliable cardiac features in real-life measurements, or to conduct a field validation study on wearable cardiac sensors.<\/jats:p>","DOI":"10.3390\/s21165357","type":"journal-article","created":{"date-parts":[[2021,8,9]],"date-time":"2021-08-09T05:17:06Z","timestamp":1628486226000},"page":"5357","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Real-Time Quality Index to Control Data Loss in Real-Life Cardiac Monitoring Applications"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9778-8206","authenticated-orcid":false,"given":"Ga\u00ebl","family":"Vila","sequence":"first","affiliation":[{"name":"Univ. Grenoble Alpes, CEA, Leti, F-38000 Grenoble, France"},{"name":"Gipsa-Lab, Univ. Grenoble Alpes & CNRS, F-38402 Grenoble, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7873-8724","authenticated-orcid":false,"given":"Christelle","family":"Godin","sequence":"additional","affiliation":[{"name":"Univ. Grenoble Alpes, CEA, Leti, F-38000 Grenoble, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sylvie","family":"Charbonnier","sequence":"additional","affiliation":[{"name":"Gipsa-Lab, Univ. Grenoble Alpes & CNRS, F-38402 Grenoble, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aur\u00e9lie","family":"Campagne","sequence":"additional","affiliation":[{"name":"LPNC UMR 5105, Univ. Grenoble Alpes & CNRS, F-38040 Grenoble, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1089\/big.2012.0002","article-title":"The Quantified Self: Fundamental Disruption in Big Data Science and Biological Discovery","volume":"1","author":"Swan","year":"2013","journal-title":"Big Data"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Picard, R.W. (1997). Affective Computing, MIT-Press.","DOI":"10.7551\/mitpress\/1140.001.0001"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/MCE.2016.2590178","article-title":"A Survey of Affective Computing for Stress Detection: Evaluating Technologies in Stress Detection for Better Health","volume":"5","author":"Greene","year":"2016","journal-title":"IEEE Consum. Electron. Mag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1109\/TITB.2005.854505","article-title":"Flexible Technologies and Smart Clothing for Citizen Medicine, Home Healthcare, and Disease Prevention","volume":"9","author":"Schmitt","year":"2005","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"10837","DOI":"10.3390\/s101210837","article-title":"Detecting Vital Signs with Wearable Wireless Sensors","volume":"10","author":"Yilmaz","year":"2010","journal-title":"Sensors"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1031","DOI":"10.1007\/s11517-006-0119-0","article-title":"Heart Rate Variability: A Review","volume":"44","author":"Acharya","year":"2006","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"235","DOI":"10.30773\/pi.2017.08.17","article-title":"Stress and Heart Rate Variability: A Meta-Analysis and Review of the Literature","volume":"15","author":"Kim","year":"2018","journal-title":"Psychiatry Investig."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Fairclough, S.H., and Mulder, L.J.M. (2011). Psychophysiological Processes of Mental Effort Investment. Motiv. Affect. Cardiovasc. Response Mech. Appl., 61\u201376.","DOI":"10.1037\/13090-003"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Vandecasteele, K., De Cooman, T., Gu, Y., Cleeren, E., Claes, K., Paesschen, W.V., Huffel, S.V., and Hunyadi, B. (2017). Automated Epileptic Seizure Detection Based on Wearable ECG and PPG in a Hospital Environment. Sensors, 17.","DOI":"10.3390\/s17102338"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1375","DOI":"10.1088\/0967-3334\/28\/11\/005","article-title":"Novel Dry Electrodes for ECG Monitoring","volume":"28","author":"Gruetzmann","year":"2007","journal-title":"Physiol. Meas."},{"key":"ref_11","unstructured":"D\u00f6ssel, O., and Schlegel, W.C. (2009, January 7\u201312). Ambulant ECG Recording with Wet and Dry Electrodes: A Direct Comparison of Two Systems. Proceedings of the World Congress on Medical Physics and Biomedical Engineering, Munich, Germany."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/S0735-6757(99)90019-0","article-title":"Pulse Oximetry: Principles and Limitations","volume":"17","author":"Sinex","year":"1999","journal-title":"Am. J. Emerg. Med."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1088\/0967-3334\/31\/9\/015","article-title":"Photoplethysmography Pulse Rate Variability as a Surrogate Measurement of Heart Rate Variability during Non-Stationary Conditions","volume":"31","author":"Gil","year":"2010","journal-title":"Physiol. Meas."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.ijcard.2012.03.119","article-title":"How Accurate Is Pulse Rate Variability as an Estimate of Heart Rate Variability? A Review on Studies Comparing Photoplethysmographic Technology with an Electrocardiogram","volume":"166","author":"Vagedes","year":"2013","journal-title":"Int. J. Cardiol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1291","DOI":"10.1109\/JBHI.2015.2458779","article-title":"BioWatch: A Noninvasive Wrist-Based Blood Pressure Monitor That Incorporates Training Techniques for Posture and Subject Variability","volume":"20","author":"Thomas","year":"2016","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1007\/s41745-017-0037-1","article-title":"Review on Heart-Rate Estimation from Photoplethysmography and Accelerometer Signals During Physical Exercise","volume":"97","author":"Periyasamy","year":"2017","journal-title":"J. Indian Inst. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sweeney, K.T., Kearney, D., Ward, T.E., Coyle, S., and Diamond, D. (2013, January 3\u20137). Employing Ensemble Empirical Mode Decomposition for Artifact Removal: Extracting Accurate Respiration Rates from ECG Data during Ambulatory Activity. Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan.","DOI":"10.1109\/EMBC.2013.6609666"},{"key":"ref_18","unstructured":"Del R\u00edo, B.A.S., Lopetegi, T., and Romero, I. (2011, January 18\u201321). Assessment of Different Methods to Estimate Electrocardiogram Signal Quality. Proceedings of the 2011 Computing in Cardiology, Hangzhou, China."},{"key":"ref_19","unstructured":"Behar, J., Oster, J., Li, Q., and Clifford, G.D. (2012, January 9\u201312). A Single Channel ECG Quality Metric. Proceedings of the 2012 Computing in Cardiology, Krakow, Poland."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1491","DOI":"10.1088\/0967-3334\/33\/9\/1491","article-title":"Dynamic Time Warping and Machine Learning for Signal Quality Assessment of Pulsatile Signals","volume":"33","author":"Li","year":"2012","journal-title":"Physiol. Meas."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Papini, G.B., Fonseca, P., Aubert, X.L., Overeem, S., Bergmans, J.W.M., and Vullings, R. (2017, January 11\u201315). Photoplethysmography Beat Detection and Pulse Morphology Quality Assessment for Signal Reliability Estimation. Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju, Korea.","DOI":"10.1109\/EMBC.2017.8036776"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1216","DOI":"10.1109\/JBHI.2016.2615316","article-title":"Quality Assessment of Ambulatory ECG Using Wavelet Entropy of the HRV Signal","volume":"21","author":"Orphanidou","year":"2017","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_23","first-page":"832","article-title":"Signal-Quality Indices for the Electrocardiogram and Photoplethysmogram: Derivation and Applications to Wireless Monitoring","volume":"19","author":"Orphanidou","year":"2015","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_24","unstructured":"(2019, April 01). How is IBI.Csv Obtained?. Available online: http:\/\/support.empatica.com\/hc\/en-us\/articles\/201912319-How-is-IBI-csv-obtained-."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ollander, S., Godin, C., Campagne, A., and Charbonnier, S. (2016, January 9\u201312). A Comparison of Wearable and Stationary Sensors for Stress Detection. Proceedings of the 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, Hungary.","DOI":"10.1109\/SMC.2016.7844917"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1007\/s10916-020-01648-w","article-title":"Validity of the Empatica E4 Wristband to Measure Heart Rate Variability (HRV) Parameters: A Comparison to Electrocardiography (ECG)","volume":"44","author":"Schuurmans","year":"2020","journal-title":"J. Med. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"148","DOI":"10.3389\/fnbeh.2020.00148","article-title":"Validating Measures of Electrodermal Activity and Heart Rate Variability Derived From the Empatica E4 Utilized in Research Settings That Involve Interactive Dyadic States","volume":"14","author":"Milstein","year":"2020","journal-title":"Front. Behav. Neurosci."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Rossi, A., Pedreschi, D., Clifton, D.A., and Morelli, D. (2020). Error Estimation of Ultra-Short Heart Rate Variability Parameters: Effect of Missing Data Caused by Motion Artifacts. Sensors, 20.","DOI":"10.3390\/s20247122"},{"key":"ref_29","first-page":"643","article-title":"Field Based Reliability and Validity of the BioharnessTM Multivariable Monitoring Device","volume":"11","author":"Johnstone","year":"2012","journal-title":"J. Sports Sci. Med."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1776","DOI":"10.1126\/science.1103572","article-title":"A Survey Method for Characterizing Daily Life Experience: The Day Reconstruction Method","volume":"306","author":"Kahneman","year":"2004","journal-title":"Science"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1109\/TBME.1985.325532","article-title":"A Real-Time QRS Detection Algorithm","volume":"BME-32","author":"Pan","year":"1985","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_32","unstructured":"Sedghamiz, H. (2014). Matlab Implementation of Pan Tompkins ECG QRS Detector, The MathWorks, Inc."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1203\/PDR.0b013e318123fbcc","article-title":"Heart Rate Variability: Impact of Differences in Outlier Identification and Management Strategies on Common Measures in Three Clinical Populations","volume":"62","author":"Kemper","year":"2007","journal-title":"Pediatr. Res."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1186\/1475-925X-11-2","article-title":"Automatic Filtering of Outliers in RR Intervals before Analysis of Heart Rate Variability in Holter Recordings: A Comparison with Carefully Edited Data","volume":"11","author":"Karlsson","year":"2012","journal-title":"Biomed. Eng. Online"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1007\/BF00648343","article-title":"Least-Squares Frequency Analysis of Unequally Spaced Data","volume":"39","author":"Lomb","year":"1976","journal-title":"Astrophys. Space Sci."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"16","DOI":"10.3847\/1538-4365\/aab766","article-title":"Understanding the Lomb\u2013Scargle Periodogram","volume":"236","author":"VanderPlas","year":"2018","journal-title":"Astrophys. J. Suppl. Ser."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1109\/10.554760","article-title":"A Triaxial Accelerometer and Portable Data Processing Unit for the Assessment of Daily Physical Activity","volume":"44","author":"Bouten","year":"1997","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1109\/TITB.2005.856864","article-title":"Implementation of a Real-Time Human Movement Classifier Using a Triaxial Accelerometer for Ambulatory Monitoring","volume":"10","author":"Karantonis","year":"2006","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"258","DOI":"10.3389\/fpubh.2017.00258","article-title":"An Overview of Heart Rate Variability Metrics and Norms","volume":"5","author":"Shaffer","year":"2017","journal-title":"Front. Public Health"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1007\/s003990070035","article-title":"Nonlinear Analysis of Complex Phenomena in Cardiological Data","volume":"11","author":"Wessel","year":"2000","journal-title":"Herzschrittmachertherapie Elektrophysiol."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Morelli, D., Rossi, A., Cairo, M., and Clifton, D.A. (2019). Analysis of the Impact of Interpolation Methods of Missing RR-Intervals Caused by Motion Artifacts on HRV Features Estimations. Sensors, 19.","DOI":"10.3390\/s19143163"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/16\/5357\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:42:42Z","timestamp":1760164962000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/16\/5357"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,9]]},"references-count":41,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["s21165357"],"URL":"https:\/\/doi.org\/10.3390\/s21165357","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202107.0036.v1","asserted-by":"object"}]},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,9]]}}}