{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T06:11:28Z","timestamp":1760249488759,"version":"build-2065373602"},"reference-count":48,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2020,12,19]],"date-time":"2020-12-19T00:00:00Z","timestamp":1608336000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100011950","name":"ITEA","doi-asserted-by":"publisher","award":["INNO4HEALTH 19008"],"award-info":[{"award-number":["INNO4HEALTH 19008"]}],"id":[{"id":"10.13039\/100011950","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Reliable and diverse labeled reference data are essential for the development of high-quality processing algorithms for medical signals, such as electrocardiogram (ECG) and photoplethysmogram (PPG). Here, we present the Platform for Analysis and Labeling of Medical time Series (PALMS) designed in Python. Its graphical user interface (GUI) facilitates three main types of manual annotations\u2014(1) fiducials, e.g., R-peaks of ECG; (2) events with an adjustable duration, e.g., arrhythmic episodes; and (3) signal quality, e.g., data parts corrupted by motion artifacts. All annotations can be attributed to the same signal simultaneously in an ergonomic and user-friendly manner. Configuration for different data and annotation types is straightforward and flexible in order to use a wide range of data sources and to address many different use cases. Above all, configuration of PALMS allows plugging-in existing algorithms to display outcomes of automated processing, such as automatic R-peak detection, and to manually correct them where needed. This enables fast annotation and can be used to further improve algorithms. The GUI is currently complemented by ECG and PPG algorithms that detect characteristic points with high accuracy. The ECG algorithm reached 99% on the MIT\/BIH arrhythmia database. The PPG algorithm was validated on two public databases with an F1-score above 98%. The GUI and optional algorithms result in an advanced software tool that allows the creation of diverse reference sets for existing datasets.<\/jats:p>","DOI":"10.3390\/s20247302","type":"journal-article","created":{"date-parts":[[2020,12,21]],"date-time":"2020-12-21T01:01:08Z","timestamp":1608512468000},"page":"7302","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Platform for Analysis and Labeling of Medical Time Series"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3047-9329","authenticated-orcid":false,"given":"Andrejs","family":"Fedjajevs","sequence":"first","affiliation":[{"name":"Stichting Imec the Netherlands, 5656 AE Eindhoven, The Netherlands"}]},{"given":"Willemijn","family":"Groenendaal","sequence":"additional","affiliation":[{"name":"Stichting Imec the Netherlands, 5656 AE Eindhoven, The Netherlands"}]},{"given":"Carlos","family":"Agell","sequence":"additional","affiliation":[{"name":"Stichting Imec the Netherlands, 5656 AE Eindhoven, The Netherlands"}]},{"given":"Evelien","family":"Hermeling","sequence":"additional","affiliation":[{"name":"Stichting Imec the Netherlands, 5656 AE Eindhoven, The Netherlands"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-018-0074-9","article-title":"Large-scale wearable data reveal digital phenotypes for daily-life stress detection","volume":"1","author":"Smets","year":"2018","journal-title":"Npj Digit. Med."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1146\/annurev-anchem-061417-125956","article-title":"Wearable and Implantable Sensors for Biomedical Applications","volume":"11","author":"Koydemir","year":"2018","journal-title":"Annu. Rev. Anal. Chem."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1675","DOI":"10.1016\/j.measurement.2012.02.006","article-title":"Smart sensors\/actuators for biomedical applications: Review","volume":"45","author":"Ponmozhi","year":"2012","journal-title":"Measurement"},{"doi-asserted-by":"crossref","unstructured":"Schaekermann, M., Beaton, G., Sanoubari, E., Lim, A., Larson, K., and Law, E. (2020, January 25\u201330). Ambiguity-aware AI Assistants for Medical Data Analysis. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems\u2013CHI\u201920, Honolulu, HI, USA.","key":"ref_4","DOI":"10.1145\/3313831.3376506"},{"doi-asserted-by":"crossref","unstructured":"Cairns, A., Bond, R., Breen, C., Finlay, D., Geldenring, D., and Peace, A. (2017). Variability of human-annotations of 12-lead ECG features collected using a web system: Students vs. practitioners. J. Electrocardiol.","key":"ref_5","DOI":"10.1016\/j.jelectrocard.2017.08.041"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1109\/MCSE.2016.91","article-title":"WaveformECG: A platform for visualizing, annotating, and analyzing ECG data","volume":"18","author":"Winslow","year":"2016","journal-title":"Comput. Sci. Eng."},{"key":"ref_7","first-page":"775","article-title":"ECGlab: User friendly ECG\/VCG analysis tool for research environments","volume":"40","author":"Vicente","year":"2013","journal-title":"Comput. Cardiol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"557","DOI":"10.3389\/fphys.2016.00557","article-title":"HRV analysis: A free software for analyzing cardiac autonomic activity","volume":"7","author":"Pichot","year":"2016","journal-title":"Front. Physiol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"472","DOI":"10.21105\/joss.00472","article-title":"PFEIFER: Preprocessing Framework for Electrograms Intermittently Fiducialized from Experimental Recordings","volume":"3","author":"Rodenhauser","year":"2018","journal-title":"J. Open Source Softw."},{"doi-asserted-by":"crossref","unstructured":"Vollmer, M. (2019, January 8\u201311). HRVTool\u2014An Open-Source Matlab Toolbox for Analyzing Heart Rate Variability. Proceedings of the 2019 Computing in Cardiology Conference (CinC), Singapore.","key":"ref_10","DOI":"10.22489\/CinC.2019.032"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1186\/s12938-017-0401-4","article-title":"SinusCor: An advanced tool for heart rate variability analysis","volume":"16","author":"Bartels","year":"2017","journal-title":"Biomed. Eng. Online"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.bspc.2014.01.011","article-title":"Matlab-based tool for ECG and HRV analysis","volume":"10","author":"Mali","year":"2014","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_13","first-page":"311","article-title":"Heart Rate Monitoring and PQRST Detection Based on Graphical User Interface with Matlab","volume":"5","year":"2015","journal-title":"Int. J. Inf. Electron. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e226","DOI":"10.7717\/peerj-cs.226","article-title":"R-DECO: An open-source Matlab based graphical user interface for the detection and correction of R-peaks","volume":"5","author":"Moeyersons","year":"2019","journal-title":"PeerJ Comput. Sci."},{"unstructured":"Karlen, W., Turner, M., Cooke, E., Dumont, G.A., and Ansermino, J.M. (2010, January 13\u201316). CapnoBase: Signal database and tools to collect, share and annotate respiratory signals. Proceedings of the Annual Meeting of the Society for Technology in Anesthesia (STA), West Palm Beach, FL, USA.","key":"ref_15"},{"key":"ref_16","first-page":"e241","article-title":"AHA\/ACCF\/HRS Recommendations for the Standardization and Interpretation of the Electrocardiogram: Part IV: The ST segment, T and U waves, and the QT interval: A scientific statement from the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology; the American College of Cardiology Foundation; and the Heart Rhythm Society: Endorsed by the International Society for Computerized Electrocardiology","volume":"119","author":"Rautaharju","year":"2009","journal-title":"J. Am. Coll. Cardiol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"258","DOI":"10.3389\/fpubh.2017.00258","article-title":"An Overview of Heart Rate Variability Metrics and Norms","volume":"5","author":"Shaffer","year":"2017","journal-title":"Front. Public Health"},{"doi-asserted-by":"crossref","unstructured":"Zong, W., Moody, G.B., and Jiang, D. (2003, January 21\u201324). A robust open-source algorithm to detect onset and duration of QRS complexes. Proceedings of the Computers in Cardiology, Thessaloniki Chalkidiki, Greece.","key":"ref_18","DOI":"10.1109\/CIC.2003.1291261"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1377","DOI":"10.1109\/TBME.2016.2549060","article-title":"QRS Detection Algorithm for Telehealth Electrocardiogram Recordings","volume":"63","author":"Khamis","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"doi-asserted-by":"crossref","unstructured":"Elgendi, M. (2013). Fast QRS Detection with an Optimized Knowledge-Based Method: Evaluation on 11 Standard ECG Databases. PLoS ONE, 8.","key":"ref_20","DOI":"10.1371\/journal.pone.0073557"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1016\/j.jare.2012.05.007","article-title":"QRS detection using K-Nearest Neighbor algorithm (KNN) and evaluation on standard ECG databases","volume":"4","author":"Saini","year":"2013","journal-title":"J. Adv. Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1109\/51.993193","article-title":"The principles of software QRS detection","volume":"21","author":"Hennig","year":"2002","journal-title":"IEEE Eng. Med. Biol. Mag."},{"doi-asserted-by":"crossref","unstructured":"Saquib, N., Papon, M.T.I., Ahmad, I., and Rahman, A. (2015, January 5\u20137). Measurement of heart rate using photoplethysmography. Proceedings of the 2015 International Conference on Networking Systems and Security (NSysS), Dhaka, Bangladesh.","key":"ref_23","DOI":"10.1109\/NSysS.2015.7043525"},{"doi-asserted-by":"crossref","unstructured":"Bolanos, M., Nazeran, H., and Haltiwanger, E. (September, January 30). Comparison of heart rate variability signal features derived from electrocardiography and photoplethysmography in healthy individuals. Proceedings of the 2006 International Conference of the IEEE Engineering in Medicine and Biology Society, New York, NY, USA.","key":"ref_24","DOI":"10.1109\/IEMBS.2006.260607"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.ijcard.2012.03.119","article-title":"How accurate is pulse rate variability as an estimate of heart rate variability? A review on studies comparing photoplethysmographic technology with an electrocardiogram","volume":"166","author":"Vagedes","year":"2013","journal-title":"Int. J. Cardiol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1093\/oxfordjournals.eurheartj.a014868","article-title":"Heart rate variability. Standards of measurement, physiological interpretation, and clinical use","volume":"17","author":"Malik","year":"1996","journal-title":"Eur. Heart J."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"574","DOI":"10.3390\/s130100574","article-title":"Design of an oximeter based on LED-LED configuration and FPGA technology","volume":"13","author":"Stojanovic","year":"2013","journal-title":"Sensors"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2027","DOI":"10.1088\/0967-3334\/35\/10\/2027","article-title":"Photoplethysmographic signal waveform index for detection of increased arterial stiffness","volume":"35","author":"Pilt","year":"2014","journal-title":"Physiol. Meas."},{"doi-asserted-by":"crossref","unstructured":"Wei, C., Sheng, L., Lihua, G., Yuquan, C., and Min, P. (2011, January 15\u201317). Study on conditioning and feature extraction algorithm of photoplethysmography signal for physiological parameters detection. Proceedings of the 4th International Congress on Image and Signal Processing (CISP), Shanghai, China.","key":"ref_29","DOI":"10.1109\/CISP.2011.6100581"},{"doi-asserted-by":"crossref","unstructured":"Peterek, T., Prauzek, M., and Penhaker, M. (2010, January 5\u20137). A new method for identification of the significant point in the plethysmografical record. Proceedings of the 2nd International Conference on Signal Processing Systems, Dalian, China.","key":"ref_30","DOI":"10.1109\/ICSPS.2010.5555642"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1016\/j.compbiomed.2009.10.006","article-title":"Adaptive threshold method for the peak detection of photoplethysmographic waveform","volume":"39","author":"Shin","year":"2009","journal-title":"Comput. Biol. Med."},{"doi-asserted-by":"crossref","unstructured":"Luo, F., Li, J., Yun, F., Chen, T., and Chen, X. (2014, January 14\u201316). An improved algorithm for the detection of photoplethysmographic percussion peaks. Proceedings of the 7th International Congress on Image and Signal Processing (CISP), Dalian, China.","key":"ref_32","DOI":"10.1109\/CISP.2014.7003906"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"e215","DOI":"10.1161\/01.CIR.101.23.e215","article-title":"PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/51.932724","article-title":"The impact of the MIT-BIH arrhythmia database","volume":"20","author":"Moody","year":"2001","journal-title":"IEEE Eng. Med. Biol. Mag."},{"unstructured":"(2012). Medical Electrical Equipment Part 2\u201347: Particular Requirements for the Basic Safety and Essential Performance of Ambulatory Electrocardiographic Systems, IEC 60601-2-47:2012.","key":"ref_35"},{"key":"ref_36","first-page":"133","article-title":"Onset and Peak Detection over Pulse Wave Using Supervised SOM Network","volume":"3","year":"2013","journal-title":"Int. J. Biosci. Biochem. Bioinform."},{"unstructured":"(2020, November 14). TimeView, GitHub. Available online: https:\/\/github.com\/TimeViewers\/timeview.","key":"ref_37"},{"doi-asserted-by":"crossref","unstructured":"Koranne, S., and Koranne, S. (2011). Hierarchical Data Format 5: HDF5. Handbook of Open Source Tools, Springer.","key":"ref_38","DOI":"10.1007\/978-1-4419-7719-9_10"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1142\/S0219691305000774","article-title":"Continuous wavelet transform modulus maxima analysis of the electrocardiogram: Beat characterisation and beat-to-beat measurement","volume":"3","author":"Legarreta","year":"2005","journal-title":"Int. J. Wavelets Multiresolut. Inf. Process."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1016\/j.bspc.2009.06.002","article-title":"On an automatic delineator for arterial blood pressure waveforms","volume":"5","author":"Li","year":"2010","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1164","DOI":"10.1093\/oxfordjournals.eurheartj.a060332","article-title":"The European ST-T database: Standard for evaluating systems for the analysis of ST-T changes in ambulatory electrocardiography","volume":"13","author":"Taddei","year":"1992","journal-title":"Eur. Heart J."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"e17326","DOI":"10.2196\/17326","article-title":"Using Biosensors and Digital Biomarkers to Assess Response to Cardiac Rehabilitation: Observational Study","volume":"22","author":"Smeets","year":"2020","journal-title":"J. Med. Internet Res."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2589","DOI":"10.1109\/JBHI.2020.2967872","article-title":"Portable Detection of Apnea and Hypopnea Events Using Bio-Impedance of the Chest and Deep Learning","volume":"24","author":"Groenendaal","year":"2020","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1197\/jamia.M1733","article-title":"Agreement, the F-measure, and reliability in information retrieval","volume":"12","author":"Hripcsak","year":"2005","journal-title":"J. Am. Med Inform. Assoc."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1914","DOI":"10.1109\/TBME.2016.2613124","article-title":"Toward a robust estimation of respiratory rate from pulse oximeters","volume":"64","author":"Pimentel","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1109\/TBME.1985.325532","article-title":"A Real-Time QRS Detection Algorithm","volume":"3","author":"Pan","year":"1985","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/10.362922","article-title":"Detection of ECG Characteristic Points Using Wavelet Transforms","volume":"42","author":"Li","year":"1995","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1617","DOI":"10.1166\/jmihi.2017.2175","article-title":"A robust peak detection algorithm for photoplethysmographic waveforms in mobile devices","volume":"7","author":"Tran","year":"2017","journal-title":"J. Med. Imaging Health Inform."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/24\/7302\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:47:17Z","timestamp":1760179637000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/24\/7302"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,19]]},"references-count":48,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["s20247302"],"URL":"https:\/\/doi.org\/10.3390\/s20247302","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,12,19]]}}}