{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,25]],"date-time":"2026-01-25T02:09:08Z","timestamp":1769306948743,"version":"3.49.0"},"reference-count":35,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2021,7,3]],"date-time":"2021-07-03T00:00:00Z","timestamp":1625270400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Functional electrical stimulation (FES) is a technique used in rehabilitation, allowing the recreation or facilitation of a movement or function, by electrically inducing the activation of targeted muscles. FES during cycling often uses activation patterns which are based on the crank angle of the pedals. Dynamic changes in their underlying predefined geometrical models (e.g., change in seating position) can lead to desynchronised contractions. Adaptive algorithms with a real-time interpretation of anatomical segments can avoid this and open new possibilities for the automatic design of stimulation patterns. However, their ability to accurately and precisely detect stimulation triggering events has to be evaluated in order to ensure their adaptability to real-case applications in various conditions. In this study, three algorithms (Hilbert, BSgonio, and Gait Cycle Index (GCI) Observer) were evaluated on passive cycling inertial data of six participants with spinal cord injury (SCI). For standardised comparison, a linear phase reference baseline was used to define target events (i.e., 10%, 40%, 60%, and 90% of the cycle\u2019s progress). Limits of agreement (LoA) of \u00b110% of the cycle\u2019s duration and Lin\u2019s concordance correlation coefficient (CCC) were used to evaluate the accuracy and precision of the algorithm\u2019s event detections. The delays in the detection were determined for each algorithm over 780 events. Analysis showed that the Hilbert and BSgonio algorithms validated the selected criteria (LoA: +5.17\/\u22126.34% and +2.25\/\u22122.51%, respectively), while the GCI Observer did not (LoA: +8.59\/\u221227.89%). When evaluating control algorithms, it is paramount to define appropriate criteria in the context of the targeted practical application. To this end, normalising delays in event detection to the cycle\u2019s duration enables the use of a criterion that stays invariable to changes in cadence. Lin\u2019s CCC, comparing both linear correlation and strength of agreement between methods, also provides a reliable way of confirming comparisons between new control methods and an existing reference.<\/jats:p>","DOI":"10.3390\/s21134571","type":"journal-article","created":{"date-parts":[[2021,7,4]],"date-time":"2021-07-04T22:35:22Z","timestamp":1625438122000},"page":"4571","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["A Novel Framework for Quantifying Accuracy and Precision of Event Detection Algorithms in FES-Cycling"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8236-8583","authenticated-orcid":false,"given":"Ronan","family":"Le Guillou","sequence":"first","affiliation":[{"name":"National Institute for Research in Computer Science and Automation (Inria), Camin Team, 34090 Montpellier, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6354-3879","authenticated-orcid":false,"given":"Martin","family":"Schmoll","sequence":"additional","affiliation":[{"name":"Le Laboratoire d\u2019Informatique, de Robotique et de Micro\u00e9lectronique de Montpellier (LIRMM), Universit\u00e9 de Montpellier (UM), 34090 Montpellier, France"}]},{"given":"Beno\u00eet","family":"Sijobert","sequence":"additional","affiliation":[{"name":"National Institute for Research in Computer Science and Automation (Inria), Camin Team, 34090 Montpellier, France"},{"name":"Institut Saint-Pierre (ISP), 34250 Palavas-les-Flots, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6769-5359","authenticated-orcid":false,"given":"David","family":"Lobato Borges","sequence":"additional","affiliation":[{"name":"N\u00facleo de Tecnologia Assistiva, Acessibilidade e Inova\u00e7\u00e3o (NTAAI), Universidade de Bras\u00edlia (UnB), Bras\u00edlia 70910-900, Brazil"}]},{"given":"Emerson","family":"Fachin-Martins","sequence":"additional","affiliation":[{"name":"N\u00facleo de Tecnologia Assistiva, Acessibilidade e Inova\u00e7\u00e3o (NTAAI), Universidade de Bras\u00edlia (UnB), Bras\u00edlia 70910-900, Brazil"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4879-1345","authenticated-orcid":false,"given":"Henrique","family":"Resende","sequence":"additional","affiliation":[{"name":"Departamento de Engenharia El\u00e9trica, Universidade Federal de Minas Gerais (UFMG), Belo Horizonte 31270-901, Brazil"}]},{"given":"Roger","family":"Pissard-Gibollet","sequence":"additional","affiliation":[{"name":"National Institute for Research in Computer Science and Automation (Inria), SED Service, 38330 Montbonnot, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3042-0941","authenticated-orcid":false,"given":"Charles","family":"Fattal","sequence":"additional","affiliation":[{"name":"Rehabilitation Center Bouffard Vercelli, USSAP, 66000 Perpignan, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7379-8004","authenticated-orcid":false,"given":"Christine","family":"Azevedo Coste","sequence":"additional","affiliation":[{"name":"National Institute for Research in Computer Science and Automation (Inria), Camin Team, 34090 Montpellier, France"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1146\/annurev.bioeng.6.040803.140103","article-title":"Functional Electrical Stimulation for Neuromuscular Applications","volume":"7","author":"Peckham","year":"2005","journal-title":"Annu. Rev. Biomed. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1007\/s004210050609","article-title":"A Simple Means of Increasing Muscle Size after Spinal Cord Injury: A Pilot Study","volume":"80","author":"Dudley","year":"1999","journal-title":"Eur. J. Appl. Physiol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1310\/sci2104-294","article-title":"Neuromuscular Electrical Stimulation-Induced Resistance Training after SCI: A Review of the Dudley Protocol","volume":"21","author":"Bickel","year":"2015","journal-title":"Top. Spinal Cord Inj. Rehabil."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/0141-5425(89)90115-5","article-title":"Use of Functional Electrical Stimulation in the Rehabilitation of Patients with Incomplete Spinal Cord Injuries","volume":"11","author":"Bajd","year":"1989","journal-title":"J. Biomed. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1080\/10790268.2001.11753549","article-title":"Modes, Benefits, and Risks of Voluntary and Electrically Induced Exercise in Persons with Spinal Cord Injury","volume":"24","author":"Jacobs","year":"2001","journal-title":"J. Spinal Cord Med."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1573","DOI":"10.1249\/MSS.0b013e318176b2f4","article-title":"Cardiorespiratory and Power Adaptations to Stimulated Cycle Training in Paraplegia","volume":"40","author":"Berry","year":"2008","journal-title":"Med. Sci. Sports Exerc."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.5405\/jmbe.718","article-title":"Review: Clinical Benefits of Functional Electrical Stimulation Cycling Exercise for Subjects with Central Neurological Impairments","volume":"31","author":"Peng","year":"2011","journal-title":"J. Med. Biol. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1016\/0141-5425(83)90003-1","article-title":"Outdoor Bicycle for Exercise in Paraplegics and Quadriplegics","volume":"5","author":"Petrofsky","year":"1983","journal-title":"J. Biomed. Eng."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Watanabe, T., Murakami, T., and Handa, Y. (2013, January 24\u201326). Preliminary Tests of a Prototype FES Control System for Cycling Wheelchair Rehabilitation. Proceedings of the IEEE 13th International Conference on Rehabilitation Robotics, Seattle, WA, USA.","DOI":"10.1109\/ICORR.2013.6650484"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1109\/MRA.2017.2751660","article-title":"Cycling with Spinal Cord Injury: A Novel System for Cycling Using Electrical Stimulation for Individuals with Paraplegia, and Preparation for Cybathlon 2016","volume":"24","author":"Bo","year":"2017","journal-title":"IEEE Rob. Autom Mag."},{"key":"ref_11","unstructured":"Perkins, T., Donaldson, N., Fitzwater, R., Phillips, G., Wood, D., Mayr, W., Bijak, M., and Jancik, C. (2001, January 12\u201315). Leg Powered Paraplegic Cycling System Using Surface Functional Electrical Stimulation. Proceedings of the 7th Vienna International Workshop on Functional Electrical Stimulation, Vienna, Austria."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1111\/aor.13573","article-title":"Fatigue in Complete Spinal Cord Injury and Implications on Total Delay","volume":"44","author":"Rinaldin","year":"2020","journal-title":"Artif. Organs"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1109\/MRA.2017.2749318","article-title":"The Cybathlon RehaBike: Inertial-Sensor-Driven Functional Electrical Stimulation Cycling by Team Hasomed","volume":"24","author":"Wiesener","year":"2017","journal-title":"IEEE Rob. Autom. Mag."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1109\/TNSRE.2004.836778","article-title":"Dynamic Simulation of FES-Cycling: Influence of Individual Parameters","volume":"12","author":"Lugner","year":"2004","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/j.ifacol.2016.12.216","article-title":"Robust Discrimination of Flexion and Extension Phases for Mobile Functional Electrical Stimulation (FES) Induced Cycling in Paraplegics","volume":"49","author":"Wiesener","year":"2016","journal-title":"IFAC-PapersOnLine"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Baptista, R., Sijobert, B., and Coste, C.A. (2018, January 1\u20135). New Approach of Cycling Phases Detection to Improve FES-Pedaling in SCI Individuals. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594162"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sijobert, B., Le Guillou, R., Fattal, C., and Azevedo Coste, C. (2019). FES-Induced Cycling in Complete SCI: A Simpler Control Method Based on Inertial Sensors. Sensors, 19.","DOI":"10.3390\/s19194268"},{"key":"ref_18","first-page":"2","article-title":"Inertial Sensor-Based Control of Functional Electrical Stimulation in Paraplegic Cycling","volume":"26","author":"Ruppin","year":"2016","journal-title":"Eur. J. Transl. Myology"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1016\/S0140-6736(86)90837-8","article-title":"Statistical Methods for Assessing Agreement between Two Methods of Clinical Measurement","volume":"327","author":"Altman","year":"1986","journal-title":"Lancet"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Gottlieb, U., Balasukumaran, T., Hoffman, J.R., and Springer, S. (2020). Agreement of Gait Events Detection during Treadmill Backward Walking by Kinematic Data and Inertial Motion Units. Sensors, 20.","DOI":"10.3390\/s20216331"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1016\/j.gaitpost.2008.09.003","article-title":"The Reliability of Three-Dimensional Kinematic Gait Measurements: A Systematic Review","volume":"29","author":"McGinley","year":"2009","journal-title":"Gait Posture"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"730","DOI":"10.1109\/TCST.2017.2771727","article-title":"Closed-Loop Position and Cadence Tracking Control for FES-Cycling Exploiting Pedal Force Direction with Antagonistic Biarticular Muscles","volume":"27","author":"Kawai","year":"2019","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1016\/j.medengphy.2005.10.006","article-title":"Comparison of Stimulation Patterns for FES-Cycling Using Measures of Oxygen Cost and Stimulation Cost","volume":"28","author":"Hunt","year":"2006","journal-title":"Med. Eng. Phys."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"7087","DOI":"10.4081\/ejtm.2017.7087","article-title":"Cycle Training Using Implanted Neural Prostheses: Team Cleveland","volume":"27","author":"McDaniel","year":"2017","journal-title":"Eur. J. Transl. Myol."},{"key":"ref_25","first-page":"1","article-title":"Continuous Gait Cycle Index Estimation for Electrical Stimulation Assisted Foot Drop Correction","volume":"11","author":"Jovic","year":"2014","journal-title":"J. Neuro Eng. Rehabil."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1109\/TRO.2008.915433","article-title":"Multisensor Input for CPG-Based Sensory - Motor Coordination","volume":"24","author":"Espiau","year":"2008","journal-title":"IEEE Trans. Rob."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Mombaur, K., and Berns, K. (2013). Modeling, Simulation and Optimization of Bipedal Walking, Springer. Cognitive Systems Monographs.","DOI":"10.1007\/978-3-642-36368-9"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1109\/5.135376","article-title":"Estimating and Interpreting the Instantaneous Frequency of a Signal. I. Fundamentals","volume":"80","author":"Boashash","year":"1992","journal-title":"Proc. IEEE"},{"key":"ref_29","unstructured":"Hahn, S.L. (1996). Hilbert Transforms in Signal Processing, Artech House."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"141","DOI":"10.11613\/BM.2015.015","article-title":"Understanding Bland Altman analysis","volume":"25","author":"Giavarina","year":"2015","journal-title":"Biochemia Med."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"255","DOI":"10.2307\/2532051","article-title":"A Concordance Correlation Coefficient to Evaluate Reproducibility","volume":"45","author":"Lin","year":"1989","journal-title":"Biometrics"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2123","DOI":"10.1213\/ANE.0000000000002924","article-title":"Agreement Analysis: What He Said, She Said versus You Said","volume":"126","author":"Vetter","year":"2018","journal-title":"Anesth. Analg."},{"key":"ref_33","first-page":"307","article-title":"A Proposal for Strength-of-Agreement Criteria for Lin\u2019s Concordance Correlation Coefficient","volume":"45","author":"McBride","year":"2005","journal-title":"NIWA Client Rep."},{"key":"ref_34","first-page":"1","article-title":"Standardizing Fatigue-Resistance Testing during Electrical Stimulation of Paralysed Human Quadriceps Muscles, a Practical Approach","volume":"18","author":"Schmoll","year":"2021","journal-title":"J. Neuro Eng. Rehabil."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1109\/TNSRE.2004.828417","article-title":"The Effect of Joint Angle on the Timing of Muscle Contractions Elicited by Neuromuscular Electrical Stimulation","volume":"12","author":"Sinclair","year":"2004","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/13\/4571\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:25:37Z","timestamp":1760163937000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/13\/4571"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,3]]},"references-count":35,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2021,7]]}},"alternative-id":["s21134571"],"URL":"https:\/\/doi.org\/10.3390\/s21134571","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,3]]}}}