{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T02:49:43Z","timestamp":1774493383297,"version":"3.50.1"},"reference-count":82,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,12,7]],"date-time":"2021-12-07T00:00:00Z","timestamp":1638835200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,12,7]],"date-time":"2021-12-07T00:00:00Z","timestamp":1638835200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Although deep neural networks (DNNs) are showing state of the art performance in clinical gait analysis, they are considered to be black-box algorithms. In other words, there is a lack of direct understanding of a DNN\u2019s ability to identify relevant features, hindering clinical acceptance. Interpretability methods have been developed to ameliorate this concern by providing a way to explain DNN predictions.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>This paper proposes the use of an interpretability method to explain DNN decisions for classifying the movement that precedes freezing of gait (FOG), one of the most debilitating symptoms of Parkinson\u2019s disease (PD). The proposed two-stage pipeline consists of (1) a convolutional neural network (CNN) to model the reduction of movement present before a FOG episode, and (2) layer-wise relevance propagation (LRP) to visualize the underlying features that the CNN perceives as important to model the pathology. The CNN was trained with the sagittal plane kinematics from a motion capture dataset of fourteen PD patients with FOG. The robustness of the model predictions and learned features was further assessed on fourteen PD patients without FOG and fourteen age-matched healthy controls.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The CNN proved highly accurate in modelling the movement that precedes FOG, with 86.8% of the strides being correctly identified. However, the CNN model was unable to model the movement for one of the seven patients that froze during the protocol. The LRP interpretability case study shows that (1) the kinematic features perceived as most relevant by the CNN are the reduced peak knee flexion and the fixed ankle dorsiflexion during the swing phase, (2) very little relevance for FOG is observed in the PD patients without FOG and the healthy control subjects, and (3) the poor predictive performance of one subject is attributed to the patient\u2019s unique and severely flexed gait signature.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The proposed pipeline can aid clinicians in explaining DNN decisions in clinical gait analysis and aid machine learning practitioners in assessing the generalization of their models by ensuring that the predictions are based on meaningful kinematic features.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-021-01699-0","type":"journal-article","created":{"date-parts":[[2021,12,7]],"date-time":"2021-12-07T09:02:52Z","timestamp":1638867772000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Modelling and identification of characteristic kinematic features preceding freezing of gait with convolutional neural networks and layer-wise relevance propagation"],"prefix":"10.1186","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2609-6883","authenticated-orcid":false,"given":"Benjamin","family":"Filtjens","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pieter","family":"Ginis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alice","family":"Nieuwboer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Muhammad Raheel","family":"Afzal","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joke","family":"Spildooren","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bart","family":"Vanrumste","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Slaets","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,12,7]]},"reference":[{"key":"1699_CR1","unstructured":"GBD 2016 Parkinson\u2019s Disease Collaborators. Global, regional, and national burden of parkinson\u2019s disease, 1990\u20132016: a systematic analysis for the global burden of disease study 2016. Lancet Neurol. 2018;17(11):939\u201353."},{"issue":"5","key":"1699_CR2","doi-asserted-by":"publisher","first-page":"423","DOI":"10.5114\/ninp.2013.38222","volume":"47","author":"M Rudzi\u0144ska","year":"2013","unstructured":"Rudzi\u0144ska M, Bukowczan S, Sto\u017cek J, Zajdel K, Mirek E, Chwala W, W\u00f3jcik-Pedziwiatr M, Banaszkiewicz K, Szczudlik A. Causes and consequences of falls in Parkinson disease patients in a prospective study. Neurol Neurochir Pol. 2013;47(5):423\u201330.","journal-title":"Neurol Neurochir Pol"},{"issue":"12","key":"1699_CR3","doi-asserted-by":"publisher","first-page":"66","DOI":"10.3390\/ijerph16122216","volume":"16","author":"PHS Pelicioni","year":"2019","unstructured":"Pelicioni PHS, Menant JC, Latt MD, Lord SR. Falls in Parkinson\u2019s disease subtypes: risk factors, locations and circumstances. Int J Environ Res Public Health. 2019;16(12):66.","journal-title":"Int J Environ Res Public Health"},{"issue":"7","key":"1699_CR4","doi-asserted-by":"publisher","first-page":"884","DOI":"10.1001\/jamaneurol.2014.753","volume":"71","author":"S Perez-Lloret","year":"2014","unstructured":"Perez-Lloret S, Negre-Pages L, Damier P, Delval A, Derkinderen P, Dest\u00e9e A, Meissner WG, Schelosky L, Tison F, Rascol O. Prevalence, determinants, and effect on quality of life of freezing of gait in Parkinson disease. JAMA Neurol. 2014;71(7):884\u201390.","journal-title":"JAMA Neurol"},{"issue":"6","key":"1699_CR5","doi-asserted-by":"publisher","first-page":"837","DOI":"10.1002\/mds.21956","volume":"23","author":"MA Hely","year":"2008","unstructured":"Hely MA, Reid WGJ, Adena MA, Halliday GM, Morris JGL. The Sydney multicenter study of Parkinson\u2019s disease: the inevitability of dementia at 20 years. Mov Disord. 2008;23(6):837\u201344.","journal-title":"Mov Disord"},{"issue":"8","key":"1699_CR6","doi-asserted-by":"publisher","first-page":"734","DOI":"10.1016\/S1474-4422(11)70143-0","volume":"10","author":"JG Nutt","year":"2011","unstructured":"Nutt JG, Bloem BR, Giladi N, Hallett M, Horak FB, Nieuwboer A. Freezing of gait: moving forward on a mysterious clinical phenomenon. Lancet Neurol. 2011;10(8):734\u201344.","journal-title":"Lancet Neurol"},{"issue":"Suppl 2","key":"1699_CR7","doi-asserted-by":"publisher","first-page":"468","DOI":"10.1002\/mds.22144","volume":"23","author":"AH Snijders","year":"2008","unstructured":"Snijders AH, Nijkrake MJ, Bakker M, Munneke M, Wind C, Bloem BR. Clinimetrics of freezing of gait. Mov Disord. 2008;23(Suppl 2):468\u201374.","journal-title":"Mov Disord"},{"key":"1699_CR8","first-page":"53","volume":"67","author":"S Fahn","year":"1995","unstructured":"Fahn S. The freezing phenomenon in Parkinsonism. Adv Neurol. 1995;67:53\u201363.","journal-title":"Adv Neurol"},{"issue":"8","key":"1699_CR9","doi-asserted-by":"publisher","first-page":"871","DOI":"10.1002\/mds.20115","volume":"19","author":"BR Bloem","year":"2004","unstructured":"Bloem BR, Hausdorff JM, Visser JE, Giladi N. Falls and freezing of gait in Parkinson\u2019s disease: a review of two interconnected, episodic phenomena. Mov Disord. 2004;19(8):871\u201384.","journal-title":"Mov Disord"},{"issue":"4","key":"1699_CR10","doi-asserted-by":"publisher","first-page":"405","DOI":"10.1097\/01.wco.0000137530.68867.93","volume":"17","author":"YAM Grimbergen","year":"2004","unstructured":"Grimbergen YAM, Munneke M, Bloem BR. Falls in Parkinson\u2019s disease. Curr Opin Neurol. 2004;17(4):405\u201315.","journal-title":"Curr Opin Neurol."},{"issue":"4","key":"1699_CR11","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1097\/01376517-200008000-00006","volume":"32","author":"P Gray","year":"2000","unstructured":"Gray P, Hildebrand K. Fall risk factors in Parkinson\u2019s disease. J Neurosci Nurs. 2000;32(4):222\u20138.","journal-title":"J Neurosci Nurs"},{"issue":"1\u20132","key":"1699_CR12","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1016\/j.jns.2006.05.015","volume":"248","author":"N Giladi","year":"2006","unstructured":"Giladi N, Hausdorff JM. The role of mental function in the pathogenesis of freezing of gait in Parkinson\u2019s disease. J Neurol Sci. 2006;248(1\u20132):173\u20136.","journal-title":"J Neurol Sci"},{"issue":"11","key":"1699_CR13","doi-asserted-by":"publisher","first-page":"1511","DOI":"10.1007\/s00702-005-0285-5","volume":"112","author":"O Moore","year":"2005","unstructured":"Moore O, Kreitler S, Ehrenfeld M, Giladi N. Quality of life and gender identity in Parkinson\u2019s disease. J Neural Transm. 2005;112(11):1511\u201322.","journal-title":"J Neural Transm"},{"issue":"7","key":"1699_CR14","doi-asserted-by":"publisher","first-page":"768","DOI":"10.1016\/S1474-4422(15)00041-1","volume":"14","author":"J Nonnekes","year":"2015","unstructured":"Nonnekes J, Snijders AH, Nutt JG, Deuschl G, Giladi N, Bloem BR. Freezing of gait: a practical approach to management. Lancet Neurol. 2015;14(7):768\u201378.","journal-title":"Lancet Neurol"},{"issue":"4","key":"1699_CR15","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1136\/practneurol-2013-000743","volume":"14","author":"Y Okuma","year":"2014","unstructured":"Okuma Y. Practical approach to freezing of gait in Parkinson\u2019s disease. Pract Neurol. 2014;14(4):222\u201330.","journal-title":"Pract Neurol"},{"issue":"4","key":"1699_CR16","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1046\/j.1468-1331.2003.00611.x","volume":"10","author":"JD Schaafsma","year":"2003","unstructured":"Schaafsma JD, Balash Y, Gurevich T, Bartels AL, Hausdorff JM, Giladi N. Characterization of freezing of gait subtypes and the response of each to levodopa in Parkinson\u2019s disease. Eur J Neurol. 2003;10(4):391\u20138.","journal-title":"Eur J Neurol"},{"issue":"10","key":"1699_CR17","doi-asserted-by":"publisher","first-page":"894","DOI":"10.1016\/j.parkreldis.2013.04.004","volume":"19","author":"UM Fietzek","year":"2013","unstructured":"Fietzek UM, Zwosta J, Schroeteler FE, Ziegler K, Ceballos-Baumann AO. Levodopa changes the severity of freezing in Parkinson\u2019s disease. Parkin Relat Disord. 2013;19(10):894\u20136.","journal-title":"Parkin. Relat Disord"},{"key":"1699_CR18","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1038\/s41531-019-0099-z","volume":"5","author":"J Lucas McKay","year":"2019","unstructured":"Lucas McKay J, Goldstein FC, Sommerfeld B, Bernhard D, Perez Parra S, Factor SA. Freezing of gait can persist after an acute levodopa challenge in Parkinson\u2019s disease. NPJ Parkin Dis. 2019;5:25.","journal-title":"NPJ Parkin. Dis."},{"issue":"7","key":"1699_CR19","doi-asserted-by":"publisher","first-page":"454","DOI":"10.1212\/WNL.0b013e3182477ec0","volume":"78","author":"AJ Espay","year":"2012","unstructured":"Espay AJ, Fasano A, van Nuenen BFL, Payne MM, Snijders AH, Bloem BR. \u201cOn\u201d state freezing of gait in Parkinson disease: a paradoxical levodopa-induced complication. Neurology. 2012;78(7):454\u20137.","journal-title":"Neurology"},{"issue":"7","key":"1699_CR20","doi-asserted-by":"publisher","first-page":"695","DOI":"10.1191\/0269215505cr906oa","volume":"19","author":"I Lim","year":"2005","unstructured":"Lim I, van Wegen E, de Goede C, Deutekom M, Nieuwboer A, Willems A, Jones D, Rochester L, Kwakkel G. Effects of external rhythmical cueing on gait in patients with Parkinson\u2019s disease: a systematic review. Clin Rehabil. 2005;19(7):695\u2013713.","journal-title":"Clin Rehabil"},{"issue":"2","key":"1699_CR21","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1136\/jnnp.200X.097923","volume":"78","author":"A Nieuwboer","year":"2007","unstructured":"Nieuwboer A, Kwakkel G, Rochester L, Jones D, van Wegen E, Willems AM, Chavret F, Hetherington V, Baker K, Lim I. Cueing training in the home improves gait-related mobility in Parkinson\u2019s disease: the RESCUE trial. J Neurol Neurosurg Psychiatry. 2007;78(2):134\u201340.","journal-title":"J Neurol Neurosurg Psychiatry"},{"issue":"6","key":"1699_CR22","doi-asserted-by":"publisher","first-page":"1148","DOI":"10.1002\/mds.10259","volume":"17","author":"TC Rubinstein","year":"2002","unstructured":"Rubinstein TC, Giladi N, Hausdorff JM. The power of cueing to circumvent dopamine deficits: a review of physical therapy treatment of gait disturbances in Parkinson\u2019s disease. Mov Disord. 2002;17(6):1148\u201360.","journal-title":"Mov Disord"},{"issue":"3","key":"1699_CR23","doi-asserted-by":"publisher","first-page":"9675","DOI":"10.1371\/journal.pone.0009675","volume":"5","author":"P Arias","year":"2010","unstructured":"Arias P, Cudeiro J. Effect of rhythmic auditory stimulation on gait in parkinsonian patients with and without freezing of gait. PLoS ONE. 2010;5(3):9675.","journal-title":"PLoS ONE"},{"issue":"4","key":"1699_CR24","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1002\/mds.27936","volume":"35","author":"C Cosentino","year":"2020","unstructured":"Cosentino C, Baccini M, Putzolu M, Ristori D, Avanzino L, Pelosin E. Effectiveness of physiotherapy on freezing of gait in Parkinson\u2019s disease: a systematic review and Meta-Analyses. Mov Disord. 2020;35(4):523\u201336.","journal-title":"Mov Disord"},{"key":"1699_CR25","first-page":"66","volume":"6","author":"P Ginis","year":"2017","unstructured":"Ginis P, Nackaerts E, Nieuwboer A, Heremans E. Cueing for people with Parkinson\u2019s disease with freezing of gait: a narrative review of the state-of-the-art and novel perspectives. Med Ann Phys Rehabil. 2017;6:66.","journal-title":"Med Ann Phys Rehabil"},{"issue":"7","key":"1699_CR26","doi-asserted-by":"publisher","first-page":"1488","DOI":"10.1007\/s00415-017-8552-6","volume":"264","author":"P Ginis","year":"2017","unstructured":"Ginis P, Heremans E, Ferrari A, Bekkers EMJ, Canning CG, Nieuwboer A. External input for gait in people with Parkinson\u2019s disease with and without freezing of gait: one size does not fit all. J Neurol. 2017;264(7):1488\u201396.","journal-title":"J Neurol"},{"issue":"6","key":"1699_CR27","doi-asserted-by":"publisher","first-page":"783","DOI":"10.1002\/mds.27709","volume":"34","author":"M Mancini","year":"2019","unstructured":"Mancini M, Bloem BR, Horak FB, Lewis SJG, Nieuwboer A, Nonnekes J. Clinical and methodological challenges for assessing freezing of gait: future perspectives. Mov Disord. 2019;34(6):783\u201390.","journal-title":"Mov Disord"},{"issue":"5","key":"1699_CR28","doi-asserted-by":"publisher","first-page":"947","DOI":"10.1109\/TNSRE.2019.2910165","volume":"27","author":"N Naghavi","year":"2019","unstructured":"Naghavi N, Wade E. Prediction of freezing of gait in Parkinson\u2019s disease using statistical inference and Lower\u2013Limb acceleration data. IEEE Trans Neural Syst Rehabil Eng. 2019;27(5):947\u201355.","journal-title":"IEEE Trans Neural Syst Rehabil Eng"},{"key":"1699_CR29","doi-asserted-by":"publisher","first-page":"394","DOI":"10.3389\/fneur.2017.00394","volume":"8","author":"L Palmerini","year":"2017","unstructured":"Palmerini L, Rocchi L, Mazilu S, Gazit E, Hausdorff JM, Chiari L. Identification of characteristic motor patterns preceding freezing of gait in Parkinson\u2019s disease using wearable sensors. Front Neurol. 2017;8:394.","journal-title":"Front Neurol"},{"key":"1699_CR30","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1007\/978-3-642-39712-7_11","volume-title":"Machine learning and data mining in pattern recognition","author":"S Mazilu","year":"2013","unstructured":"Mazilu S, Calatroni A, Gazit E, Roggen D, Hausdorff JM, Tr\u00f6ster G. Feature learning for detection and prediction of freezing of gait in Parkinson\u2019s disease. In: Perner P, editor. Machine learning and data mining in pattern recognition. Berlin: Springer; 2013. p. 144\u201358."},{"key":"1699_CR31","first-page":"66","volume":"6","author":"F Demrozi","year":"2019","unstructured":"Demrozi F, Bacchin R, Tamburin S, Cristani M, Pravadelli G. Towards a wearable system for predicting the freezing of gait in people affected by Parkinson\u2019s disease. IEEE J Biomed Health Inform. 2019;6:66.","journal-title":"IEEE J Biomed Health Inform"},{"issue":"2","key":"1699_CR32","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1007\/s00221-002-1354-8","volume":"149","author":"JM Hausdorff","year":"2003","unstructured":"Hausdorff JM, Schaafsma JD, Balash Y, Bartels AL, Gurevich T, Giladi N. Impaired regulation of stride variability in Parkinson\u2019s disease subjects with freezing of gait. Exp Brain Res. 2003;149(2):187\u201394.","journal-title":"Exp Brain Res"},{"issue":"Pt 8","key":"1699_CR33","doi-asserted-by":"publisher","first-page":"2151","DOI":"10.1093\/brain\/awp053","volume":"132","author":"R Chee","year":"2009","unstructured":"Chee R, Murphy A, Danoudis M, Georgiou-Karistianis N, Iansek R. Gait freezing in Parkinson\u2019s disease and the stride length sequence effect interaction. Brain. 2009;132(Pt 8):2151\u201360.","journal-title":"Brain"},{"issue":"8","key":"1699_CR34","doi-asserted-by":"publisher","first-page":"1999","DOI":"10.1111\/j.1460-9568.2008.06167.x","volume":"27","author":"M Plotnik","year":"2008","unstructured":"Plotnik M, Giladi N, Hausdorff JM. Bilateral coordination of walking and freezing of gait in Parkinson\u2019s disease. Eur J Neurosci. 2008;27(8):1999\u20132006.","journal-title":"Eur J Neurosci"},{"key":"1699_CR35","doi-asserted-by":"crossref","unstructured":"Filtjens B, Nieuwboer A, D\u2019cruz N, Spildooren J, Slaets P, Vanrumste B. A data-driven approach for detecting gait events during turning in people with Parkinson\u2019s disease and freezing of gait. Gait Post. 2020;80:130\u20136.","DOI":"10.1016\/j.gaitpost.2020.05.026"},{"key":"1699_CR36","doi-asserted-by":"crossref","unstructured":"Filtjens B, Ginis P, Nieuwboer A, Slaets P, Vanrumste B, Automated freezing of gait assessment with marker-based motion capture and multi-stage graph convolutional neural networks approaches expert-level detection. arXiv e-prints. 2021;2103\u201315449. arXiv:2103.15449","DOI":"10.1186\/s12984-022-01025-3"},{"issue":"4","key":"1699_CR37","doi-asserted-by":"publisher","first-page":"1215","DOI":"10.1109\/JBHI.2019.2923209","volume":"24","author":"K Hu","year":"2020","unstructured":"Hu K, Wang Z, Mei S, Ehgoetz Martens KA, Yao T, Lewis SJG, Feng DD. Vision-based freezing of gait detection with anatomic directed graph representation. IEEE J Biomed Health Inform. 2020;24(4):1215\u201325.","journal-title":"IEEE J Biomed Health Inform"},{"key":"1699_CR38","unstructured":"Masiala S, Huijbers W, Atzmueller M, Feature-Set-Engineering for detecting freezing of gait in parkinson\u2019s disease using deep recurrent neural networks. pre-print. 2019. arXiv:1909.03428"},{"key":"1699_CR39","doi-asserted-by":"crossref","unstructured":"Camps J, Sam\u00e0 A, Mart\u00edn M, Rodr\u00edguez-Mart\u00edn D, P\u00e9rez-L\u00f3pez C, Alcaine S, Mestre B, Prats A, Crespo MC, Cabestany J, Bay\u00e9s \u00c0, Catal\u00e0 A. Deep learning for detecting freezing of gait episodes in Parkinson\u2019s disease based on accelerometers. In: Advances in computational intelligence. Springer; 2017. pp. 344\u201355.","DOI":"10.1007\/978-3-319-59147-6_30"},{"issue":"7","key":"1699_CR40","doi-asserted-by":"publisher","first-page":"66","DOI":"10.3390\/s20071895","volume":"20","author":"L Sigcha","year":"2020","unstructured":"Sigcha L, Costa N, Pav\u00f3n I, Costa S, Arezes P, L\u00f3pez JM, De Arcas G. Deep learning approaches for detecting freezing of gait in Parkinson\u2019s disease patients through on-body acceleration sensors. Sensors. 2020;20(7):66.","journal-title":"Sensors"},{"key":"1699_CR41","doi-asserted-by":"crossref","unstructured":"Wang Z, Yan W, Oates T, Time series classification from scratch with deep neural networks: a strong baseline. 2016. arXiv:1611.06455","DOI":"10.1109\/IJCNN.2017.7966039"},{"issue":"7623","key":"1699_CR42","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1038\/538020a","volume":"538","author":"D Castelvecchi","year":"2016","unstructured":"Castelvecchi D. Can we open the black box of AI? Nature. 2016;538(7623):20\u20133.","journal-title":"Nature"},{"key":"1699_CR43","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","volume":"58","author":"A Barredo Arrieta","year":"2020","unstructured":"Barredo Arrieta A, D\u00edaz-Rodr\u00edguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, Garcia S, Gil-Lopez S, Molina D, Benjamins R, Chatila R, Herrera F. Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf Fusion. 2020;58:82\u2013115.","journal-title":"Inf Fusion"},{"key":"1699_CR44","unstructured":"Ancona M, Ceolini E, \u00d6ztireli C, Gross M, Towards better understanding of gradient-based attribution methods for deep neural networks. 2017. arXiv:1711.06104"},{"issue":"7","key":"1699_CR45","doi-asserted-by":"publisher","first-page":"0130140","DOI":"10.1371\/journal.pone.0130140","volume":"10","author":"S Bach","year":"2015","unstructured":"Bach S, Binder A, Montavon G, Klauschen F, M\u00fcller K-R, Samek W. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE. 2015;10(7):0130140.","journal-title":"PLoS ONE"},{"key":"1699_CR46","doi-asserted-by":"publisher","first-page":"194","DOI":"10.3389\/fnagi.2019.00194","volume":"11","author":"M B\u00f6hle","year":"2019","unstructured":"B\u00f6hle M, Eitel F, Weygandt M, Ritter K. Layer-wise relevance propagation for explaining deep neural network decisions in MRI-based Alzheimer\u2019s disease classification. Front Aging Neurosci. 2019;11:194.","journal-title":"Front Aging Neurosci"},{"key":"1699_CR47","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1016\/j.jneumeth.2016.10.008","volume":"274","author":"I Sturm","year":"2016","unstructured":"Sturm I, Lapuschkin S, Samek W, M\u00fcller K-R. Interpretable deep neural networks for single-trial EEG classification. J Neurosci Methods. 2016;274:141\u20135.","journal-title":"J Neurosci Methods"},{"issue":"1","key":"1699_CR48","doi-asserted-by":"publisher","first-page":"2391","DOI":"10.1038\/s41598-019-38748-8","volume":"9","author":"F Horst","year":"2019","unstructured":"Horst F, Lapuschkin S, Samek W, M\u00fcller K-R, Sch\u00f6llhorn WI. Explaining the unique nature of individual gait patterns with deep learning. Sci Rep. 2019;9(1):2391.","journal-title":"Sci Rep"},{"issue":"15","key":"1699_CR49","doi-asserted-by":"publisher","first-page":"2563","DOI":"10.1002\/mds.23327","volume":"25","author":"J Spildooren","year":"2010","unstructured":"Spildooren J, Vercruysse S, Desloovere K, Vandenberghe W, Kerckhofs E, Nieuwboer A. Freezing of gait in Parkinson\u2019s disease: the impact of dual-tasking and turning. Mov Disord. 2010;25(15):2563\u201370.","journal-title":"Mov Disord"},{"issue":"4","key":"1699_CR50","doi-asserted-by":"publisher","first-page":"459","DOI":"10.1016\/j.gaitpost.2009.07.108","volume":"30","author":"A Nieuwboer","year":"2009","unstructured":"Nieuwboer A, Rochester L, Herman T, Vandenberghe W, Emil GE, Thomaes T, Giladi N. Reliability of the new freezing of gait questionnaire: agreement between patients with Parkinson\u2019s disease and their carers. Gait Post. 2009;30(4):459\u201363.","journal-title":"Gait Post"},{"issue":"3","key":"1699_CR51","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1002\/jor.1100080310","volume":"8","author":"MP Kadaba","year":"1990","unstructured":"Kadaba MP, Ramakrishnan HK, Wootten ME. Measurement of lower extremity kinematics during level walking. J Orthop Res. 1990;8(3):383\u201392.","journal-title":"J Orthop Res"},{"issue":"5","key":"1699_CR52","doi-asserted-by":"publisher","first-page":"575","DOI":"10.1016\/0167-9457(91)90046-Z","volume":"10","author":"RB Davis","year":"1991","unstructured":"Davis RB, \u00d5unpuu S, Tyburski D, Gage JR. A gait analysis data collection and reduction technique. Hum Mov Sci. 1991;10(5):575\u201387.","journal-title":"Hum Mov Sci"},{"issue":"3","key":"1699_CR53","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1016\/j.apmr.2005.11.021","volume":"87","author":"CG Canning","year":"2006","unstructured":"Canning CG, Ada L, Johnson JJ, McWhirter S. Walking capacity in mild to moderate Parkinson\u2019s disease. Arch Phys Med Rehabil. 2006;87(3):371\u20135.","journal-title":"Arch Phys Med Rehabil"},{"issue":"4","key":"1699_CR54","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1093\/ageing\/30.4.319","volume":"30","author":"A Bowen","year":"2001","unstructured":"Bowen A, Wenman R, Mickelborough J, Foster J, Hill E, Tallis R. Dual-task effects of talking while walking on velocity and balance following a stroke. Age Ageing. 2001;30(4):319\u201323.","journal-title":"Age Ageing"},{"key":"1699_CR55","doi-asserted-by":"publisher","first-page":"298","DOI":"10.1016\/j.neuroscience.2012.01.024","volume":"207","author":"J Spildooren","year":"2012","unstructured":"Spildooren J, Vercruysse S, Meyns P, Vandenbossche J, Heremans E, Desloovere K, Vandenberghe W, Nieuwboer A. Turning and unilateral cueing in Parkinson\u2019s disease patients with and without freezing of gait. Neuroscience. 2012;207:298\u2013306.","journal-title":"Neuroscience"},{"issue":"4","key":"1699_CR56","doi-asserted-by":"publisher","first-page":"710","DOI":"10.1016\/j.gaitpost.2007.07.007","volume":"27","author":"JA Zeni Jr","year":"2008","unstructured":"Zeni JA Jr, Richards JG, Higginson JS. Two simple methods for determining gait events during treadmill and overground walking using kinematic data. Gait Post. 2008;27(4):710\u20134.","journal-title":"Gait Post"},{"key":"1699_CR57","unstructured":"Hsu C-W, Chang C-C, Lin C-J, A practical guide to support vector classification. Technical report, Department of Computer Science, National Taiwan University. 2003. http:\/\/www.csie.ntu.edu.tw\/~cjlin\/papers.html"},{"issue":"4","key":"1699_CR58","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1007\/s10618-019-00619-1","volume":"33","author":"H Ismail Fawaz","year":"2019","unstructured":"Ismail Fawaz H, Forestier G, Weber J, Idoumghar L, Muller P-A. Deep learning for time series classification: a review. Data Min Knowl Discov. 2019;33(4):917\u201363.","journal-title":"Data Min Knowl Discov"},{"issue":"11","key":"1699_CR59","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"Y Lecun","year":"1998","unstructured":"Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE. 1998;86(11):2278\u2013324.","journal-title":"Proc IEEE"},{"key":"1699_CR60","volume-title":"Deep learning","author":"IJ Goodfellow","year":"2016","unstructured":"Goodfellow IJ, Bengio Y, Courville A. Deep learning. Cambridge: MIT Press; 2016."},{"key":"1699_CR61","unstructured":"Lin M, Chen Q, Yan S. Network in network. 2013. arXiv:1312.4400"},{"key":"1699_CR62","first-page":"1929","volume":"15","author":"N Srivastava","year":"2014","unstructured":"Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15:1929\u201358.","journal-title":"J Mach Learn Res"},{"key":"1699_CR63","doi-asserted-by":"crossref","unstructured":"Boser BE, Guyon IM, Vapnik VN. A training algorithm for optimal margin classifiers. In: Proceedings of the fifth annual workshop on computational learning theory (COLT\u201992). New York: Association for Computing Machinery; 1992. pp 144\u201352.","DOI":"10.1145\/130385.130401"},{"issue":"3","key":"1699_CR64","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1007\/BF00994018","volume":"20","author":"C Cortes","year":"1995","unstructured":"Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20(3):273\u201397.","journal-title":"Mach Learn"},{"issue":"85","key":"1699_CR65","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay \u00c9. Scikit-learn: machine learning in python. J Mach Learn Res. 2011;12(85):2825\u201330.","journal-title":"J Mach Learn Res"},{"key":"1699_CR66","unstructured":"Bergstra JS, Bardenet R, Bengio Y, K\u00e9gl B, Algorithms for hyper-parameter optimization. In: Shawe-Taylor J, Zemel RS, Bartlett PL, Pereira F, Weinberger KQ, editors. Advances in neural information processing systems, vol 24. Red Hook: Curran Associates, Inc.; 2011. pp 2546\u201354. http:\/\/papers.nips.cc\/paper\/4443-algorithms-for-hyper-parameter-optimization.pdf"},{"issue":"2","key":"1699_CR67","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1007\/BF02295996","volume":"12","author":"Q McNEMAR","year":"1947","unstructured":"McNEMAR Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. 1947;12(2):153\u20137.","journal-title":"Psychometrika"},{"key":"1699_CR68","unstructured":"Raschka S. Model evaluation, model selection, and algorithm selection in machine learning. 2018. arXiv:1811.12808"},{"key":"1699_CR69","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1016\/j.patcog.2016.11.008","volume":"65","author":"G Montavon","year":"2017","unstructured":"Montavon G, Lapuschkin S, Binder A, Samek W, M\u00fcller K-R. Explaining nonlinear classification decisions with deep Taylor decomposition. Pattern Recognit. 2017;65:211\u201322.","journal-title":"Pattern Recognit"},{"issue":"Pt 7","key":"1699_CR70","doi-asserted-by":"publisher","first-page":"1650","DOI":"10.1093\/brain\/awh189","volume":"127","author":"A Nieuwboer","year":"2004","unstructured":"Nieuwboer A, Dom R, De Weerdt W, Desloovere K, Janssens L, Stijn V. Electromyographic profiles of gait prior to onset of freezing episodes in patients with Parkinson\u2019s disease. Brain. 2004;127(Pt 7):1650\u201360.","journal-title":"Brain"},{"issue":"18","key":"1699_CR71","doi-asserted-by":"publisher","first-page":"66","DOI":"10.3390\/s19183898","volume":"19","author":"N Naghavi","year":"2019","unstructured":"Naghavi N, Miller A, Wade E. Towards real-time prediction of freezing of gait in patients with Parkinson\u2019s disease: addressing the class imbalance problem. Sensors. 2019;19(18):66.","journal-title":"Sensors"},{"issue":"9","key":"1699_CR72","doi-asserted-by":"publisher","first-page":"1909","DOI":"10.1109\/TNSRE.2019.2933626","volume":"27","author":"A Arami","year":"2019","unstructured":"Arami A, Poulakakis-Daktylidis A, Tai YF, Burdet E. Prediction of gait freezing in parkinsonian patients: a binary classification augmented with time series prediction. IEEE Trans Neural Syst Rehabil Eng. 2019;27(9):1909\u201319.","journal-title":"IEEE Trans Neural Syst Rehabil Eng"},{"issue":"9","key":"1699_CR73","doi-asserted-by":"publisher","first-page":"1268","DOI":"10.1007\/s00415-006-0514-3","volume":"254","author":"A Nieuwboer","year":"2007","unstructured":"Nieuwboer A, Chavret F, Willems A-M, Desloovere K. Does freezing in Parkinson\u2019s disease change limb coordination? J Neurol. 2007;254(9):1268.","journal-title":"J Neurol"},{"key":"1699_CR74","first-page":"459321","volume":"2012","author":"M Plotnik","year":"2012","unstructured":"Plotnik M, Giladi N, Hausdorff JM. Is freezing of gait in Parkinson\u2019s disease a result of multiple gait impairments? Implications for treatment. Parkin Dis. 2012;2012:459321.","journal-title":"Parkin. Dis"},{"issue":"1","key":"1699_CR75","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1016\/j.cmpb.2014.01.012","volume":"114","author":"A Barre","year":"2014","unstructured":"Barre A, Armand S. Biomechanical ToolKit: open-source framework to visualize and process biomechanical data. Comput Methods Programs Biomed. 2014;114(1):80\u20137.","journal-title":"Comput Methods Programs Biomed"},{"key":"1699_CR76","unstructured":"Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg J, Monga R, Moore S, Murray DG, Steiner B, Tucker P, Vasudevan V, Warden P, Wicke M, Yu Y, Zheng X. TensorFlow: a system for large-scale machine learning. In: Proceedings of the 12th USENIX conference on operating systems design and implementation (OSDI\u201916). USA: USENIX Association; 2016. pp 265\u201383."},{"issue":"1","key":"1699_CR77","doi-asserted-by":"publisher","first-page":"014008","DOI":"10.1088\/1749-4699\/8\/1\/014008","volume":"8","author":"J Bergstra","year":"2015","unstructured":"Bergstra J, Komer B, Eliasmith C, Yamins D, Cox DD. Hyperopt: a python library for model selection and hyperparameter optimization. Comput Sci Discov. 2015;8(1):014008.","journal-title":"Comput Sci Discov"},{"key":"1699_CR78","unstructured":"Ridgway G, Bipolar Colormap; 2020. https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/26026-bipolar-colormap Accessed 17 June 2020"},{"issue":"3","key":"1699_CR79","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1080\/10255842.2010.527837","volume":"15","author":"TC Pataky","year":"2012","unstructured":"Pataky TC. One-dimensional statistical parametric mapping in python. Comput Methods Biomech Biomed Engin. 2012;15(3):295\u2013301.","journal-title":"Comput Methods Biomech Biomed Engin"},{"key":"1699_CR80","first-page":"66","volume":"6","author":"S Raschka","year":"2018","unstructured":"Raschka S. Mlxtend: providing machine learning and data science utilities and extensions to python\u2019s scientific computing stack. J Open Source Softw. 2018;6:66.","journal-title":"J Open Source Softw"},{"issue":"15","key":"1699_CR81","doi-asserted-by":"publisher","first-page":"2129","DOI":"10.1002\/mds.22340","volume":"23","author":"CG Goetz","year":"2008","unstructured":"Goetz CG, Tilley BC, Shaftman SR, Stebbins GT, Fahn S, Martinez-Martin P, Poewe W, Sampaio C, Stern MB, Dodel R, Dubois B, Holloway R, Jankovic J, Kulisevsky J, Lang AE, Lees A, Leurgans S, LeWitt PA, Nyenhuis D, Olanow CW, Rascol O, Schrag A, Teresi JA, van Hilten JJ, LaPelle N. Movement Disorder Society UPDRS Revision Task Force: movement disorder society-sponsored revision of the unified Parkinson\u2019s disease rating scale (MDS-UPDRS): scale presentation and clinimetric testing results. Mov Disord. 2008;23(15):2129\u201370.","journal-title":"Mov Disord"},{"issue":"5","key":"1699_CR82","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1212\/WNL.17.5.427","volume":"17","author":"MM Hoehn","year":"1967","unstructured":"Hoehn MM, Yahr MD. Parkinsonism: onset, progression and mortality. Neurology. 1967;17(5):427\u201342.","journal-title":"Neurology"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01699-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-021-01699-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01699-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,10]],"date-time":"2023-02-10T09:28:55Z","timestamp":1676021335000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-021-01699-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,7]]},"references-count":82,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["1699"],"URL":"https:\/\/doi.org\/10.1186\/s12911-021-01699-0","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,7]]},"assertion":[{"value":"23 September 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 November 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 December 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The study was approved by the local ethics committee of the University Hospital Leuven and all subjects gave written informed consent.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that there is no competing interests regarding the publication of this article.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"341"}}