{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T22:43:43Z","timestamp":1781822623898,"version":"3.54.5"},"reference-count":49,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2018,6,27]],"date-time":"2018-06-27T00:00:00Z","timestamp":1530057600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000832","name":"Dowager Countess Eleanor Peel Trust","doi-asserted-by":"publisher","award":["JWP 15\/07-71"],"award-info":[{"award-number":["JWP 15\/07-71"]}],"id":[{"id":"10.13039\/501100000832","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Falls in older adults present a major growing healthcare challenge and reliable detection of falls is crucial to minimise their consequences. The majority of development and testing has used laboratory simulations. As simulations do not cover the wide range of real-world scenarios performance is poor when retested using real-world data. There has been a move from the use of simulated falls towards the use of real-world data. This review aims to assess the current methods for real-world evaluation of fall detection systems, identify their limitations and propose improved robust methods of evaluation. Twenty-two articles met the inclusion criteria and were assessed with regard to the composition of the datasets, data processing methods and the measures of performance. Real-world tests of fall detection technology are inherently challenging and it is clear the field is in its infancy. Most studies used small datasets and studies differed on how to quantify the ability to avoid false alarms and how to identify non-falls, a concept which is virtually impossible to define and standardise. To increase robustness and make results comparable, larger standardised datasets are needed containing data from a range of participant groups. Measures that depend on the definition and identification of non-falls should be avoided. Sensitivity, precision and F-measure emerged as the most suitable robust measures for evaluating the real-world performance of fall detection systems.<\/jats:p>","DOI":"10.3390\/s18072060","type":"journal-article","created":{"date-parts":[[2018,6,27]],"date-time":"2018-06-27T11:02:05Z","timestamp":1530097325000},"page":"2060","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":61,"title":["Methods for the Real-World Evaluation of Fall Detection Technology: A Scoping Review"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1890-1440","authenticated-orcid":false,"given":"Robert W.","family":"Broadley","sequence":"first","affiliation":[{"name":"School of Health Sciences, University of Salford, Salford, M6 6PU, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5987-447X","authenticated-orcid":false,"given":"Jochen","family":"Klenk","sequence":"additional","affiliation":[{"name":"Department of Clinical Gerontology, Robert-Bosch-Hospital, 70376 Stuttgart, Germany"},{"name":"Institute of Epidemiology and Medical Biometry, Ulm University, 89081 Ulm, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9889-2243","authenticated-orcid":false,"given":"Sibylle B.","family":"Thies","sequence":"additional","affiliation":[{"name":"School of Health Sciences, University of Salford, Salford, M6 6PU, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2164-3892","authenticated-orcid":false,"given":"Laurence P. J.","family":"Kenney","sequence":"additional","affiliation":[{"name":"School of Health Sciences, University of Salford, Salford, M6 6PU, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0722-2760","authenticated-orcid":false,"given":"Malcolm H.","family":"Granat","sequence":"additional","affiliation":[{"name":"School of Health Sciences, University of Salford, Salford, M6 6PU, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,6,27]]},"reference":[{"key":"ref_1","unstructured":"World Health Organisation (2007). WHO Global Report on Falls Prevention in Older Age, World Health Organization."},{"key":"ref_2","unstructured":"Department of Health (UK) (2009). Falls and Fractures: Effective Interventions in Health and Social Care, Technical Report."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"871","DOI":"10.1111\/j.1532-5415.1995.tb05529.x","article-title":"Incidence of Injury-Causing Falls among Older Adults by Place of Residence: A Population-Based Study","volume":"43","author":"Luukinen","year":"1995","journal-title":"J. Am. Geriatr. Soc."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1701","DOI":"10.1056\/NEJM198812293192604","article-title":"Risk Factors for Falls among Elderly Persons Living in the Community","volume":"319","author":"Tinetti","year":"1988","journal-title":"N. Engl. J. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1007\/s00198-009-1100-1","article-title":"Cost of Falls in Old Age: A Systematic Review","volume":"21","author":"Heinrich","year":"2010","journal-title":"Osteoporos. Int."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"a2227","DOI":"10.1136\/bmj.a2227","article-title":"Inability to Get up after Falling, Subsequent Time on Floor, and Summoning Help: Prospective Cohort Study in People over 90","volume":"337","author":"Fleming","year":"2008","journal-title":"BMJ"},{"key":"ref_7","unstructured":"Treml, J., Husk, J., Lowe, D., and Vasilakis, N. (2010). Falling Standards, Broken Promises: Report of the National Audit of Falls and Bone Health in Older People 2010, Royal College of Physicians. Technical Report."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1146","DOI":"10.1111\/j.1532-5415.1995.tb07017.x","article-title":"Falls in Community-Dwelling Older Persons","volume":"43","author":"King","year":"1995","journal-title":"J. Am. Geriatr. Soc."},{"key":"ref_9","unstructured":"Lord, S.R., Sherrington, C., and Menz, H.B. (2001). Falls in Older People : Risk Factors and Strategies for Prevention, Cambridge University Press."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2663","DOI":"10.1001\/jama.1989.03420180087036","article-title":"Risk Factors for Recurrent Nonsyncopal Falls: A Prospective Study","volume":"261","author":"Nevitt","year":"1989","journal-title":"JAMA"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1136\/bmj.282.6260.266","article-title":"How Dangerous Are Falls in Old People at Home?","volume":"282","author":"Wild","year":"1981","journal-title":"Br. Med. J. (Clin. Res. Ed.)"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1093\/ageing\/afm169","article-title":"Fear of Falling: Measurement Strategy, Prevalence, Risk Factors and Consequences among Older Persons","volume":"37","author":"Scheffer","year":"2008","journal-title":"Age Ageing"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1093\/geront\/38.5.549","article-title":"Covariates of Fear of Falling and Associated Activity Curtailment","volume":"38","author":"Howland","year":"1998","journal-title":"Gerontologist"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"516","DOI":"10.1046\/j.1532-5415.2002.50119.x","article-title":"Characteristics Associated with Fear of Falling and Activity Restriction in Community-Living Older Persons","volume":"50","author":"Murphy","year":"2002","journal-title":"J. Am. Geriatr. Soc."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1007\/s00391-010-0127-4","article-title":"Satisfaction and Use of Personal Emergency Response Systems","volume":"43","author":"Hautzinger","year":"2010","journal-title":"Z. Gerontol. Geriatr."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1038\/533452a","article-title":"1,500 Scientists Lift the Lid on Reproducibility","volume":"533","author":"Baker","year":"2016","journal-title":"Nature"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bagala, F., Becker, C., Cappello, A., Chiari, L., Aminian, K., Hausdorff, J.M., Zijlstra, W., and Klenk, J. (2012). Evaluation of Accelerometer-Based Fall Detection Algorithms on Real-World Falls. PLoS ONE, 7.","DOI":"10.1371\/journal.pone.0037062"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"706","DOI":"10.1007\/s00391-013-0559-8","article-title":"Fall Detection with Body-Worn Sensors : A Systematic Review","volume":"46","author":"Schwickert","year":"2013","journal-title":"Z. Gerontol. Geriatr."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1519\/JPT.0b013e3182abe779","article-title":"Fall Detection Devices and Their Use with Older Adults: A Systematic Review","volume":"37","author":"Chaudhuri","year":"2014","journal-title":"J. Geriatr. Phys. Ther."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Skubic, M., Harris, B.H., Stone, E., Ho, K.C., Su, B.Y., and Rantz, M. (2016, January 16\u201320). Testing Non-Wearable Fall Detection Methods in the Homes of Older Adults. Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7590763"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Aziz, O., Klenk, J., Schwickert, L., Chiari, L., Becker, C., Park, E.J., Mori, G., and Robinovitch, S.N. (2017). Validation of Accuracy of SVM-Based Fall Detection System Using Real-World Fall and Non-Fall Datasets. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0180318"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1016\/j.rehab.2011.07.962","article-title":"Evaluation under Real-Life Conditions of a Stand-Alone Fall Detector for the Elderly Subjects","volume":"54","author":"Bloch","year":"2011","journal-title":"Ann. Phys. Rehabil. Med."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Bourke, A.K., Klenk, J., Schwickert, L., Aminian, K., Ihlen, E.A.F., Mellone, S., Helbostad, J.L., Chiari, L., and Becker, C. (2016, January 16\u201320). Fall Detection Algorithms for Real-World Falls Harvested from Lumbar Sensors in the Elderly Population: A Machine Learning Approach. Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7591534"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2415","DOI":"10.1111\/jgs.13804","article-title":"Real World Accuracy and Use of a Wearable Fall Detection Device by Older Adults","volume":"63","author":"Chaudhuri","year":"2015","journal-title":"J. Am. Geriatr. Soc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1080\/10739149.2016.1268155","article-title":"Enhanced Characterization of an Accelerometer-Based Fall Detection Algorithm Using a Repository","volume":"45","author":"Chen","year":"2017","journal-title":"Instrum. Sci. Technol."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Debard, G., Karsmakers, P., Deschodt, M., Vlaeyen, E., Van Den Bergh, J., Dejaeger, E., Milisen, K., Goedeme, T., Tuytelaars, T., and Vanrumste, B. Camera Based Fall Detection Using Multiple Features Validated with Real Life Video. Workshop Proceedings of the 7th International Conference on Intelligent Environments, The Netherlands, 2011.","DOI":"10.1007\/978-3-642-34091-8_16"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"149","DOI":"10.3233\/AIS-160369","article-title":"Camera-Based Fall Detection Using Real-World versus Simulated Data: How Far Are We from the Solution?","volume":"8","author":"Debard","year":"2016","journal-title":"J. Ambient Intell. Smart Environ."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Debard, G., Mertens, M., Goedeme, T., Tuytelaars, T., and Vanrumste, B. (2017). Three Ways to Improve the Performance of Real-Life Camera-Based Fall Detection Systems. J. Sens.","DOI":"10.1155\/2017\/8241910"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1007\/s00391-014-0805-8","article-title":"Multimodal Sensor-Based Fall Detection within the Domestic Environment of Elderly People","volume":"47","author":"Feldwieser","year":"2014","journal-title":"Z. Gerontol. Geriatr."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1007\/s00391-012-0400-9","article-title":"GAL@Home: A Feasibility Study of Sensor-Based in-Home Fall Detection","volume":"45","author":"Gietzelt","year":"2012","journal-title":"Z. Gerontol. Geriatr."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Godfrey, A., Bourke, A., Del Din, S., Morris, R., Hickey, A., Helbostad, J.L., Rochester, L., Godfrey, A., Bourke, A., and Del Din, S. (2016, January 16\u201320). Towards Holistic Free-Living Assessment in Parkinson\u2019s Disease: Unification of Gait and Fall Algorithms with a Single Accelerometer. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine And Biology Society, Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7590786"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Hu, X., Dor, R., Bosch, S., Khoong, A., Li, J., Stark, S., and Lu, C. (2017, January 29\u201331). Challenges in Studying Falls of Community-Dwelling Older Adults in the Real World. Proceedings of the 2017 IEEE International Conference on Smart Computing (SMARTCOMP), Hong Kong, China.","DOI":"10.1109\/SMARTCOMP.2017.7946993"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1159\/000362720","article-title":"Sensitivity and False Alarm Rate of a Fall Sensor in Long-Term Fall Detection in the Elderly","volume":"61","author":"Kangas","year":"2015","journal-title":"Gerontology"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1111\/jgs.13708","article-title":"Evaluation of an Automated Falls Detection Device in Nursing Home Residents","volume":"64","author":"Lipsitz","year":"2016","journal-title":"J. Am. Geriatr. Soc."},{"key":"ref_35","unstructured":"Liu, L., Popescu, M., Skubic, M., and Rantz, M. (2014, January 26\u201330). An Automatic Fall Detection Framework Using Data Fusion of Doppler Radar and Motion Sensor Network. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Chicago, IL, USA."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"11575","DOI":"10.3390\/s150511575","article-title":"A Wavelet-Based Approach to Fall Detection","volume":"15","author":"Palmerini","year":"2015","journal-title":"Sensors"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Rezaee, K., Haddadnia, J., and Delbari, A. (2013, January 10\u201312). Intelligent Detection of the Falls in the Elderly Using Fuzzy Inference System and Video-Based Motion Estimation Method. Proceedings of the 2013 8th Iranian Conference on Machine Vision and Image Processing (MVIP), Zanjan, Iran.","DOI":"10.1109\/IranianMVIP.2013.6779996"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Soaz, C., Lederer, C., and Daumer, M. (September, January 28). A New Method to Estimate the Real Upper Limit of the False Alarm Rate in a 3 Accelerometry-Based Fall Detector for the Elderly. Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA.","DOI":"10.1109\/EMBC.2012.6345915"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1109\/JBHI.2014.2312180","article-title":"Fall Detection in Homes of Older Adults Using the Microsoft Kinect","volume":"19","author":"Stone","year":"2015","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Yu, S., Chen, H., and Brown, R.A. (2017). Hidden Markov Model-Based Fall Detection with Motion Sensor Orientation Calibration: A Case for Real-Life Home Monitoring. IEEE J. Biomed. Health Inform.","DOI":"10.1109\/JBHI.2017.2782079"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"19806","DOI":"10.3390\/s141019806","article-title":"Survey on Fall Detection and Fall Prevention Using Wearable and External Sensors","volume":"14","author":"Delahoz","year":"2014","journal-title":"Sensors"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/0022-3956(75)90026-6","article-title":"\u201cMini-Mental State\u201d: A Practical Method for Grading the Cognitive State of Patients for the Clinician","volume":"12","author":"Folstein","year":"1975","journal-title":"J. Psychiatr. Res."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1007\/s10994-009-5119-5","article-title":"Measuring Classifier Performance: A Coherent Alternative to the Area under the ROC Curve","volume":"77","author":"Hand","year":"2009","journal-title":"Mach. Learn."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"822","DOI":"10.1093\/bioinformatics\/btq037","article-title":"Small-Sample Precision of ROC-Related Estimates","volume":"26","author":"Hanczar","year":"2010","journal-title":"Bioinformatics"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1111\/j.1466-8238.2007.00358.x","article-title":"AUC: A Misleading Measure of the Performance of Predictive Distribution Models","volume":"17","author":"Lobo","year":"2008","journal-title":"Glob. Ecol. Biogeogr."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Davis, J., and Goadrich, M. (2006, January 25\u201329). The Relationship between Precision-Recall and ROC Curves. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA.","DOI":"10.1145\/1143844.1143874"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"720","DOI":"10.1007\/s00391-013-0554-0","article-title":"Development of a Standard Fall Data Format for Signals from Body-Worn Sensors","volume":"46","author":"Klenk","year":"2013","journal-title":"Z. Gerontol. Geriatr."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1186\/s11556-016-0168-9","article-title":"The FARSEEING Real-World Fall Repository: A Large-Scale Collaborative Database to Collect and Share Sensor Signals from Real-World Falls","volume":"13","author":"Klenk","year":"2016","journal-title":"Eur. Rev. Aging Phys. Act."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1007\/s00391-012-0403-6","article-title":"Proposal for a Multiphase Fall Model Based on Real-World Fall Recordings with Body-Fixed Sensors","volume":"45","author":"Becker","year":"2012","journal-title":"Z. Gerontol. Geriatr."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/7\/2060\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:10:28Z","timestamp":1760195428000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/7\/2060"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,27]]},"references-count":49,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2018,7]]}},"alternative-id":["s18072060"],"URL":"https:\/\/doi.org\/10.3390\/s18072060","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,6,27]]}}}