{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T16:06:33Z","timestamp":1775145993394,"version":"3.50.1"},"reference-count":47,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2023,6,5]],"date-time":"2023-06-05T00:00:00Z","timestamp":1685923200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1734501"],"award-info":[{"award-number":["1734501"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Walking in real-world environments involves constant decision-making, e.g., when approaching a staircase, an individual decides whether to engage (climbing the stairs) or avoid. For the control of assistive robots (e.g., robotic lower-limb prostheses), recognizing such motion intent is an important but challenging task, primarily due to the lack of available information. This paper presents a novel vision-based method to recognize an individual\u2019s motion intent when approaching a staircase before the potential transition of motion mode (walking to stair climbing) occurs. Leveraging the egocentric images from a head-mounted camera, the authors trained a YOLOv5 object detection model to detect staircases. Subsequently, an AdaBoost and gradient boost (GB) classifier was developed to recognize the individual\u2019s intention of engaging or avoiding the upcoming stairway. This novel method has been demonstrated to provide reliable (97.69%) recognition at least 2 steps before the potential mode transition, which is expected to provide ample time for the controller mode transition in an assistive robot in real-world use.<\/jats:p>","DOI":"10.3390\/s23115355","type":"journal-article","created":{"date-parts":[[2023,6,6]],"date-time":"2023-06-06T02:08:15Z","timestamp":1686017295000},"page":"5355","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Vision-Based Recognition of Human Motion Intent during Staircase Approaching"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2180-5006","authenticated-orcid":false,"given":"Md Rafi","family":"Islam","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35487, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5212-8538","authenticated-orcid":false,"given":"Md Rejwanul","family":"Haque","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, The University of Alabama, Tuscaloosa, AL 35487, USA"}]},{"given":"Masudul H.","family":"Imtiaz","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Clarkson University, Potsdam, NY 13699, USA"}]},{"given":"Xiangrong","family":"Shen","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, The University of Alabama, Tuscaloosa, AL 35487, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7792-4234","authenticated-orcid":false,"given":"Edward","family":"Sazonov","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35487, USA"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,5]]},"reference":[{"key":"ref_1","unstructured":"Gill, J., and Moore, M.J. (2022, November 03). The State of Aging and Health in America 2013, Available online: https:\/\/www.cdc.gov\/aging."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1016\/j.apmr.2007.11.005","article-title":"Estimating the Prevalence of Limb Loss in the United States: 2005 to 2050","volume":"89","author":"MacKenzie","year":"2008","journal-title":"Arch. Phys. Med. Rehabil."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1111\/j.1532-5415.1977.tb00671.x","article-title":"Physical Fitness and Age, with Emphasis on Cardiovascular Function in the Elderly","volume":"25","author":"Hodgson","year":"1977","journal-title":"J. Am. Geriatr. Soc."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"6","DOI":"10.3109\/03790799009166594","article-title":"Mobility after stroke: Reliability of measures of impairment and disability","volume":"12","author":"Collen","year":"1990","journal-title":"Int. Disabil. Stud."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/MRA.2014.2360303","article-title":"A Robotic Leg Prosthesis: Design, Control, and Implementation","volume":"21","author":"Lawson","year":"2014","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_6","unstructured":"Lee, H., Ferguson, P.W., and Rosen, J. (2019). Wearable Robotics: Systems and Applications, Academic Press."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1109\/TBME.2009.2034734","article-title":"Multiclass Real-Time Intent Recognition of a Powered Lower Limb Prosthesis","volume":"57","author":"Varol","year":"2010","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1109\/TBME.2008.2003293","article-title":"A Strategy for Identifying Locomotion Modes Using Surface Electromyography","volume":"56","author":"Huang","year":"2009","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"562061","DOI":"10.3389\/frobt.2020.562061","article-title":"ExoNet Database: Wearable Camera Images of Human Locomotion Environments","volume":"7","author":"Laschowski","year":"2020","journal-title":"Front. Robot. AI"},{"key":"ref_10","first-page":"868","article-title":"Preliminary Design of an Environment Recognition System for Controlling Robotic Lower-Limb Prostheses and Exoskeletons","volume":"2019","author":"Laschowski","year":"2019","journal-title":"IEEE Int. Conf. Rehabil. Robot."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Jang, J., Kim, K., Lee, J., Lim, B., and Shim, Y. (2016, January 9\u201314). Assistance strategy for stair ascent with a robotic hip exoskeleton. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759832"},{"key":"ref_12","unstructured":"Maugliani, N., Caimmi, M., Malosio, M., Airoldi, F., Borro, D., Rosquete, D., Sergio, A., Giusino, D., Fraboni, F., and Ranieri, G. (2022). Wearable Robotics: Challenges and Trends: Proceedings of the 5th International Symposium on Wearable Robotics, WeRob2020, and of WearRAcon Europe 2020, Online, 13\u201316 October 2020, Springer International Publishing."},{"key":"ref_13","unstructured":"(2022, August 31). Ultralytics YOLOv5. Available online: https:\/\/docs.ultralytics.com\/yolov5."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tricomi, E., Mossini, M., Missiroli, F., Lotti, N., Xiloyannis, M., Roveda, L., and Masia, L. (2022). Environment-based Assistance Modulation for a Hip Exosuit via Computer Vision. arXiv.","DOI":"10.1109\/LRA.2023.3256135"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Lee, K., Kalyanram, V., Zhengl, C., Sane, S., and Lee, K. (2022, January 23\u201327). Vision-based Ascending Staircase Detection with Interpretable Classification Model for Stair Climbing Robots. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9812456"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1743-0003-12-1","article-title":"Control strategies for active lower extremity prosthetics and orthotics: A review","volume":"12","author":"Tucker","year":"2015","journal-title":"J. Neuroeng. Rehabil."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1109\/TNSRE.2016.2521160","article-title":"State of the art and future directions for lower limb robotic exoskeletons","volume":"25","author":"Young","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"763","DOI":"10.1109\/TBME.2017.2718528","article-title":"Translational Motion Tracking of Leg Joints for Enhanced Prediction of Walking Tasks","volume":"65","author":"Stolyarov","year":"2018","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, M., Zhang, F., and Huang, H.H. (2017). An Adaptive Classification Strategy for Reliable Locomotion Mode Recognition. Sensors, 17.","DOI":"10.3390\/s17092020"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1109\/TMRB.2019.2952148","article-title":"Deep Generative Models with Data Augmentation to Learn Robust Representations of Movement Intention for Powered Leg Prostheses","volume":"1","author":"Hu","year":"2019","journal-title":"IEEE Trans. Med. Robot. Bionics"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1109\/TNSRE.2015.2420539","article-title":"Development of an Environment-Aware Locomotion Mode Recognition System for Powered Lower Limb Prostheses","volume":"24","author":"Liu","year":"2016","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Laschowski, B., McNally, W., Wong, A., and McPhee, J. (2021, January 1\u20135). Computer Vision and Deep Learning for Environment-Adaptive Control of Robotic Lower-Limb Exoskeletons. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, Mexico.","DOI":"10.1101\/2021.04.02.438126"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hesch, J.A., Mariottini, G.L., and Roumeliotis, S.I. (2010, January 18\u201322). Descending-stair detection, approach, and traversal with an autonomous tracked vehicle. Proceedings of the IEEE\/RSJ 2010 International Conference on Intelligent Robots and Systems, IROS 2010\u2014Conference Proceedings, Taipei, Taiwan.","DOI":"10.1109\/IROS.2010.5649411"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Carbonara, S., and Guaragnella, C. (2014, January 23\u201325). Efficient stairs detection algorithm Assisted navigation for vision impaired people. Proceedings of the 2014 IEEE International Symposium on Innovations in Intelligent Systems and Applications (INISTA) Proceedings, Alberobello, Italy.","DOI":"10.1109\/INISTA.2014.6873637"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Cong, Y., Li, X., Liu, J., and Tang, Y. (2008, January 6\u20138). A stairway detection algorithm based on vision for UGV stair climbing. Proceedings of the 2008 IEEE International Conference on Networking, Sensing and Control, ICNSC, Sanya, China.","DOI":"10.1109\/ICNSC.2008.4525517"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Murakami, S., Shimakawa, M., Kivota, K., and Kato, T. (2014, January 3\u20136). Study on stairs detection using RGB-depth images. Proceedings of the 2014 Joint 7th International Conference on Soft Computing and Intelligent Systems, SCIS 2014 and 15th International Symposium on Advanced Intelligent Systems, ISIS 2014, Kitakyushu, Japan.","DOI":"10.1109\/SCIS-ISIS.2014.7044705"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1463","DOI":"10.1177\/0278364918798039","article-title":"Robust stairway-detection and localization method for mobile robots using a graph-based model and competing initializations","volume":"37","author":"Westfechtel","year":"2018","journal-title":"Int. J. Robot. Res."},{"key":"ref_28","first-page":"522","article-title":"Staircase detection using a lightweight look-behind fully convolutional neural network","volume":"Volume 1000","author":"Diamantis","year":"2019","journal-title":"Engineering Applications of Neural Networks, Proceedings of the 20th International Conference, EANN 2019, Xersonisos, Crete, Greece, 24\u201326 May 2019"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_30","unstructured":"Jocher, G., and Borovec, J. (2023, March 17). ultralytics\/yolov5. Available online: https:\/\/github.com\/ultralytics\/yolov5\/blob\/b94b59e199047aa8bf2cdd4401ae9f5f42b929e6\/data\/hyps\/hyp.scratch-low.yaml#L6-L34."},{"key":"ref_31","unstructured":"(2022, November 03). Training the YOLOv5 Object Detector on a Custom Dataset\u2014PyImageSearch. Available online: https:\/\/pyimagesearch.com\/2022\/06\/20\/training-the-yolov5-object-detector-on-a-custom-dataset\/."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2776","DOI":"10.3390\/s140202776","article-title":"Online Phase Detection Using Wearable Sensors for Walking with a Robotic Prosthesis","volume":"14","author":"Kamnik","year":"2014","journal-title":"Sensors"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1500","DOI":"10.1109\/TNSRE.2016.2636367","article-title":"A Real-Time Gait Event Detection for Lower Limb Prosthesis Control and Evaluation","volume":"25","author":"Maqbool","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"eaap8373","DOI":"10.1126\/scitranslmed.aap8373","article-title":"Proprioception from a neurally controlled lower-extremity prosthesis","volume":"10","author":"Clites","year":"2018","journal-title":"Sci. Transl. Med."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Doulah, A., Shen, X., and Sazonov, E. (2017). Early Detection of the Initiation of Sit-to-Stand Posture Transitions Using Orthosis-Mounted Sensors. Sensors, 17.","DOI":"10.3390\/s17122712"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1109\/TNSRE.2019.2895221","article-title":"Environmental Features Recognition for Lower Limb Prostheses Toward Predictive Walking","volume":"27","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"800","DOI":"10.1016\/j.phpro.2012.03.160","article-title":"AdaBoost for Feature Selection, Classification and Its Relation with SVM, A Review","volume":"25","author":"Wang","year":"2012","journal-title":"Phys. Procedia"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"21","DOI":"10.3389\/fnbot.2013.00021","article-title":"Gradient boosting machines, a tutorial","volume":"7","author":"Natekin","year":"2013","journal-title":"Front. Neurorobot."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"458","DOI":"10.1109\/TASE.2020.2993399","article-title":"Environmental Context Prediction for Lower Limb Prostheses With Uncertainty Quantification","volume":"18","author":"Zhong","year":"2021","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_40","unstructured":"(2022, September 08). Train Custom Data YOLOv5 Documentation. Available online: https:\/\/github.com\/ultralytics\/yolov5\/wiki\/Train-Custom-Data."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"120288","DOI":"10.1016\/j.eswa.2023.120288","article-title":"Deep learning-based approaches for human motion decoding in smart walkers for rehabilitation","volume":"228","author":"Lopes","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_42","unstructured":"Darafsh, S., Ghidary, S.S., and Zamani, M.S. (2021). Real-Time Activity Recognition and Intention Recognition Using a Vision-based Embedded System. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Shafiee, M.J., Chywl, B., Li, F., and Wong, A. (2017). Fast YOLO: A fast you only look once system for real-time embedded object detection in video. arXiv.","DOI":"10.15353\/vsnl.v3i1.171"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Wang, Z., Zhang, J., Zhao, Z., and Su, F. (2020, January 6\u201310). Efficient yolo: A lightweight model for embedded deep learning object detection. Proceedings of the 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), London, UK.","DOI":"10.1109\/ICMEW46912.2020.9105997"},{"key":"ref_45","unstructured":"Johnston, J. (2023, February 25). Tutorial: Running Yolov5 Machine Learning Detection on a Raspberry Pi 4. Medium, 8 April 2021. Available online: https:\/\/jordan-johnston271.medium.com\/tutorial-running-yolov5-machine-learning-detection-on-a-raspberry-pi-4-3938add0f719."},{"key":"ref_46","unstructured":"Virahonda, S. (2023, February 24). Deploying YOLOv5 Model on Raspberry Pi with Coral USB Accelerator. Available online: https:\/\/www.codeproject.com\/Articles\/5293079\/Deploying-YOLOv5-Model-on-Raspberry-Pi-with-Coral."},{"key":"ref_47","unstructured":"Heydarian, A. (2023, February 24). Yolov5 Object Detection on NVIDIA Jetson Nano. Available online: https:\/\/towardsdatascience.com\/yolov5-object-detection-on-nvidia-jetson-nano-148cfa21a024."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/11\/5355\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:48:46Z","timestamp":1760125726000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/11\/5355"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,5]]},"references-count":47,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["s23115355"],"URL":"https:\/\/doi.org\/10.3390\/s23115355","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,5]]}}}