{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,18]],"date-time":"2025-12-18T14:17:12Z","timestamp":1766067432667,"version":"build-2065373602"},"reference-count":52,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2021,11,30]],"date-time":"2021-11-30T00:00:00Z","timestamp":1638230400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Powered wheelchairs have enhanced the mobility and quality of life of people with special needs. The next step in the development of powered wheelchairs is to incorporate sensors and electronic systems for new control applications and capabilities to improve their usability and the safety of their operation, such as obstacle avoidance or autonomous driving. However, autonomous powered wheelchairs require safe navigation in different environments and scenarios, making their development complex. In our research, we propose, instead, to develop contactless control for powered wheelchairs where the position of the caregiver is used as a control reference. Hence, we used a depth camera to recognize the caregiver and measure at the same time their relative distance from the powered wheelchair. In this paper, we compared two different approaches for real-time object recognition using a 3DHOG hand-crafted object descriptor based on a 3D extension of the histogram of oriented gradients (HOG) and a convolutional neural network based on YOLOv4-Tiny. To evaluate both approaches, we constructed Miun-Feet\u2014a custom dataset of images of labeled caregiver\u2019s feet in different scenarios, with backgrounds, objects, and lighting conditions. The experimental results showed that the YOLOv4-Tiny approach outperformed 3DHOG in all the analyzed cases. In addition, the results showed that the recognition accuracy was not improved using the depth channel, enabling the use of a monocular RGB camera only instead of a depth camera and reducing the computational cost and heat dissipation limitations. Hence, the paper proposes an additional method to compute the caregiver\u2019s distance and angle from the Powered Wheelchair (PW) using only the RGB data. This work shows that it is feasible to use the location of the caregiver\u2019s feet as a control signal for the control of a powered wheelchair and that it is possible to use a monocular RGB camera to compute their relative positions.<\/jats:p>","DOI":"10.3390\/jimaging7120255","type":"journal-article","created":{"date-parts":[[2021,11,30]],"date-time":"2021-11-30T22:01:11Z","timestamp":1638309671000},"page":"255","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Evaluation of 2D-\/3D-Feet-Detection Methods for Semi-Autonomous Powered Wheelchair Navigation"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1167-8322","authenticated-orcid":false,"given":"Cristian Vilar","family":"Gim\u00e9nez","sequence":"first","affiliation":[{"name":"Department of Electronics Design, Mid Sweden University, Holmgatan 10, 851 70 Sundsvall, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0282-5471","authenticated-orcid":false,"given":"Silvia","family":"Krug","sequence":"additional","affiliation":[{"name":"Department of Electronics Design, Mid Sweden University, Holmgatan 10, 851 70 Sundsvall, Sweden"},{"name":"System Design Department, IMMS Institut f\u00fcr Mikroelektronik- und Mechatronik-Systeme Gemeinn\u00fctzige GmbH (IMMS GmbH), Ehrenbergstra\u00dfe 27, 98693 Ilmenau, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8992-3607","authenticated-orcid":false,"given":"Faisal Z.","family":"Qureshi","sequence":"additional","affiliation":[{"name":"Department of Electronics Design, Mid Sweden University, Holmgatan 10, 851 70 Sundsvall, Sweden"},{"name":"Faculty of Science, University of Ontario Institute of Technology, 2000 Simcoe St. N., Oshawa, ON L1G OC5, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8607-4083","authenticated-orcid":false,"given":"Mattias","family":"O\u2019Nils","sequence":"additional","affiliation":[{"name":"Department of Electronics Design, Mid Sweden University, Holmgatan 10, 851 70 Sundsvall, Sweden"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"419","DOI":"10.4236\/ojn.2018.87033","article-title":"Wanting a Life in Decency!\u2014A Qualitative Study from Experienced Electric Wheelchairs Users\u2019 perspective","volume":"8","author":"Kristiansen","year":"2018","journal-title":"Open J. Nurs."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Vilar, C., Th\u00f6rnberg, B., and Krug, S. (2019, January 3\u20135). Evaluation of Embedded Camera Systems for Autonomous Wheelchairs. Proceedings of the 5th International Conference on Vehicle Technology and Intelligent Transport Systems (VEHITS), Crete, Greece.","DOI":"10.5220\/0007678700760085"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Dupre, R., and Argyriou, V. (2015, January 21\u201324). 3D Voxel HOG and Risk Estimation. Proceedings of the International Conference on Digital Signal Processing (DSP), Singapore.","DOI":"10.1109\/ICDSP.2015.7251919"},{"key":"ref_4","first-page":"13","article-title":"Processing chain for 3D histogram of gradients based real-time object recognition","volume":"18","author":"Vilar","year":"2020","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Vilar, C., Krug, S., and O\u2019Nils, M. (2021). Realworld 3d object recognition using a 3d extension of the hog descriptor and a depth camera. Sensors, 21.","DOI":"10.3390\/s21030910"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Xiong, M., Hotter, R., Nadin, D., Patel, J., Tartakovsky, S., Wang, Y., Patel, H., Axon, C., Bosiljevac, H., and Brandenberger, A. (2019, January 6\u20139). A Low-Cost, Semi-Autonomous Wheelchair Controlled by Motor Imagery and Jaw Muscle Activation. Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy.","DOI":"10.1109\/SMC.2019.8914544"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kader, M.A., Alam, M.E., Jahan, N., Bhuiyan, M.A.B., Alam, M.S., and Sultana, Z. (2019, January 18\u201320). Design and implementation of a head motion-controlled semi-autonomous wheelchair for quadriplegic patients based on 3-axis accelerometer. Proceedings of the 2019 22nd International Conference on Computer and Information Technology (ICCIT), Dhaka, Bangladesh.","DOI":"10.1109\/ICCIT48885.2019.9038512"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Subramanian, M., Songur, N., Adjei, D., Orlov, P., and Faisal, A.A. (2019, January 23\u201327). A.Eye Drive: Gaze-based semi-autonomous wheelchair interface. Proceedings of the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, Germany.","DOI":"10.1109\/EMBC.2019.8856608"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Grewal, H.S., Jayaprakash, N.T., Matthews, A., Shrivastav, C., and George, K. (2018, January 14\u201317). Autonomous wheelchair navigation in unmapped indoor environments. Proceedings of the 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Houston, TX, USA.","DOI":"10.1109\/I2MTC.2018.8409854"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Grewal, H., Matthews, A., Tea, R., and George, K. (2017, January 13\u201315). LiDAR-based autonomous wheelchair. Proceedings of the 2017 IEEE Sensors Applications Symposium (SAS), Glassboro, NJ, USA.","DOI":"10.1109\/SAS.2017.7894082"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Li, Z., Xiong, Y., and Zhou, L. (2017, January 9\u201310). ROS-Based Indoor Autonomous Exploration and Navigation Wheelchair. Proceedings of the 2017 10th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China.","DOI":"10.1109\/ISCID.2017.55"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kobayashi, Y., Suzuki, R., and Kuno, Y. (2012, January 25\u201328). Robotic wheelchair with omni-directional vision for moving alongside a caregiver. Proceedings of the IECON 2012\u201438th Annual Conference on IEEE Industrial Electronics Society, Montreal, QC, Canada.","DOI":"10.1109\/IECON.2012.6389220"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kobayashi, T., Chugo, D., Yokota, S., Muramatsu, S., and Hashimoto, H. (2015, January 19\u201321). Design of personal mobility motion based on cooperative movement with a companion. Proceedings of the 2015 6th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), Gyor, Hungary.","DOI":"10.1109\/CogInfoCom.2015.7390584"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Motokucho, T., and Oda, N. (2014, January 14\u201316). Vision-based human-following control using optical flow field for power assisted wheelchair. Proceedings of the 2014 IEEE 13th International Workshop on Advanced Motion Control (AMC), Yokohama, Japan.","DOI":"10.1109\/AMC.2014.6823293"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1007\/s11554-020-01033-3","article-title":"A cylindrical shape descriptor for registration of unstructured point clouds from real-time 3D sensors","volume":"18","author":"He","year":"2020","journal-title":"J. Real-Time Image Process."},{"key":"ref_16","unstructured":"Wu, Z., and Song, S. (2015, January 7\u201312). 3D ShapeNets: A Deep Representation for Volumetric Shapes. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA."},{"key":"ref_17","first-page":"1","article-title":"Autonomous navigation for a holonomic drive robot in an unknown environment using a depth camera","volume":"11509","author":"Aleman","year":"2020","journal-title":"Opt. Photonics Inf. Process. XIV"},{"key":"ref_18","unstructured":"Tabib, W., Goel, K., Yao, J., Boirum, C., and Michael, N. (2020). Autonomous Cave Surveying with an Aerial Robot. IEEE Trans. Robot., 1\u201317."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1177\/0278364911436019","article-title":"Rigid 3D geometry matching for grasping of known objects in cluttered scenes","volume":"31","author":"Papazov","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Guo, Y., Bennamoun, M., Sohel, F., Lu, M., Wan, J., and Kwok, N. (2015). A Comprehensive Performance Evaluation of 3D Local Feature Descriptors. Int. J. Comput. Vis., 116.","DOI":"10.1007\/s11263-015-0824-y"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Bayramoglu, N., and Alatan, A.A. (2010, January 23\u201326). Shape index SIFT: Range image recognition using local features. Proceedings of the International Conference on Pattern Recognition (ICPR), Istanbul, Turkey.","DOI":"10.1109\/ICPR.2010.95"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1833","DOI":"10.1109\/ACCESS.2017.2658681","article-title":"3D Object Recognition in Cluttered Scenes with Robust Shape Description and Correspondence Selection","volume":"5","author":"Tang","year":"2017","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.cviu.2014.04.011","article-title":"SHOT: Unique signatures of histograms for surface and texture description q","volume":"125","author":"Salti","year":"2014","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"714","DOI":"10.1109\/TCSVT.2018.2813083","article-title":"Aligning 2.5D Scene Fragments with Distinctive Local Geometric Features","volume":"29","author":"Yang","year":"2019","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1109\/34.765655","article-title":"Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes","volume":"21","author":"Johnson","year":"1999","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Rusu, R.B., Bradski, G., Thibaux, R., and Hsu, J. (2010, January 18\u201322). Fast 3D recognition and pose using the viewpoint feature histogram. Proceedings of the IEEE\/RSJ 2010 International Conference on Intelligent Robots and Systems (IROS), Taipei, Taiwan.","DOI":"10.1109\/IROS.2010.5651280"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Rusu, R.B., Blodow, N., and Beetz, M. (2009, January 12\u201317). Fast Point Feature Histograms (FPFH) for 3D registration. Proceedings of the International Conference on Robotics and Automation (ICRA), Kobe, Japan.","DOI":"10.1109\/ROBOT.2009.5152473"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wohlkinger, W., and Vincze, M. (2011, January 7\u201311). Ensemble of shape functions for 3D object classification. Proceedings of the International Conference on Robotics and Biomimetics (ROBIO), Karon Beach, Thailand.","DOI":"10.1109\/ROBIO.2011.6181760"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"161958","DOI":"10.1109\/ACCESS.2020.3021455","article-title":"A Voxelized Fractal Descriptor for 3D Object Recognition","volume":"8","author":"Domenech","year":"2020","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2020.2990920","article-title":"A Pipeline for 3-D Object Recognition Based on Local Shape Description in Cluttered Scenes","volume":"59","author":"Tao","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhou, Y., and Tuzel, O. (2017). VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection. arXiv.","DOI":"10.1109\/CVPR.2018.00472"},{"key":"ref_32","unstructured":"Maturana, D., and Scherer, S. (October, January 28). VoxNet: A 3D Convolutional Neural Network for Real-Time Object Recognition. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany."},{"key":"ref_33","unstructured":"Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017, January 21\u201326). PointNet: Deep learning on point sets for 3D classification and segmentation. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Yang, Y., Feng, C., Shen, Y., and Tian, D. (2018, January 18\u201323). FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00029"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Kanezaki, A., Matsushita, Y., and Nishida, Y. (2018, January 18\u201323). RotationNet: Joint Object Categorization and Pose Estimation Using Multiviews from Unsupervised Viewpoints. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00526"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"984","DOI":"10.1016\/j.ins.2020.09.057","article-title":"Hierarchical multi-view context modelling for 3D object classification and retrieval","volume":"547","author":"Liu","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Simon, M., Milz, S., Amende, K., and Gross, H.M. (2018). Complex-YOLO: Real-time 3D Object Detection on Point Clouds. arXiv.","DOI":"10.1109\/CVPRW.2019.00158"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"102053","DOI":"10.1016\/j.displa.2021.102053","article-title":"Review of multi-view 3D object recognition methods based on deep learning","volume":"69","author":"Qi","year":"2021","journal-title":"Displays"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Takahashi, M., Moro, A., Ji, Y., and Umeda, K. (2020). Expandable YOLO: 3D Object Detection from RGB-D Images. CoRR, abs\/2006.14837.","DOI":"10.1109\/REM49740.2020.9313886"},{"key":"ref_40","unstructured":"Couprie, C., Farabet, C., Najman, L., and LeCun, Y. (2013). Indoor Semantic Segmentation using depth information. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Rahman, M.M., Tan, Y., Xue, J., and Lu, K. (2017, January 10\u201314). RGB-D object recognition with multimodal deep convolutional neural networks. Proceedings of the 2017 IEEE International Conference on Multimedia and Expo (ICME), Hong Kong, China.","DOI":"10.1109\/ICME.2017.8019538"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"50","DOI":"10.5898\/JHRI.3.2.Morales","article-title":"Walking Together: Side-by-Side Walking Model for an Interacting Robot","volume":"3","author":"Morales","year":"2014","journal-title":"J. Hum.-Robot Interact."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Kobayashi, T., Chugo, D., Yokota, S., Muramatsu, S., and Hashimoto, H. (2016, January 14\u201317). A driving assistance for a powered wheelchair with a walking partner. Proceedings of the 2016 IEEE International Conference on Industrial Technology (ICIT), Taipei, Taiwan.","DOI":"10.1109\/ICIT.2016.7475050"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Udsatid, P., Niparnan, N., and Sudsang, A. (2012, January 11\u201314). Human position tracking for side by side walking mobile robot using foot positions. Proceedings of the 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO), Guangzhou, China.","DOI":"10.1109\/ROBIO.2012.6491160"},{"key":"ref_45","unstructured":"(2021, November 29). \u00a9 Intel Corporation. Available online: https:\/\/www.intelrealsense.com\/depth-camera-d455."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1006\/cviu.1999.0832","article-title":"MLESAC: A new robust estimator with application to estimating image geometry","volume":"78","author":"Torr","year":"2000","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_47","unstructured":"Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"012034","DOI":"10.1088\/1742-6596\/1920\/1\/012034","article-title":"Vehicle and Pedestrian Detection Based on Improved YOLOv4-tiny Model","volume":"1920","author":"Ma","year":"2021","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_49","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of Oriented Gradients for Human Detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Vilar, C., Krug, S., and Thornberg, B. (2019). Rotational Invariant Object Recognition for Robotic Vision. 3rd International Conference on Automation, Control and Robots ( ICACR), ACM.","DOI":"10.1145\/3365265.3365273"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"012029","DOI":"10.1088\/1742-6596\/1911\/1\/012029","article-title":"Heavy-Vehicle Detection Based on YOLOv4 featuring Data Augmentation and Transfer-Learning Techniques","volume":"1911","author":"Sowmya","year":"2021","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Kang, B., Liu, Z., Wang, X., Yu, F., Feng, J., and Darrell, T. (2019). Few-shot Object Detection via Feature Reweighting. arXiv.","DOI":"10.1109\/ICCV.2019.00851"}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/7\/12\/255\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:37:56Z","timestamp":1760168276000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/7\/12\/255"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,30]]},"references-count":52,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["jimaging7120255"],"URL":"https:\/\/doi.org\/10.3390\/jimaging7120255","relation":{},"ISSN":["2313-433X"],"issn-type":[{"type":"electronic","value":"2313-433X"}],"subject":[],"published":{"date-parts":[[2021,11,30]]}}}