{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T00:12:44Z","timestamp":1789690364220,"version":"4.0.1"},"reference-count":45,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,12,24]],"date-time":"2020-12-24T00:00:00Z","timestamp":1608768000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJERPH"],"abstract":"<jats:p>This paper deals with the development of an Advanced Driver Assistance System (ADAS) for a smart electric wheelchair in order to improve the autonomy of disabled people. Our use case, built from a formal clinical study, is based on the detection, depth estimation, localization and tracking of objects in wheelchair\u2019s indoor environment, namely: door and door handles. The aim of this work is to provide a perception layer to the wheelchair, enabling this way the detection of these keypoints in its immediate surrounding, and constructing of a short lifespan semantic map. Firstly, we present an adaptation of the YOLOv3 object detection algorithm to our use case. Then, we present our depth estimation approach using an Intel RealSense camera. Finally, as a third and last step of our approach, we present our 3D object tracking approach based on the SORT algorithm. In order to validate all the developments, we have carried out different experiments in a controlled indoor environment. Detection, distance estimation and object tracking are experimented using our own dataset, which includes doors and door handles.<\/jats:p>","DOI":"10.3390\/ijerph18010091","type":"journal-article","created":{"date-parts":[[2020,12,24]],"date-time":"2020-12-24T22:56:45Z","timestamp":1608850605000},"page":"91","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":48,"title":["Deep Learning-Based Object Detection, Localisation and Tracking for Smart Wheelchair Healthcare Mobility"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8570-3781","authenticated-orcid":false,"given":"Louis","family":"Lecrosnier","sequence":"first","affiliation":[{"name":"\u00c9cole Sup\u00e9rieure d\u2019Ing\u00e9nieurs en G\u00e9nie \u00c9lectrique, 76800 Saint-\u00c9tienne-du-Rouvay, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6230-2966","authenticated-orcid":false,"given":"Redouane","family":"Khemmar","sequence":"additional","affiliation":[{"name":"\u00c9cole Sup\u00e9rieure d\u2019Ing\u00e9nieurs en G\u00e9nie \u00c9lectrique, 76800 Saint-\u00c9tienne-du-Rouvay, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5360-2185","authenticated-orcid":false,"given":"Nicolas","family":"Ragot","sequence":"additional","affiliation":[{"name":"\u00c9cole Sup\u00e9rieure d\u2019Ing\u00e9nieurs en G\u00e9nie \u00c9lectrique, 76800 Saint-\u00c9tienne-du-Rouvay, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4037-2880","authenticated-orcid":false,"given":"Benoit","family":"Decoux","sequence":"additional","affiliation":[{"name":"\u00c9cole Sup\u00e9rieure d\u2019Ing\u00e9nieurs en G\u00e9nie \u00c9lectrique, 76800 Saint-\u00c9tienne-du-Rouvay, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5130-4798","authenticated-orcid":false,"given":"Romain","family":"Rossi","sequence":"additional","affiliation":[{"name":"\u00c9cole Sup\u00e9rieure d\u2019Ing\u00e9nieurs en G\u00e9nie \u00c9lectrique, 76800 Saint-\u00c9tienne-du-Rouvay, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Naceur","family":"Kefi","sequence":"additional","affiliation":[{"name":"SUP\u2019COM: \u00c9cole Sup\u00e9rieure des Communications de Tunis, Carthage University, Aryanah 2080, Tunis"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jean-Yves","family":"Ertaud","sequence":"additional","affiliation":[{"name":"\u00c9cole Sup\u00e9rieure d\u2019Ing\u00e9nieurs en G\u00e9nie \u00c9lectrique, 76800 Saint-\u00c9tienne-du-Rouvay, France"},{"name":"SUP\u2019COM: \u00c9cole Sup\u00e9rieure des Communications de Tunis, Carthage University, Aryanah 2080, Tunis"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Mauri, A., Khemmar, R., Decoux, B., Ragot, N., Rossi, R., Trabelsi, R., Boutteau, R., Ertaud, J.Y., and Savatier, X. (2020). Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility. Sensors, 20.","DOI":"10.3390\/s20020532"},{"key":"ref_2","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). SSD: Single Shot MultiBox Detector. Lect. Notes Comput. Sci., 21\u201337.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_4","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Dollar, P., and Girshick, R. (2017, January 22\u201329). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhao, C., Sun, Q., Zhang, C., Tang, Y., and Qian, F. (2020). Monocular Depth Estimation Based on Deep Learning: An Overview, Science China Technological Sciences.","DOI":"10.1007\/s11431-020-1582-8"},{"key":"ref_9","unstructured":"Laga, H., Jospin, L.V., Boussaid, F., and Bennamoun, M. (2020). A Survey on Deep Learning Techniques for Stereo-based Depth Estimation. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"Segnet: A deep convolutional encoder-decoder architecture for image segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Godard, C., Mac Aodha, O., and Brostow, G.J. (2017, January 21\u201326). Unsupervised monocular depth estimation with left-right consistency. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.699"},{"key":"ref_13","unstructured":"Lee, J.H., Han, M.K., Ko, D.W., and Suh, I.H. (2019). From big to small: Multi-scale local planar guidance for monocular depth estimation. arXiv."},{"key":"ref_14","unstructured":"Alhashim, I., and Wonka, P. (2018). High quality monocular depth estimation via transfer learning. arXiv."},{"key":"ref_15","unstructured":"Huang, G., Li, Y., Pleiss, G., Liu, Z., Hopcroft, J.E., and Weinberger, K.Q. (2017). Snapshot ensembles: Train 1, get m for free. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Sato, R., and Kimura, A. (2020, January 5\u20137). A simple refinement for depth information predicted with DNN. Proceedings of the International Workshop on Advanced Imaging Technology (IWAIT) 2020, International Society for Optics and Photonics, Yogyakarta, Indonesia.","DOI":"10.1117\/12.2566610"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chen, Z., Khemmar, R., Decoux, B., Atahouet, A., and Ertaud, J.Y. (2019, January 14\u201322). Real Time Object Detection, Tracking, and Distance and Motion Estimation based on Deep Learning: Application to Smart Mobility. Proceedings of the 2019 Eighth International Conference on Emerging Security Technologies (EST), Essex, UK.","DOI":"10.1109\/EST.2019.8806222"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wu, Y., Lim, J., and Yang, M.H. (2015). Object Tracking Benchmark. IEEE Trans. Pattern Anal. Mach. Intell., 37.","DOI":"10.1109\/TPAMI.2014.2388226"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Held, D., Thrun, S., and Savarese, S. (2016). Learning to Track at 100 FPS with Deep Regression Networks. Lect. Notes Comput. Sci., 749\u2013765.","DOI":"10.1007\/978-3-319-46448-0_45"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. (2016, January 25\u201328). Simple online and realtime tracking. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7533003"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wojke, N., Bewley, A., and Paulus, D. (2017). Simple Online and Realtime Tracking with a Deep Association Metric. arXiv.","DOI":"10.1109\/ICIP.2017.8296962"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ning, G., Zhang, Z., Huang, C., Ren, X., Wang, H., Cai, C., and He, Z. (2017, January 28\u201331). Spatially supervised recurrent convolutional neural networks for visual object tracking. Proceedings of the 2017 IEEE International Symposium on Circuits and Systems (ISCAS), Baltimore, MD, USA.","DOI":"10.1109\/ISCAS.2017.8050867"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, X., and Belaroussi, R. (2016). Semi-Dense 3D Semantic Mapping from Monocular SLAM. arXiv.","DOI":"10.1109\/ITSC.2017.8317942"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"McCormac, J., Handa, A., Davison, A., and Leutenegger, S. (June, January 29). SemanticFusion: Dense 3D semantic mapping with convolutional neural networks. Proceedings of the 2017 IEEE International Conference on Robotics and automation (ICRA), Marina Bay Sands, Singapore.","DOI":"10.1109\/ICRA.2017.7989538"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"3037","DOI":"10.1109\/LRA.2019.2923960","article-title":"Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery","volume":"4","author":"Grinvald","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1007\/s10846-019-01136-5","article-title":"Extending Maps with Semantic and Contextual Object Information for Robot Navigation: A Learning-Based Framework Using Visual and Depth Cues","volume":"99","author":"Martins","year":"2020","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Li, F.F. (2009, January 22\u201324). ImageNet: A Large-Scale Hierarchical Image Database. Proceedings of the 2009 IEEE Conference on Computer Vision and PATTERN recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Jeong, H., Park, K., and Ha, Y. (March, January 27). Image Preprocessing for Efficient Training of YOLO Deep Learning Networks. Proceedings of the 2018 IEEE International Conference on Big Data and Smart Computing (BigComp), Kyoto, Japan.","DOI":"10.1109\/BigComp.2018.00113"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1956","DOI":"10.1007\/s11263-020-01316-z","article-title":"The Open Images Dataset V4","volume":"128","author":"Kuznetsova","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Doyle, S., Feldman, M., Shih, N., Tomaszewski, J., and Madabhushi, A. (2012). Cascaded discrimination of normal, abnormal, and confounder classes in histopathology: Gleason grading of prostate cancer. BMC Bioinform., 13.","DOI":"10.1186\/1471-2105-13-282"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., and Zitnick, C.L. (2014). Microsoft COCO: Common Objects in Context. Lect. Notes Comput. Sci., 740\u2013755.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Caesar, H., Uijlings, J., and Ferrari, V. (2018, January 18\u201322). COCO-Stuff: Thing and Stuff Classes in Context. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00132"},{"key":"ref_34","unstructured":"Li, J., Raventos, A., Bhargava, A., Tagawa, T., and Gaidon, A. (2018). Learning to Fuse Things and Stuff. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The Pascal Visual Object Classes (VOC) Challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_36","unstructured":"Krizhevsky, A., Nair, V., and Hinton, G. (2020, November 10). Learning Multiple Layers of Features from Tiny Images, Cifar-10 and Cifar-100 Datasets. Available online: https:\/\/www.cs.toronto.edu\/kriz\/cifar.html."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Bashiri, F.S., LaRose, E., Peissig, P., and Tafti, A.P. (2018). MCIndoor20000: A fully-labeled image dataset to advance indoor objects detection. Data Brief, 17.","DOI":"10.1016\/j.dib.2017.12.047"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The KITTI dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., and Beijbom, O. (2019). nuscenes: A multimodal dataset for autonomous driving. arXiv.","DOI":"10.1109\/CVPR42600.2020.01164"},{"key":"ref_40","unstructured":"(2020, October 05). Robotic Operating System. Available online: https:\/\/www.ros.org\/."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Khemmar, R., Gouveia, M., Decoux, B., and Ertaud, J.Y. (2019, January 27\u201331). Real Time Pedestrian and Object Detection and Tracking-based Deep Learning. Application to Drone Visual Tracking. WSCG\u20192019-27. Proceedings of the International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision\u20192019, Plzen, Czech Republic.","DOI":"10.24132\/CSRN.2019.2902.2.5"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Merriaux, P., Dupuis, Y., Boutteau, R., Vasseur, P., and Savatier, X. (2017). A Study of Vicon System Positioning Performance. Sensors, 17.","DOI":"10.3390\/s17071591"},{"key":"ref_43","unstructured":"(2020, November 30). YOLOv5 New Version\u2014Improvements And Evaluation. Available online: https:\/\/blog.roboflow.com\/yolov5-improvements-and-evaluation\/."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Godard, C., Mac Aodha, O., Firman, M., and Brostow, G. (2018). Digging into self-supervised monocular depth estimation. arXiv.","DOI":"10.1109\/ICCV.2019.00393"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Tonioni, A., Tosi, F., Poggi, M., Mattoccia, S., and Stefano, L.D. (2019, January 16\u201320). Real-time self-adaptive deep stereo. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00028"}],"container-title":["International Journal of Environmental Research and Public Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/1\/91\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:45:49Z","timestamp":1760179549000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/1\/91"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,24]]},"references-count":45,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["ijerph18010091"],"URL":"https:\/\/doi.org\/10.3390\/ijerph18010091","relation":{},"ISSN":["1660-4601"],"issn-type":[{"value":"1660-4601","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,24]]}}}