{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T10:53:58Z","timestamp":1775645638281,"version":"3.50.1"},"reference-count":44,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2021,9,18]],"date-time":"2021-09-18T00:00:00Z","timestamp":1631923200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Robotics Programme under its Robotics Enabling Capabilities and Technologies- Agency for Science, Technology and Research","award":["Funding Agency Project No. 192~25~00051, 192~22~00058"],"award-info":[{"award-number":["Funding Agency Project No. 192~25~00051, 192~22~00058"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Staircase cleaning is a crucial and time-consuming task for maintenance of multistory apartments and commercial buildings. There are many commercially available autonomous cleaning robots in the market for building maintenance, but few of them are designed for staircase cleaning. A key challenge for automating staircase cleaning robots involves the design of Environmental Perception Systems (EPS), which assist the robot in determining and navigating staircases. This system also recognizes obstacles and debris for safe navigation and efficient cleaning while climbing the staircase. This work proposes an operational framework leveraging the vision based EPS for the modular re-configurable maintenance robot, called sTetro. The proposed system uses an SSD MobileNet real-time object detection model to recognize staircases, obstacles and debris. Furthermore, the model filters out false detection of staircases by fusion of depth information through the use of a MobileNet and SVM. The system uses a contour detection algorithm to localize the first step of the staircase and depth clustering scheme for obstacle and debris localization. The framework has been deployed on the sTetro robot using the Jetson Nano hardware from NVIDIA and tested with multistory staircases. The experimental results show that the entire framework takes an average of 310 ms to run and achieves an accuracy of 94.32% for staircase recognition tasks and 93.81% accuracy for obstacle and debris detection tasks during real operation of the robot.<\/jats:p>","DOI":"10.3390\/s21186279","type":"journal-article","created":{"date-parts":[[2021,9,21]],"date-time":"2021-09-21T22:35:20Z","timestamp":1632263720000},"page":"6279","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["sTetro-Deep Learning Powered Staircase Cleaning and Maintenance Reconfigurable Robot"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3243-9814","authenticated-orcid":false,"given":"Balakrishnan","family":"Ramalingam","sequence":"first","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6504-1530","authenticated-orcid":false,"given":"Rajesh","family":"Elara Mohan","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"given":"Selvasundari","family":"Balakrishnan","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3407-5495","authenticated-orcid":false,"given":"Karthikeyan","family":"Elangovan","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"given":"Braulio","family":"F\u00e9lix G\u00f3mez","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4010-570X","authenticated-orcid":false,"given":"Thejus","family":"Pathmakumar","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"given":"Manojkumar","family":"Devarassu","sequence":"additional","affiliation":[{"name":"Engineering Product Development Pillar, Singapore University of Technology and Design (SUTD), Singapore 487372, Singapore"}]},{"given":"Madan","family":"Mohan Rayaguru","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Delhi Technological University, Delhi 110042, India"}]},{"given":"Chanthini","family":"Baskar","sequence":"additional","affiliation":[{"name":"School of Electronics, Vellore Institute of Technology Chennai, Vellore 600127, India"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chang, C.L., Chang, C.Y., Tang, Z.Y., and Chen, S.T. (2018). High-Efficiency Automatic Recharging Mechanism for Cleaning Robot Using Multi-Sensor. Sensors, 18.","DOI":"10.3390\/s18113911"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ramalingam, B., Yin, J., Rajesh Elara, M., Tamilselvam, Y.K., Mohan Rayguru, M., Muthugala, M.A.V.J., and F\u00e9lix G\u00f3mez, B. (2020). A Human Support Robot for the Cleaning and Maintenance of Door Handles Using a Deep-Learning Framework. Sensors, 20.","DOI":"10.3390\/s20123543"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Yuyao, S., Elara, M.R., Kalimuthu, M., and Devarassu, M. (2018, January 20\u201322). sTetro: A Modular Reconfigurable Cleaning Robot. Proceedings of the 2018 International Conference on Reconfigurable Mechanisms and Robots (ReMAR), Delft, The Netherlands.","DOI":"10.1109\/REMAR.2018.8449883"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.robot.2009.07.020","article-title":"Optimized obstacle avoidance trajectory generation for a reconfigurable staircase climbing wheelchair","volume":"58","author":"Morales","year":"2010","journal-title":"Robot. Auton. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"944","DOI":"10.1109\/JSYST.2014.2309477","article-title":"Trajectory planning for a stair-climbing mobility system using laser distance sensors","volume":"10","author":"Chocoteco","year":"2014","journal-title":"IEEE Syst. J."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yanagida, T., Elara Mohan, R., Pathmakumar, T., Elangovan, K., and Iwase, M. (2017). Design and Implementation of a Shape Shifting Rolling\u2013Crawling\u2013Wall-Climbing Robot. Appl. Sci., 7.","DOI":"10.3390\/app7040342"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Mihankhah, E., Kalantari, A., Aboosaeedan, E., Taghirad, H.D., Ali, S., and Moosavian, A. (2009, January 22\u201325). Autonomous staircase detection and stair climbing for a tracked mobile robot using fuzzy controller. Proceedings of the 2008 IEEE International Conference on Robotics and Biomimetics, Bangkok, Thailand.","DOI":"10.1109\/ROBIO.2009.4913304"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Eich, M., Grimminger, F., and Kirchner, F. (2009, January 18\u201322). A Versatile Stair-Climbing Robot for Search and Rescue Applications. Proceedings of the 2008 IEEE International Workshop on Safety, Security and Rescue Robotics, Guilin, China.","DOI":"10.1109\/SSRR.2008.4745874"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Khandelwal, K., Patel, R., Shenoy, A., Farooquee, S., and George, G. (2015, January 4\u20136). Application of stair climbing robot. Proceedings of the 2015 International Conference on Technologies for Sustainable Development (ICTSD), Mumbai, India.","DOI":"10.1109\/ICTSD.2015.7095884"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kakudou, T., Nagai, I., and Watanabe, K. (2010). A cleaning robot for stairs and the simulation of stair movement. Emerging Trends In Mobile Robotics, World Scientific.","DOI":"10.1142\/9789814329927_0159"},{"key":"ref_11","unstructured":"Kakudou, T., Watanabe, K., and Nagai, I. (2011, January 26\u201329). Study on mobile mechanism for a stair cleaning robot-Design of translational locomotion mechanism. Proceedings of the 2011 11th International Conference on Control, Automation and Systems, Gyeonggi-do, Korea."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Megalingam, R.K., Prem, A., Nair, A.H., Pillai, A.J., and Nair, B.S. (2016, January 6\u20138). Stair case cleaning robot: Design considerations and a case study. Proceedings of the 2016 International Conference on Communication and Signal Processing (ICCSP), Melmaruvathur, India.","DOI":"10.1109\/ICCSP.2016.7754247"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"8190802","DOI":"10.1155\/2018\/8190802","article-title":"Design of sTetro: A modular, reconfigurable, and autonomous staircase cleaning robot","volume":"2018","author":"Ilyas","year":"2018","journal-title":"J. Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1016\/j.jvcir.2013.11.005","article-title":"RGB-D image-based detection of stairs, pedestrian crosswalks and traffic signs","volume":"25","author":"Wang","year":"2014","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1109\/MRA.2012.2191995","article-title":"Perception, planning, and execution for mobile manipulation in unstructured environments","volume":"19","author":"Chitta","year":"2012","journal-title":"IEEE Robot. Autom. Mag. Spec. Issue Mob. Manip."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.autcon.2019.01.022","article-title":"Computer vision for real-time extrusion quality monitoring and control in robotic construction","volume":"101","author":"Kazemian","year":"2019","journal-title":"Autom. Constr."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1016\/j.autcon.2018.10.009","article-title":"Vision-based integrated mobile robotic system for real-time applications in construction","volume":"96","author":"Asadi","year":"2018","journal-title":"Autom. Constr."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Pendleton, S.D., Andersen, H., Du, X., Shen, X., Meghjani, M., Eng, Y.H., Rus, D., and Ang, M.H. (2017). Perception, Planning, Control, and Coordination for Autonomous Vehicles. Machines, 5.","DOI":"10.3390\/machines5010006"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Do, T., Duong, M., Dang, Q., and Le, M. (2018, January 23\u201324). Real-Time Self-Driving Car Navigation Using Deep Neural Network. Proceedings of the 2018 4th International Conference on Green Technology and Sustainable Development (GTSD), Ho Chi Minh City, Vietnam.","DOI":"10.1109\/GTSD.2018.8595590"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1016\/j.autcon.2018.11.009","article-title":"Construction waste recycling robot for nails and screws: Computer vision technology and neural network approach","volume":"97","author":"Wang","year":"2019","journal-title":"Autom. Constr."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ramalingam, B., Lakshmanan, A.K., Ilyas, M., Le, A.V., and Elara, M.R. (2018). Cascaded Machine-Learning Technique for Debris Classification in Floor-Cleaning Robot Application. Appl. Sci., 8.","DOI":"10.3390\/app8122649"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yin, J., Apuroop, K.G.S., Tamilselvam, Y.K., Mohan, R.E., Ramalingam, B., and Le, A.V. (2020). Table Cleaning Task by Human Support Robot Using Deep Learning Technique. Sensors, 20.","DOI":"10.3390\/s20061698"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"5137139","DOI":"10.1155\/2019\/5137139","article-title":"Visual Inspection of the Aircraft Surface Using a Teleoperated Reconfigurable Climbing Robot and Enhanced Deep Learning Technique","volume":"2019","author":"Ramalingam","year":"2019","journal-title":"Int. J. Aerosp. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ramalingam, B., Tun, T., Mohan, R.E., G\u00f3mez, B.F., Cheng, R., Balakrishnan, S., Mohan Rayaguru, M., and Hayat, A.A. (2021). AI Enabled IoRT Framework for Rodent Activity Monitoring in a False Ceiling Environment. Sensors, 21.","DOI":"10.3390\/s21165326"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Bao, Z., Li, A., Cui, Z., and Zhang, J. (2018, January 11\u201313). Visual Place Recognition Based on Multi-level CNN Features. Proceedings of the 3rd International Conference on Robotics, Control and Automation, Chengdu, China.","DOI":"10.1145\/3265639.3265684"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Patil, U., Gujarathi, A., Kulkarni, A., Jain, A., Malke, L., Tekade, R., Paigwar, K., and Chaturvedi, P. (2019, January 25\u201327). Deep Learning Based Stair Detection and Statistical Image Filtering for Autonomous Stair Climbing. Proceedings of the 2019 Third IEEE International Conference on Robotic Computing (IRC), Naples, Italy.","DOI":"10.1109\/IRC.2019.00031"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ramalingam, B., Hayat, A.A., Elara, M.R., F\u00e9lix G\u00f3mez, B., Yi, L., Pathmakumar, T., Rayguru, M.M., and Subramanian, S. (2021). Deep Learning Based Pavement Inspection Using Self-Reconfigurable Robot. Sensors, 21.","DOI":"10.3390\/s21082595"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Pathmakumar, T., Kalimuthu, M., Elara, M.R., and Ramalingam, B. (2021). An Autonomous Robot-Aided Auditing Scheme for Floor Cleaning. Sensors, 21.","DOI":"10.3390\/s21134332"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ophoff, T., Van Beeck, K., and Goedem\u00e9, T. (2019). Exploring RGB+ Depth Fusion for Real-Time Object Detection. Sensors, 19.","DOI":"10.3390\/s19040866"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.neucom.2018.01.055","article-title":"Object detection via deeply exploiting depth information","volume":"286","author":"Hou","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_31","unstructured":"Couprie, C., Farabet, C., Najman, L., and LeCun, Y. (2013). Indoor Semantic Segmentation using depth information. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014). Learning Rich Features from RGB-D Images for Object Detection and Segmentation. Computer Vision\u2014ECCV 2014, Springer International Publishing.","DOI":"10.1007\/978-3-319-10590-1"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ophoff, T., Goedem\u00e9, T., and Van Beeck, K. (2018, January 27\u201330). Improving Real-Time Pedestrian Detectors with RGB+Depth Fusion. Proceedings of the 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Auckland, New Zealand.","DOI":"10.1109\/AVSS.2018.8639110"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). Ssd: Single shot multibox detector. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_35","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv."},{"key":"ref_36","unstructured":"Tieleman, T., and Hinton, G. (2012). Lecture 6.5-RMSProp, COURSERA: Neural Networks for Machine Learning, University of Toronto. Technical Report."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.autcon.2016.06.008","article-title":"Vision-based detection of loosened bolts using the Hough transform and support vector machines","volume":"71","author":"Cha","year":"2016","journal-title":"Autom. Constr."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A computational approach to edge detection","volume":"6","author":"Canny","year":"1986","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Crnoki\u00c4, B., Rezi\u00c4, S., and Pehar, S. (2016). Comparision of Edge Detection Methods for Obstacles Detection in a Mobile Robot Environment. Annals of DAAAM & Proceedings, DAAAM International.","DOI":"10.2507\/27th.daaam.proceedings.035"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1145\/361237.361242","article-title":"Use of the Hough transformation to detect lines and curves in pictures","volume":"15","author":"Duda","year":"1972","journal-title":"Commun. ACM"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Munoz, R., Rong, X., and Tian, Y. (2016, January 11\u201315). Depth-aware indoor staircase detection and recognition for the visually impaired. Proceedings of the 2016 IEEE International Conference on Multimedia Expo Workshops (ICMEW), Seattle, WA, USA.","DOI":"10.1109\/ICMEW.2016.7574706"},{"key":"ref_42","first-page":"1","article-title":"Classification of trash for recyclability status","volume":"2016","author":"Yang","year":"2016","journal-title":"CS229 Proj. Rep."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Rad, M.S., von Kaenel, A., Droux, A., Tieche, F., Ouerhani, N., Ekenel, H.K., and Thiran, J.P. (2017, January 10\u201313). A Computer Vision System to Localize and Classify Wastes on the Streets. Proceedings of the International Conference on Computer Vision Systems, Shenzhen, China.","DOI":"10.1007\/978-3-319-68345-4_18"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Mittal, G., Yagnik, K.B., Garg, M., and Krishnan, N.C. (2016, January 12\u201316). Spotgarbage: Smartphone app to detect garbage using deep learning. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Heidelberg, Germany.","DOI":"10.1145\/2971648.2971731"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6279\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:02:09Z","timestamp":1760166129000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6279"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,18]]},"references-count":44,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21186279"],"URL":"https:\/\/doi.org\/10.3390\/s21186279","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,18]]}}}