{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T17:05:48Z","timestamp":1775063148950,"version":"3.50.1"},"reference-count":25,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2020,9,26]],"date-time":"2020-09-26T00:00:00Z","timestamp":1601078400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Rehabilitative mobility aids are being used extensively for physically impaired people. Efforts are being made to develop human machine interfaces (HMIs), manipulating the biosignals to better control the electromechanical mobility aids, especially the wheelchairs. Creating precise control commands such as move forward, left, right, backward and stop, via biosignals, in an appropriate HMI is the actual challenge, as the people with a high level of disability (quadriplegia and paralysis, etc.) are unable to drive conventional wheelchairs. Therefore, a novel system driven by optical signals addressing the needs of such a physically impaired population is introduced in this paper. The present system is divided into two parts: the first part comprises of detection of eyeball movements together with the processing of the optical signal, and the second part encompasses the mechanical assembly module, i.e., control of the wheelchair through motor driving circuitry. A web camera is used to capture real-time images. The processor used is Raspberry-Pi with Linux operating system. In order to make the system more congenial and reliable, the voice-controlled mode is incorporated in the wheelchair. To appraise the system\u2019s performance, a basic wheelchair skill test (WST) is carried out. Basic skills like movement on plain and rough surfaces in forward, reverse direction and turning capability were analyzed for easier comparison with other existing wheelchair setups on the bases of controlling mechanisms, compatibility, design models, and usability in diverse conditions. System successfully operates with average response time of 3 s for eye and 3.4 s for voice control mode.<\/jats:p>","DOI":"10.3390\/s20195510","type":"journal-article","created":{"date-parts":[[2020,9,28]],"date-time":"2020-09-28T08:02:58Z","timestamp":1601280178000},"page":"5510","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Eye and Voice-Controlled Human Machine Interface System for Wheelchairs Using Image Gradient Approach"],"prefix":"10.3390","volume":"20","author":[{"given":"Saba","family":"Anwer","sequence":"first","affiliation":[{"name":"School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad 45200, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0190-0700","authenticated-orcid":false,"given":"Asim","family":"Waris","sequence":"additional","affiliation":[{"name":"School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad 45200, Pakistan"}]},{"given":"Hajrah","family":"Sultan","sequence":"additional","affiliation":[{"name":"School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad 45200, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6020-1512","authenticated-orcid":false,"given":"Shahid Ikramullah","family":"Butt","sequence":"additional","affiliation":[{"name":"School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology, Islamabad 45200, Pakistan"}]},{"given":"Muhammad Hamza","family":"Zafar","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, University of Engineering and Technology Lahore-FSD Campus, Faisalabad 38000, Pakistan"}]},{"given":"Moaz","family":"Sarwar","sequence":"additional","affiliation":[{"name":"Department of Computer Sciences, Government College University, Faisalabad 38000, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8752-7224","authenticated-orcid":false,"given":"Imran Khan","family":"Niazi","sequence":"additional","affiliation":[{"name":"Center of Chiropractic Research, New Zealand College of Chiropractic, Auckland 0600, New Zealand"},{"name":"Department of Health Science and Technology, Center for Sensory-Motor Interaction, Aalborg University, 9000 Alborg, Denmark"},{"name":"Faculty of Health and Environmental Sciences, Health and Rehabilitation Research Institute, AUT University, Auckland 0627, New Zealand"}]},{"given":"Muhammad","family":"Shafique","sequence":"additional","affiliation":[{"name":"Head of Department, Biomedical Engineering, Riphah International University, Islamabad 45710, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1688-4448","authenticated-orcid":false,"given":"Amit N.","family":"Pujari","sequence":"additional","affiliation":[{"name":"School of Engineering and Technology, University of Hertfordshire, Hatfield AL10 9AB, UK"},{"name":"School of Engineering, University of Aberdeen, Aberdeen AB24 3FX, UK"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Champaty, B., Jose, J., Pal, K., and Thirugnanam, A. (2014, January 24\u201326). Development of EOG based human machine interface control system for motorized wheelchair. Proceedings of the 2014 Annual International Conference on Emerging Research Areas: Magnetics, Machines and Drives (AICERA\/iCMMD), Kottayam, India.","DOI":"10.1109\/AICERA.2014.6908256"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1682\/JRRD.2004.08.0101","article-title":"Smart wheelchairs: A literaturereview","volume":"42","author":"Simpson","year":"2005","journal-title":"J. Rehabil. Res. Dev."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1109\/MIS.2007.36","article-title":"Integrating human inputs with autonomousbehaviors on an intelligent wheelchair platform","volume":"22","author":"Parikh","year":"2007","journal-title":"IEEE Intell. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Valbuena, D., Cyriacks, M., Friman, O., Volosyak, I., and Graser, A. (2007, January 13\u201315). Brain-computer interface for high-level control of rehabilitation robotic systems. Proceedings of the 2007 IEEE 10th International Conference on Rehabilitation Robotics, Noordwijk, The Netherlands.","DOI":"10.1109\/ICORR.2007.4428489"},{"key":"ref_5","unstructured":"Choi, K., Sato, M., and Koike, Y. (2006, January 16\u201321). A new, human-centered wheelchair system controlled by the EMG signal. Proceedings of the 2006 IEEE International Joint Conference on Neural Network Proceedings, Vancouver, BC, Canada."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1007\/s10514-016-9568-y","article-title":"Intelligent wheelchair control strategies for older adults with cognitive impairment: User attitudes, needs, and preferences","volume":"41","author":"Viswanathan","year":"2017","journal-title":"Auton. Robots"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"012064","DOI":"10.1088\/1742-6596\/1432\/1\/012064","article-title":"Arduino based voice controlled wheelchair","volume":"1432","author":"Hou","year":"2020","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_8","unstructured":"Khairosfaizal, W.W.M., and Nor\u2019aini, A.J. (2009, January 6\u20138). Eyes detection in facial images using circular hough transform. Proceedings of the 2009 5th International Colloquium on Signal Processing & Its Applications, Kuala Lumpur, Malaysia."},{"key":"ref_9","unstructured":"Zhao, S., and Grigat, R.R. (2006, January 20\u201324). Robust eye detection under active infrared illumination. Proceedings of the 18th International Conference on Pattern Recognition (ICPR\u201906), Hong Kong, China."},{"key":"ref_10","first-page":"127","article-title":"Facial feature detection using Haar classifiers","volume":"21","author":"Wilson","year":"2006","journal-title":"J. Comput. Sci. Coll."},{"key":"ref_11","unstructured":"Rad, A.A., Faez, K., and Qaragozlou, N. (2003, January 10\u201312). Fast Circle Detection Using Gradient Pair Vectors. Proceedings of the Seventh International Conference on Digital Image Computing: Techniques and Applications, Sydney, Australia."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Chen, B.C., Wu, P.C., and Chien, S.Y. (2015, January 27\u201330). Real-time eye localization, blink detection, and gaze estimation system without infrared illumination. Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada.","DOI":"10.1109\/ICIP.2015.7350892"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1136\/bjsm.19.4.230","article-title":"Physiotherapy and electromyography in muscle cramp","volume":"19","author":"Helin","year":"1985","journal-title":"Br. J. Sports Med."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1159\/000441447","article-title":"Clinical neurophysiological and automated EEG-based diagnosis of the alzheimer\u2019s disease","volume":"74","author":"Bhat","year":"2015","journal-title":"Eur. Neurol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mirza, I.A., Tripathy, A., Chopra, S., D\u2019Sa, M., Rajagopalan, K., D\u2019Souza, A., and Sharma, N. (2015, January 4\u20136). Mind-controlled wheelchair using an EEG headset and arduino microcontroller. Proceedings of the 2015 International Conference on Technologies for Sustainable Development (ICTSD), Mumbai, India.","DOI":"10.1109\/ICTSD.2015.7095887"},{"key":"ref_16","first-page":"48","article-title":"Implementation of head and finger movement based automatic wheel chair","volume":"1","author":"Tameemsultana","year":"2011","journal-title":"Bonfring Int. J. Power Syst. Integr. Circuits"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1573","DOI":"10.1016\/j.jbiomech.2010.01.027","article-title":"Filtering the surface EMG signal: Movement artifact and baseline noise contamination","volume":"43","author":"Gilmore","year":"2010","journal-title":"J. Biomech."},{"key":"ref_18","first-page":"15","article-title":"Accelerometer based hand gesture controlled wheelchair","volume":"4","author":"Goyal","year":"2013","journal-title":"Int. J. Emerg. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1023\/A:1016359503796","article-title":"Wheelchair guidance strategies using EOG","volume":"34","author":"Barea","year":"2002","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Arora, P., Sharma, A., Soni, A.S., and Garg, A. (2015, January 17\u201320). Control of wheelchair dummy for differently abled patients via iris movement using image processing in MATLAB. Proceedings of the 2015 Annual IEEE India Conference (INDICON), New Delhi, India.","DOI":"10.1109\/INDICON.2015.7443610"},{"key":"ref_21","unstructured":"Kaehler, A., and Bradski, G. (2016). Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library, O\u2019Reilly Media, Inc."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Juhong, A., Treebupachatsakul, T., and Pintavirooj, C. (2018, January 7\u20139). Smart eye-tracking system. Proceedings of the 2018 International Workshop on Advanced Image Technology (IWAIT), Chiang Mai, Thailand.","DOI":"10.1109\/IWAIT.2018.8369701"},{"key":"ref_23","first-page":"125","article-title":"Accurate eye centre localisation by means of gradients","volume":"11","author":"Timm","year":"2011","journal-title":"Visapp"},{"key":"ref_24","unstructured":"Org, R.P. (2017, June 02). Raspberry Pi 3 Model B. Available online: https:\/\/www.raspberrypi.org\/products\/raspberry-pi-3-model-b\/."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1080\/10400435.2015.1039149","article-title":"Impact of mass and weight distribution on manual wheelchair propulsion torque","volume":"27","author":"Sprigle","year":"2015","journal-title":"Assist. Technol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5510\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:13:52Z","timestamp":1760177632000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5510"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,26]]},"references-count":25,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2020,10]]}},"alternative-id":["s20195510"],"URL":"https:\/\/doi.org\/10.3390\/s20195510","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,26]]}}}