{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,25]],"date-time":"2026-08-25T20:24:39Z","timestamp":1787689479003,"version":"build-2784847793"},"reference-count":58,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,4,26]],"date-time":"2022-04-26T00:00:00Z","timestamp":1650931200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"GRRC program of Gyeonggi province.","award":["GRRC-Gachon2020(B02), AI-based Medical Information Analysis"],"award-info":[{"award-number":["GRRC-Gachon2020(B02), AI-based Medical Information Analysis"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The growing aging population suffers from high levels of vision and cognitive impairment, often resulting in a loss of independence. Such individuals must perform crucial everyday tasks such as cooking and heating with systems and devices designed for visually unimpaired individuals, which do not take into account the needs of persons with visual and cognitive impairment. Thus, the visually impaired persons using them run risks related to smoke and fire. In this paper, we propose a vision-based fire detection and notification system using smart glasses and deep learning models for blind and visually impaired (BVI) people. The system enables early detection of fires in indoor environments. To perform real-time fire detection and notification, the proposed system uses image brightness and a new convolutional neural network employing an improved YOLOv4 model with a convolutional block attention module. The h-swish activation function is used to reduce the running time and increase the robustness of YOLOv4. We adapt our previously developed smart glasses system to capture images and inform BVI people about fires and other surrounding objects through auditory messages. We create a large fire image dataset with indoor fire scenes to accurately detect fires. Furthermore, we develop an object mapping approach to provide BVI people with complete information about surrounding objects and to differentiate between hazardous and nonhazardous fires. The proposed system shows an improvement over other well-known approaches in all fire detection metrics such as precision, recall, and average precision.<\/jats:p>","DOI":"10.3390\/s22093307","type":"journal-article","created":{"date-parts":[[2022,4,26]],"date-time":"2022-04-26T21:37:53Z","timestamp":1651009073000},"page":"3307","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":87,"title":["Automatic Fire Detection and Notification System Based on Improved YOLOv4 for the Blind and Visually Impaired"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1424-0799","authenticated-orcid":false,"given":"Mukhriddin","family":"Mukhiddinov","sequence":"first","affiliation":[{"name":"Department of Computer Engineering, Gachon University, Sujeong-gu, Seongnam-si 13120, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5923-8695","authenticated-orcid":false,"given":"Akmalbek Bobomirzaevich","family":"Abdusalomov","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence, Tashkent University of Information Technologies Named after Muhammad Al-Khwarizmi, Tashkent 100200, Uzbekistan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinsoo","family":"Cho","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Gachon University, Sujeong-gu, Seongnam-si 13120, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e144","DOI":"10.1016\/S2214-109X(20)30489-7","article-title":"Causes of blindness and vision impairment in 2020 and trends over 30 years, and prevalence of avoidable blindness in relation to VISION 2020: The Right to Sight: An analysis for the Global Burden of Disease Study","volume":"9","author":"Steinmetz","year":"2021","journal-title":"Lancet Glob. Health"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Dunai Dunai, L., Chillar\u00f3n P\u00e9rez, M., Peris-Fajarn\u00e9s, G., and Lengua Lengua, I. (2017). Euro banknote recognition system for blind people. Sensors, 17.","DOI":"10.3390\/s17010184"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.ijhcs.2015.08.003","article-title":"Sonification of guidance data during road crossing for people with visual impairments or blindness","volume":"85","author":"Sergio","year":"2016","journal-title":"Int. J. Hum.-Comput. Stud."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Li, X., Cui, H., Rizzo, J.-R., Wong, E., and Fang, Y. (2019). Cross-Safe: A computer vision-based approach to make all intersection-related pedestrian signals accessible for the visually impaired. Advances in Intelligent Systems and Computing, Springer.","DOI":"10.1007\/978-3-030-17798-0_13"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Seo, W., and Jung, H. (2021). Challenges and opportunities to improve the accessibility of YouTube for people with visual impairments as content creators. Univers. Access Inf. Soc., 1\u20134.","DOI":"10.1007\/s10209-020-00787-8"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, Y., Liang, W., Huang, H., Zhang, Y., Li, D., and Yu, L.-F. (2021, January 8\u201313). Toward automatic audio description generation for accessible videos. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan.","DOI":"10.1145\/3411764.3445347"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hu, W., Wang, K., Yang, K., Cheng, R., Ye, Y., Sun, L., and Xu, Z. (2020). A Comparative Study in Real-Time Scene Sonification for Visually Impaired People. Sensors, 20.","DOI":"10.3390\/s20113222"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"51975","DOI":"10.1109\/ACCESS.2018.2870334","article-title":"Deep-see face: A mobile face recognition system dedicated to visually impaired people","volume":"6","author":"Mocanu","year":"2018","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ahmetovic, D., Sato, D., Oh, U., Ishihara, T., Kitani, K., and Asakawa, C. (2020, January 25\u201330). Recog: Supporting blind people in recognizing personal objects. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3313831.3376143"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Vitiello, G., Sebillo, M., Fornaro, L., Di Gregorio, M., Cirillo, S., De Rosa, M., Fuccella, V., and Costagliola, G. (2018, January 28\u201330). Do you like my outfit? Cromnia, a mobile assistant for blind users. Proceedings of the 4th EAI International Conference on Smart Objects and Technologies for Social Good, Bologna, Italy.","DOI":"10.1145\/3284869.3284908"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Joshi, R.C., Yadav, S., Dutta, M.K., and Travieso-Gonzalez, C.M. (2020). Efficient Multi-Object Detection and Smart Navigation Using Artificial Intelligence for Visually Impaired People. Entropy, 22.","DOI":"10.3390\/e22090941"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"170406","DOI":"10.1109\/ACCESS.2019.2955046","article-title":"An indoor and outdoor navigation system for visually impaired people","volume":"7","author":"Croce","year":"2019","journal-title":"IEEE Access"},{"key":"ref_13","unstructured":"Ahrens, M., and Maheshwari, R. (2021). Home Structure Fires, National Fire Protection Association."},{"key":"ref_14","unstructured":"Deborah, L. (2020). Fire and Rescue Incident Statistics, Home Office."},{"key":"ref_15","unstructured":"(2022, April 20). Smoke Alarms Fail in a Third of House Fires. Available online: https:\/\/www.bbc.co.uk\/news\/ukengland-50598387."},{"key":"ref_16","unstructured":"Ahrens, M. (2019). Home Fires Started by Smoking, National Fire Protection Association."},{"key":"ref_17","unstructured":"Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mukhiddinov, M., and Cho, J. (2021). Smart Glass System Using Deep Learning for the Blind and Visually Impaired. Electronics, 10.","DOI":"10.3390\/electronics10222756"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ramadhan, A.J. (2018). Wearable smart system for visually impaired people. Sensors, 18.","DOI":"10.3390\/s18030843"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1375","DOI":"10.1109\/JIOT.2018.2842229","article-title":"Wearable vision assistance system based on binocular sensors for visually impaired users","volume":"6","author":"Jiang","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1109\/TCE.2017.014980","article-title":"Smart guiding glasses for visually impaired people in indoor environment","volume":"63","author":"Bai","year":"2017","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jamil, F., Ahmad, S., Iqbal, N., and Kim, D.-H. (2020). Towards a Remote Monitoring of Patient Vital Signs Based on IoT-Based Blockchain Integrity Management Platforms in Smart Hospitals. Sensors, 20.","DOI":"10.3390\/s20082195"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"10343","DOI":"10.1109\/ACCESS.2018.2886308","article-title":"A sensing framework for indoor spatial awareness for blind and visually impaired users","volume":"7","author":"Yang","year":"2018","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"922","DOI":"10.1109\/JSYST.2014.2320639","article-title":"Navigation assistance for the visually impaired using RGB-D sensor with range expansion","volume":"10","author":"Aladren","year":"2014","journal-title":"IEEE Syst. J."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1109\/MCOM.2018.1700706","article-title":"Indoor anti-collision alarm system based on wearable Internet of Things for smart healthcare","volume":"56","author":"Xiao","year":"2018","journal-title":"IEEE Commun. Mag."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1109\/TOH.2020.2996748","article-title":"Integrating wearable haptics and obstacle avoidance for the visually impaired in indoor navigation: A user-centered approach","volume":"14","author":"Barontini","year":"2020","journal-title":"IEEE Trans. Haptics"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Plikynas, D., Indriulionis, A., Laukaitis, A., and Sakalauskas, L. (2022). Indoor-guided navigation for people who are blind: Crowdsourcing for route mapping and assistance. Appl. Sci., 12.","DOI":"10.3390\/app12010523"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/THMS.2018.2799588","article-title":"Design and Construction of Electronic Aid for Visually Impaired People","volume":"48","author":"Patil","year":"2018","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"130767","DOI":"10.1109\/ACCESS.2021.3111544","article-title":"An Outdoor Navigation Assistance System for Visually Impaired People in Public Transportation","volume":"9","year":"2021","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Shin, K., McConville, R., Metatla, O., Chang, M., Han, C., Lee, J., and Roudaut, A. (2022). Outdoor localization using BLE RSSI and accessible pedestrian signals for the visually impaired at intersections. Sensors, 22.","DOI":"10.3390\/s22010371"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1109\/TOH.2019.2945561","article-title":"FingerSight: A Vibrotactile Wearable Ring for Assistance With Locating and Reaching Objects in Peripersonal Space","volume":"13","author":"Satpute","year":"2020","journal-title":"IEEE Trans. Haptics"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"164661","DOI":"10.1109\/ACCESS.2020.3022425","article-title":"An Evaluation of a Wearable Assistive Device for Augmenting Social Interactions","volume":"8","author":"Qiu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1109\/JAS.2019.1911546","article-title":"A new fire detection method using a multi-expert system based on color dispersion, similarity and centroid motion in indoor environment","volume":"7","author":"Wang","year":"2020","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"17335","DOI":"10.1109\/JSEN.2021.3081831","article-title":"Design and Implementation of an Intelligent Autonomous Surveillance System for Indoor Environments","volume":"21","author":"Chang","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"182381","DOI":"10.1109\/ACCESS.2019.2960209","article-title":"Unsupervised Segmentation of Fire and Smoke from Infra-Red Videos","volume":"7","author":"Ajith","year":"2019","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Gagliardi, A., and Saponara, S. (2020). AdViSED: Advanced video smoke detection for real-time measurements in anti-fire indoor and outdoor systems. Energies, 13.","DOI":"10.3390\/en13082098"},{"key":"ref_37","first-page":"87","article-title":"Machine vision based indoor fire detection using static and dynamic features","volume":"11","author":"Khan","year":"2018","journal-title":"Int. J. Control. Autom."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1016\/j.neucom.2017.04.083","article-title":"Early fire detection using convolutional neural networks during surveillance for effective disaster management","volume":"288","author":"Muhammad","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Avazov, K., Mukhiddinov, M., Makhmudov, F., and Cho, Y.I. (2021). Fire Detection Method in Smart City Environments Using a Deep-Learning-Based Approach. Electronics, 1.","DOI":"10.3390\/electronics11010073"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on image data augmentation for deep learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J. Big Data"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2351","DOI":"10.1007\/s10462-021-10066-4","article-title":"A comprehensive survey of recent trends in deep learning for digital images augmentation","volume":"55","author":"Khalifa","year":"2021","journal-title":"Artif. Intell. Rev."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"4591","DOI":"10.3233\/JIFS-191375","article-title":"Accident risk prediction and avoidance in intelligent semi-autonomous vehicles based on road safety data and driver biological behaviours","volume":"38","author":"Ahmad","year":"2020","journal-title":"J. Intell. Fuzzy Syst."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"8483","DOI":"10.1109\/TIP.2021.3115672","article-title":"Autopedestrian: An automatic data augmentation and loss function search scheme for pedestrian detection","volume":"30","author":"Tang","year":"2021","journal-title":"IEEE Trans. Image Processing"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Abdusalomov, A., Baratov, N., Kutlimuratov, A., and Whangbo, T.K. (2021). An improvement of the fire detection and classification method using YOLOv3 for surveillance systems. Sensors, 21.","DOI":"10.3390\/s21196519"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Valikhujaev, Y., Abdusalomov, A., and Cho, Y.I. (2020). Automatic fire and smoke detection method for surveillance systems based on dilated CNNs. Atmosphere, 11.","DOI":"10.3390\/atmos11111241"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Abdusalomov, A., Mukhiddinov, M., Djuraev, O., Khamdamov, U., and Whangbo, T.K. (2020). Automatic salient object extraction based on locally adaptive thresholding to generate tactile graphics. Appl. Sci., 10.","DOI":"10.3390\/app10103350"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Fu, H., Song, G., and Wang, Y. (2021). Improved YOLOv4 Marine Target Detection Combined with CBAM. Symmetry, 13.","DOI":"10.3390\/sym13040623"},{"key":"ref_48","unstructured":"Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv."},{"key":"ref_49","first-page":"713","article-title":"Saliency cuts: Salient region extraction based on local adaptive thresholding for image information recognition of the visually impaired","volume":"17","author":"Mukhriddin","year":"2020","journal-title":"Int. Arab J. Inf. Technol."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1259","DOI":"10.1109\/ACCESS.2021.3132787","article-title":"Spatio-Temporal Self-Attention Network for Fire Detection and Segmentation in Video Surveillance","volume":"10","author":"Shahid","year":"2022","journal-title":"IEEE Access"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"2050052","DOI":"10.1142\/S0219691320500526","article-title":"Improvement of the end-to-end scene text recognition method for \u201ctext-to-speech\u201d conversion","volume":"18","author":"Makhmudov","year":"2020","journal-title":"Int. J. Wavelets Multiresolution Inf. Processing"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Mukhiddinov, M., and Kim, S.Y. (2021). A Systematic Literature Review on the Automatic Creation of Tactile Graphics for the Blind and Visually Impaired. Processes, 9.","DOI":"10.3390\/pr9101726"},{"key":"ref_53","unstructured":"Songtao, L., Huang, D., and Wang, Y. (2018, January 8\u201314). Receptive field block net for accurate and fast object detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany."},{"key":"ref_54","unstructured":"Wei, L., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016, January 11\u201314). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Zhang, S., Wen, L., Bian, X., Lei, Z., and Li, S.Z. (2018, January 18\u201323). Single-shot refinement neural network for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00442"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"3191","DOI":"10.1109\/JSEN.2019.2894665","article-title":"Fire sensing technologies: A review","volume":"19","author":"Gaur","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_57","unstructured":"Business and Economic Research Limited (2019). Economic Cost of Unwanted Fire Alarms 2019 (Report No. 171), Fire and Emergency."},{"key":"ref_58","unstructured":"Ahrens, M. (2021). Smoke Alarms in US Home Fires, National Fire Protection Association."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/9\/3307\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:01:13Z","timestamp":1760137273000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/9\/3307"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,26]]},"references-count":58,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["s22093307"],"URL":"https:\/\/doi.org\/10.3390\/s22093307","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,26]]}}}