{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T08:26:38Z","timestamp":1783067198885,"version":"3.54.6"},"reference-count":43,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2021,4,4]],"date-time":"2021-04-04T00:00:00Z","timestamp":1617494400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Pedestrian fatalities and injuries most likely occur in vehicle-pedestrian crashes. Meanwhile, engineers have tried to reduce the problems by developing a pedestrian detection function in Advanced Driver-Assistance Systems (ADAS) and autonomous vehicles. However, the system is still not perfect. A remaining problem in pedestrian detection is the performance reduction at nighttime, although pedestrian detection should work well regardless of lighting conditions. This study presents an evaluation of pedestrian detection performance in different lighting conditions, then proposes to adopt multispectral image and deep neural network to improve the detection accuracy. In the evaluation, different image sources including RGB, thermal, and multispectral format are compared for the performance of the pedestrian detection. In addition, the optimizations of the architecture of the deep neural network are performed to achieve high accuracy and short processing time in the pedestrian detection task. The result implies that using multispectral images is the best solution for pedestrian detection at different lighting conditions. The proposed deep neural network accomplishes a 6.9% improvement in pedestrian detection accuracy compared to the baseline method. Moreover, the optimization for processing time indicates that it is possible to reduce 22.76% processing time by only sacrificing 2% detection accuracy.<\/jats:p>","DOI":"10.3390\/s21072536","type":"journal-article","created":{"date-parts":[[2021,4,5]],"date-time":"2021-04-05T11:48:29Z","timestamp":1617623309000},"page":"2536","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":44,"title":["Pedestrian Detection Using Multispectral Images and a Deep Neural Network"],"prefix":"10.3390","volume":"21","author":[{"given":"Jason","family":"Nataprawira","sequence":"first","affiliation":[{"name":"College of Information Science and Engineering, Ritsumeikan University, Shiga 525-8577, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanlei","family":"Gu","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Ritsumeikan University, Shiga 525-8577, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Igor","family":"Goncharenko","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Ritsumeikan University, Shiga 525-8577, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shunsuke","family":"Kamijo","sequence":"additional","affiliation":[{"name":"Institute of Industrial Science, The University of Tokyo, Tokyo 153-8505, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,4]]},"reference":[{"key":"ref_1","unstructured":"(2021, January 22). Road Safety Report 2019. International Traffic Safety Data and Analysis Group of the International Transportation Forum. Available online: https:\/\/www.itf-oecd.org\/sites\/default\/files\/docs\/irtad-road-safety-annual-report-2019.pdf."},{"key":"ref_2","unstructured":"Wakabayashi, D. (2020, December 08). Self-Driving Uber Car Kills Pedestrian in Arizona, Where Robots Roam. Available online: https:\/\/www.nytimes.com\/2018\/03\/19\/technology\/uber-driverless-fatality.html."},{"key":"ref_3","unstructured":"(2020, January 30). Traffic Safety Facts 2017 Data: Pedestrians. National Highway Traffic Safety Administration of the US Department of Transportation, Available online: https:\/\/www.nhtsa.gov\/technology-innovation\/automated-vehicles-safety."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Hwang, S., Park, J., Kim, N., Choi, Y., and Kweon, I.S. (2015, January 7\u201312). Multispectral Pedestrian Detection: Benchmark Dataset and Baseline. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298706"},{"key":"ref_5","unstructured":"(2021, January 22). Automatic Emergency Braking with Pedestrian Detection. American Automobile Association. Available online: https:\/\/www.aaa.com\/AAA\/common\/aar\/files\/Research-Report-Pedestrian-Detection.pdf."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1016\/j.iatssr.2019.11.005","article-title":"Automated Driving Recognition Technologies for Adverse Weather Conditions","volume":"43","author":"Yoneda","year":"2019","journal-title":"IATSS Res."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Heuer, M., Al-Hamadi, A., Rain, A., and Meinecke, M.M. (2014, January 8\u201311). Detection and Tracking Approach using an Automotive Radar to Increase Active Pedestrian Safety. Proceedings of the 2014 IEEE Intelligent Vehicles Symposium, Dearborn, MI, USA.","DOI":"10.1109\/IVS.2014.6856589"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"696","DOI":"10.1002\/rob.20312","article-title":"LIDAR and Vision-based Pedestrian Detection System","volume":"26","author":"Premebida","year":"2009","journal-title":"J. Field Robot."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Camara, F., Bellotto, N., Cosar, S., Nathanael, D., Althoff, M., Wu, J., Ruenz, J., Dietrich, A., and Fox, C. (2020). Pedestrian Models for Autonomous Driving Part I: Low-level Models, from Sensing to Tracking. IEEE Trans. Intell. Transp. Syst., 1\u201321.","DOI":"10.1109\/TITS.2020.3006768"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Viola, P., Jones, M.J., and Snow, D. (2003, January 13\u201316). Detecting Pedestrians Using Patterns of Motion and Appearance. Proceedings of the Ninth IEEE International Conference on Computer Vision, Nice, France.","DOI":"10.1109\/ICCV.2003.1238422"},{"key":"ref_11","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201326). Histograms of Oriented Gradients for Human Detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.neucom.2018.01.092","article-title":"Computer Vision and Deep Learning Techniques for Pedestrian Detection and Tracking: A Survey","volume":"300","author":"Brunetti","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhang, L., Li, S.Z., Yuan, X., and Xiang, S. (2007, January 17\u201322). Real-Time Object Classification in Video Surveillance Based on Appearance Learning. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.","DOI":"10.1109\/CVPR.2007.383503"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Moradi, M.J., and Hariri-Ardebili, M.A. (2019). Developing a Library of Shear Walls Database and the Neural Network based Predictive Meta-model. Appl. Sci., 9.","DOI":"10.3390\/app9122562"},{"key":"ref_15","first-page":"1831","article-title":"Wavelet Kernel-Based Convolutional Neural Network for Localization of Partial Discharge Sources Within a Power Apparatus","volume":"17","author":"Ganguly","year":"2020","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1955","DOI":"10.1016\/j.aej.2020.11.043","article-title":"Evaluation of Flow Pattern Recognition and Void Fraction Measurement in Two Phase Flow Independent of Oil Pipeline\u2019s Scale Layer Thickness","volume":"60","author":"Roshani","year":"2021","journal-title":"Alex. Eng. J."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.eswa.2020.113275","article-title":"A Cost-sensitive Convolution Neural Network Learning for Control Chart Pattern Recognition","volume":"150","author":"Fuqua","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., Malik, J., Berkeley, U.C., and Malik, J. (2014, January 23\u201328). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 11\u201318). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision, Las Condes, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2016). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv.","DOI":"10.1109\/TPAMI.2016.2577031"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 8\u201316). SSD: Single Shot Multibox Detector. Proceedings of the 2016 European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Redmon, J., and Farhadi, A. (2017, January 21\u201326). YOLO9000: Better, Faster, Stronger. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.690"},{"key":"ref_24","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhao, X., Li, W., Zhang, Y., Gulliver, T.A., Chang, S., and Feng, Z. (2016, January 18\u201321). A Faster RCNN-based Pedestrian Detection System. Proceedings of the 2016 IEEE 84th Vehicular Technology Conference (VTC-Fall), Montreal, QC, Canada.","DOI":"10.1109\/VTCFall.2016.7880852"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Lan, W., Dang, J., Wang, Y., and Wang, S. (2018, January 5\u20138). Pedestrian Detection based on YOLO Network Model. Proceedings of the 2018 IEEE International Conference on Mechatronics and Automation, Changchun, China.","DOI":"10.1109\/ICMA.2018.8484698"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1504\/IJCAT.2021.113643","article-title":"Pedestrian Detection Algorithm based on Improved SSD","volume":"65","author":"Liu","year":"2021","journal-title":"Int. J. Comput. Appl. Technol."},{"key":"ref_28","unstructured":"Piniarski, K., Paw\u0142owski, P., and D\u0105browski, A. (2014, January 22\u201324). Pedestrian Detection by Video Processing in Automotive Night Vision System. Proceedings of the 2014 Signal Processing: Algorithms, Architectures, Arrangements, and Applications, Poznan, Poland."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Sun, H., Wang, C., and Wang, B. (2011, January 10\u201312). Night Vision Pedestrian Detection Using a Forward-Looking Infrared Camera. Proceedings of the 2011 International Workshop on Multi-Platfor\/Multi-Sensor Remote Sensing and Mapping, Xiamen, China.","DOI":"10.1109\/M2RSM.2011.5697384"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Luo, Y., Remillard, J., and Hoetzer, D. (2010, January 21\u201324). Pedestrian Detection in Near-Infrared Night Vision System. Proceedings of the 2010 IEEE Intelligent Vehicles Symposium, La Jolla, CA, USA.","DOI":"10.1109\/IVS.2010.5548089"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Govardhan, P., and Pati, U.C. (2014, January 8\u201310). NIR Image Based Pedestrian Detection in Night Vision with Cascade Classification and Validation. Proceedings of the 2014 IEEE International Conference on Advanced Communications, Control and Computing Technologies, Ramanathapuram, India.","DOI":"10.1109\/ICACCCT.2014.7019339"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Han, T.Y., and Song, B.C. (2016, January 26\u201328). Night Vision Pedestrian Detection Based on Adaptive Preprocessing Using near Infrared Camera. Proceedings of the 2016 IEEE International Conference on Consumer Electronics-Asia, Seoul, Korea.","DOI":"10.1109\/ICCE-Asia.2016.7804763"},{"key":"ref_33","unstructured":"Chebrolu, K.N.R., and Kumar, P.N. (2019, January 4\u20136). Deep Learning Based Pedestrian Detection at All Light Conditions. Proceedings of the 2019 IEEE International Conference on Communication and Signal Processing, Chennai, India."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.infrared.2018.08.029","article-title":"Multispectral Pedestrian Detection Based on Deep Convolutional Neural Networks","volume":"94","author":"Hou","year":"2018","journal-title":"Infrared Phys. Technol."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Konig, D., Adam, M., Jarvers, C., Layher, G., Neumann, H., and Teutsch, M. (2017, January 21\u201326). Fully Convolutional Region Proposal Networks for Multispectral Person Detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.36"},{"key":"ref_36","unstructured":"Choi, H., Kim, S., Park, K., and Sohn, K. (2016, January 4\u20138). Multi-Spectral Pedestrian Detection Based on Accumulated Object Proposal with Fully Convolutional Networks. Proceedings of the 2016 International Conference on Pattern Recognition, Canc\u00fan, Mexico."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Nataprawira, J., Gu, Y., Goncharenko, I., and Kamijo, S. (2021, January 10\u201312). Pedestrian Detection on Multispectral Images in Different Lighting Conditions. Proceedings of the 39th IEEE International Conference on Consumer Electronics, Online Meeting.","DOI":"10.1109\/ICCE50685.2021.9427627"},{"key":"ref_38","unstructured":"Nataprawira, J. (2021). Pedestrian Detection in Different Lighting Conditions Using Deep Neural Network and Multispectral Images. [Bachelor\u2019s Thesis, Ritsumeikan University]."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Versaci, M., and Morabito, F.C. (2021). Image Edge Detection: A New Approach Based on Fuzzy Entropy and Fuzzy Divergence. Int. J. Fuzzy Syst., 1\u201319.","DOI":"10.1007\/s40815-020-01030-5"},{"key":"ref_40","unstructured":"Glenn, J., Yonghye, K., Josh, V.M., Gabriel, B., Fatih, B., and Daniel, S. (2020, June 23). ultralytics\/yolov3: 43.1mAP@0.5:0.95 on COCO2014. Available online: https:\/\/zenodo.org\/record\/3785397#.YGlfdT8RXIU."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Lu, Y., Zhang, L., and Xie, W. (2020, January 22\u201324). YOLO-Compact: An Efficient YOLO Network for Single Category Real-Time Object Detection. Proceedings of the 32nd Chinese Control and Decision Conference, Hefei, China.","DOI":"10.1109\/CCDC49329.2020.9164580"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Chen, K., and Deng, J.D. (2019, January 15\u201318). An Optimized CNN Model for Pedestrian Implement on Development Boards. Proceedings of the 2019 IEEE 8th Global Conference on Consumer Electronics, Osaka, Japan.","DOI":"10.1109\/GCCE46687.2019.9015597"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Dollar, P., Wojek, C., Schiele, B., and Perona, P. (2009, January 20\u201326). Pedestrian Detection: A Benchmark. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206631"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/7\/2536\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:58:21Z","timestamp":1760363901000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/7\/2536"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,4]]},"references-count":43,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["s21072536"],"URL":"https:\/\/doi.org\/10.3390\/s21072536","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,4]]}}}