{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T06:57:09Z","timestamp":1760597829938,"version":"build-2065373602"},"reference-count":63,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2018,9,7]],"date-time":"2018-09-07T00:00:00Z","timestamp":1536278400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["NRF-2018R1D1A1B07041921","NRF-2017R1C1B5074062"],"award-info":[{"award-number":["NRF-2018R1D1A1B07041921","NRF-2017R1C1B5074062"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010418","name":"Institute for Information and communications Technology Promotion","doi-asserted-by":"publisher","award":["IITP-2018-2013-1-00684"],"award-info":[{"award-number":["IITP-2018-2013-1-00684"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Conventional nighttime face detection studies mostly use near-infrared (NIR) light cameras or thermal cameras, which are robust to environmental illumination variation and low illumination. However, for the NIR camera, it is difficult to adjust the intensity and angle of the additional NIR illuminator according to its distance from an object. As for the thermal camera, it is expensive to use as a surveillance camera. For these reasons, we propose a nighttime face detection method based on deep learning using a single visible-light camera. In a long-distance night image, it is difficult to detect faces directly from the entire image due to noise and image blur. Therefore, we propose Two-Step Faster region-based convolutional neural network (R-CNN) based on the image preprocessed by histogram equalization (HE). As a two-step scheme, our method sequentially performs the detectors of body and face areas, and locates the face inside a limited body area. By using our two-step method, the processing time by Faster R-CNN can be reduced while maintaining the accuracy of face detection by Faster R-CNN. Using a self-constructed database called Dongguk Nighttime Face Detection database (DNFD-DB1) and an open database of Fudan University, we proved that the proposed method performs better compared to other existing face detectors. In addition, the proposed Two-Step Faster R-CNN outperformed single Faster R-CNN and our method with HE showed higher accuracies than those without our preprocessing in nighttime face detection.<\/jats:p>","DOI":"10.3390\/s18092995","type":"journal-article","created":{"date-parts":[[2018,9,7]],"date-time":"2018-09-07T11:47:41Z","timestamp":1536320861000},"page":"2995","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":41,"title":["Face Detection in Nighttime Images Using Visible-Light Camera Sensors with Two-Step Faster Region-Based Convolutional Neural Network"],"prefix":"10.3390","volume":"18","author":[{"given":"Se Woon","family":"Cho","sequence":"first","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pil-dong-ro 1-gil, Jung-gu, Seoul 04620, Korea"}]},{"given":"Na Rae","family":"Baek","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pil-dong-ro 1-gil, Jung-gu, Seoul 04620, Korea"}]},{"given":"Min Cheol","family":"Kim","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pil-dong-ro 1-gil, Jung-gu, Seoul 04620, Korea"}]},{"given":"Ja Hyung","family":"Koo","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pil-dong-ro 1-gil, Jung-gu, Seoul 04620, Korea"}]},{"given":"Jong Hyun","family":"Kim","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pil-dong-ro 1-gil, Jung-gu, Seoul 04620, Korea"}]},{"given":"Kang Ryoung","family":"Park","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pil-dong-ro 1-gil, Jung-gu, Seoul 04620, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2018,9,7]]},"reference":[{"key":"ref_1","unstructured":"Viola, P., and Jones, M. (2001, January 8\u201314). Rapid object detection using a boosted cascade of simple features. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA."},{"key":"ref_2","unstructured":"Jin, H., Liu, Q., Lu, H., and Tong, X. (2004, January 18\u201320). Face detection using improved LBP under Bayesian framework. Proceedings of the 3rd International Conference on Image and Graphics, Hong Kong, China."},{"key":"ref_3","unstructured":"Zhang, L., Chu, R., Xiang, S., Liao, S., and Li, S.Z. (2007, January 27\u201329). Face detection based on multi-block LBP representation. Proceedings of the International Conference on Biometrics, Seoul, Korea."},{"key":"ref_4","first-page":"1345","article-title":"Face detection in real time based on HOG","volume":"3","author":"Rekha","year":"2014","journal-title":"Int. J. Adv. Res. Comput. Eng. Technol."},{"key":"ref_5","unstructured":"Cerna, L.R., C\u00e1mara-Ch\u00e1vez, G., and Menotti, D. (2013, January 22\u201325). Face detection: Histogram of oriented gradients and bag of feature method. Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_6","unstructured":"Pavlidis, I., and Symosek, P. (2000, January 16). The imaging issue in an automatic face\/disguise detection system. Proceedings of the IEEE Workshop on Computer Vision Beyond the Visible Specturm: Methods and Applications, Hilton Head, SC, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ukimura, O. (2011). Fusion of infrared and visible images for robust person detection. Image Fusion, InTech.","DOI":"10.5772\/602"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Bourlai, T. (2016). Understanding thermal face detection: Challenges and evaluation. Face Recognition across the Imaging Spectrum, Springer International Publishing.","DOI":"10.1007\/978-3-319-28501-6"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/s17122741","article-title":"Adapting local features for face detection in thermal image","volume":"17","author":"Ma","year":"2017","journal-title":"Sensors"},{"key":"ref_10","unstructured":"Murphy-Chutorian, E., Doshi, A., and Trivedi, M.M. (October, January 30). Head pose estimation for driver assistance systems: A robust algorithm and experimental evaluation. Proceedings of the IEEE Intelligent Transportation Systems Conference, Seattle, WA, USA."},{"key":"ref_11","first-page":"275","article-title":"Occupant detection through near-infrared imaging","volume":"14","author":"Hao","year":"2011","journal-title":"Tamkang J. Sci. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lemoff, B.E., Martin, R.B., Sluch, M., Kafka, K.M., McCormick, W., and Ice, R. (May, January 29). Automated night\/day standoff detection, tracking, and identification of personnel for installation protection. Proceedings of the SPIE 8711, Sensors, and Command, Control, Communications, and Intelligence (C3I) Technologies for Homeland Security and Homeland Defense XII, Baltimore, MD, USA.","DOI":"10.1117\/12.2016342"},{"key":"ref_13","first-page":"1","article-title":"Combination of near-infrared and thermal imaging techniques for the remote and simultaneous measurements of breathing and heart rates under sleep situation","volume":"13","author":"Hu","year":"2018","journal-title":"PLoS ONE"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., and Guadarrama, S. (2017, January 21\u201326). Speed\/accuracy trade-offs for modern convolutional object detectors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.351"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1109\/TPAMI.2015.2448075","article-title":"A fast and accurate unconstrained face detector","volume":"38","author":"Liao","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","unstructured":"Hu, P., and Ramanan, D. (2007, January 21\u201326). Finding tiny faces. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint face detection and alignment using multitask cascaded convolutional networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Redmon, J., and Farhadi, A. (2017, January 21\u201326). Yolo9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.690"},{"key":"ref_20","unstructured":"Lienhart, R., and Maydt, J. (2002, January 22\u201325). An extended set of haar-like features for rapid object detection. Proceedings of the International Conference on Image Processing, Rochester, NY, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Shamia, D., and Chandy, D.A. (2017, January 28\u201329). Analyzing the performance of Viola Jones face detector on the LDHF database. Proceedings of the IEEE International Conference on Siginal Processing and Communication, Coimbatore, India.","DOI":"10.1109\/CSPC.2017.8305860"},{"key":"ref_23","first-page":"1","article-title":"Illumination invariant face detection using hybrid skin segmentation method","volume":"1","author":"Ojo","year":"2013","journal-title":"Eur. J. Comput. Sci. Inf. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1117\/1.2179080","article-title":"Efficient color face detection algorithm under different lighting conditions","volume":"15","author":"Chow","year":"2006","journal-title":"J. Electron. Imaging"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Li, J., Zhang, D., Zhang, K., Hu, K., and Yang, L. (2017, January 11\u201313). Real-time face detection during the night. Proceedings of the 4th International Conference on Systems and Informatics, Hangzhou, China.","DOI":"10.1109\/ICSAI.2017.8248358"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Moazzam, M.G., Parveen, M.R., and Bhuiyan, M.A.-A. (2011). Human face detection under complex lighting conditions. Int. J. Adv. Comput. Sci. Appl., 85\u201390.","DOI":"10.14569\/SpecialIssue.2011.010112"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1016\/S0262-8856(99)00006-2","article-title":"Detecting human faces in color images","volume":"18","author":"Cai","year":"1999","journal-title":"Image Vis. Comput."},{"key":"ref_28","unstructured":"Goldberg, D.E. (1989). Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Laytner, P., Ling, C., and Xiao, Q. (2014, January 9\u201312). Robust face detection from still images. Proceedings of the IEEE Symposium on Computational Intelligence in Biometrics and Identity Management, Orlando, FL, USA.","DOI":"10.1109\/CIBIM.2014.7015446"},{"key":"ref_30","first-page":"93","article-title":"Local enhancement for robust face detection in poor SNR images","volume":"9","author":"Rizwan","year":"2009","journal-title":"Int. J. Comput. Sci. Netw. Secur."},{"key":"ref_31","unstructured":"Comaschi, F., Stuijk, S., Basten, T., and Corporaal, H. (October, January 29). RASW: A run-time adaptive sliding window to improve Viola-Jones object detection. Proceedings of the 7th International Conference on Distributed Smart Cameras, Palm Springs, CA, USA."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3750","DOI":"10.1016\/j.patcog.2014.06.004","article-title":"Nighttime face recognition at large standoff: Cross-distance and cross-spectral matching","volume":"47","author":"Kang","year":"2014","journal-title":"Pattern Recognit."},{"key":"ref_33","first-page":"2799","article-title":"Night time face recognition at large standoff","volume":"2","author":"Chhapre","year":"2017","journal-title":"Int. Res. J. Eng. Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1717","DOI":"10.1109\/TIFS.2012.2213813","article-title":"Long range cross-spectral face recognition: Matching SWIR against visible light images","volume":"7","author":"Nicolo","year":"2012","journal-title":"IEEE Trans. Inf. Forensic Secur."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Bourlai, T., Dollen, J.V., Mavridis, N., and Kolanko, C. (2012, January 23\u201327). Evaluating the efficiency of a night-time, middle-range infrared sensor for applications in human detection and recognition. Proceedings of the SPIE, Infrared Imaging Systems: Design, Analysis, Modeling, and Testing XXIII, Baltimore, MD, USA.","DOI":"10.1117\/12.917831"},{"key":"ref_36","unstructured":"(2018, June 29). Dongguk Night-Time Face Detection Database (DNFD-DB1) and Algorithm Including CNN Model. Available online: http:\/\/dm.dgu.edu\/link.html."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1186","DOI":"10.1109\/TCE.2007.4341603","article-title":"Multi-histogram equalization methods for contrast enhancement and brightness preserving","volume":"53","author":"Menotti","year":"2007","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1016\/j.protcy.2014.08.031","article-title":"Enhancement technique for improving the reliability of disparity map under low light condition","volume":"14","author":"Aditya","year":"2014","journal-title":"Procedia Technol."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Lee, S.-L., and Tseng, C.-C. (2017, January 12\u201314). Color image enhancement using histogram equalization method without changing hue and saturation. Proceedings of the IEEE International Conference on Consumer Electronics, Taipei, Taiwan.","DOI":"10.1109\/ICCE-China.2017.7991117"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1109\/TCE.2007.381734","article-title":"A dynamic histogram equalization for image contrast enhancement","volume":"53","author":"Kabir","year":"2007","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_41","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Parkhi, O.M., Vedaldi, A., and Zisserman, A. (2015, January 7\u201310). Deep face recognition. Proceedings of the British Machine Vision Conference, Swansea, UK.","DOI":"10.5244\/C.29.41"},{"key":"ref_44","unstructured":"Gonzalez, R.C., and Woods, R.E. (2010). Digital Image Processing, Prentice Hall. [3rd ed.]."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_46","unstructured":"Dodge, S., and Karam, L. (August, January 31). A study and comparison of human and deep learning recognition performance under visual distortions. Proceedings of the 26th International Conference on Computer Communication and Networks, Vancouver, BC, Canada."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Dodge, S., and Karam, L. (2016, January 6\u20138). Understanding how image quality affects deep neural networks. Proceedings of the 8th International Conference on Quality of Multimedia Experience, Lisbon, Portugal.","DOI":"10.1109\/QoMEX.2016.7498955"},{"key":"ref_48","unstructured":"(2018, May 02). Webcam C600. Available online: http:\/\/www.logitech.com\/en-us\/support\/5869."},{"key":"ref_49","unstructured":"(2018, March 26). Open Database of Fudan University. Available online: https:\/\/cv.fudan.edu.cn\/_upload\/tpl\/06\/f4\/1780\/template1780\/humandetection.htm."},{"key":"ref_50","unstructured":"(2018, January 31). GeForce GTX 1070. Available online: https:\/\/www.geforce.co.uk\/hardware\/desktop-gpus\/geforce-gtx-1070\/specifications."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014, January 3\u20137). Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA.","DOI":"10.1145\/2647868.2654889"},{"key":"ref_52","unstructured":"(2018, August 17). Matlab 2017a. Available online: https:\/\/www.mathworks.com\/company\/newsroom\/mathworks-announces-release-2017a-of-the-matlab-and-simulink-pro.html."},{"key":"ref_53","unstructured":"(2018, August 17). CUDA. Available online: https:\/\/en.wikipedia.org\/wiki\/CUDA."},{"key":"ref_54","unstructured":"(2018, August 17). CUDNN. Available online: https:\/\/developer.nvidia.com\/cudnn."},{"key":"ref_55","unstructured":"Bottou, L. (2010, January 22\u201327). Large-scale machine learning with stochastic gradient descent. Proceedings of the 19th International Conference on Computational Statistics, Paris, France."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 11\u201318). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_57","unstructured":"(2018, July 05). Precision and Recall. Available online: https:\/\/en.wikipedia.org\/wiki\/Precision_and_recall."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A survey on transfer learning","volume":"22","author":"Pan","year":"2010","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_59","unstructured":"(2018, August 20). Student\u2019s t-Test. Available online: https:\/\/en.wikipedia.org\/wiki\/Student%27s_t-test."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1111\/j.1469-185X.2007.00027.x","article-title":"Effect size, confidence interval and statistical significance: A practical guide for biologists","volume":"82","author":"Nakagawa","year":"2007","journal-title":"Biol. Rev."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1037\/0033-2909.112.1.155","article-title":"A power primer","volume":"112","author":"Cohen","year":"1992","journal-title":"Psychol. Bull."},{"key":"ref_62","unstructured":"(2018, August 26). Darknet: Open Source Neural Networks in C. Available online: https:\/\/pjreddie.com\/darknet\/."},{"key":"ref_63","unstructured":"(2018, August 26). Caffe. Available online: http:\/\/caffe.berkeleyvision.org\/installation.html."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/2995\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:19:24Z","timestamp":1760195964000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/2995"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,7]]},"references-count":63,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2018,9]]}},"alternative-id":["s18092995"],"URL":"https:\/\/doi.org\/10.3390\/s18092995","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2018,9,7]]}}}