{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,6]],"date-time":"2026-01-06T12:46:57Z","timestamp":1767703617652,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2015,5,5]],"date-time":"2015-05-05T00:00:00Z","timestamp":1430784000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the development of intelligent surveillance systems, the need for accurate detection of pedestrians by cameras has increased. However, most of the previous studies use a single camera system, either a visible light or thermal camera, and their performances are affected by various factors such as shadow, illumination change, occlusion, and higher background temperatures. To overcome these problems, we propose a new method of detecting pedestrians using a dual camera system that combines visible light and thermal cameras, which are robust in various outdoor environments such as mornings, afternoons, night and rainy days. Our research is novel, compared to previous works, in the following four ways: First, we implement the dual camera system where the axes of visible light and thermal cameras are parallel in the horizontal direction. We obtain a geometric transform matrix that represents the relationship between these two camera axes. Second, two background images for visible light and thermal cameras are adaptively updated based on the pixel difference between an input thermal and pre-stored thermal background images. Third, by background subtraction of thermal image considering the temperature characteristics of background and size filtering with morphological operation, the candidates from whole image (CWI) in the thermal image is obtained. The positions of CWI (obtained by background subtraction and the procedures of shadow removal, morphological operation, size filtering, and filtering of the ratio of height to width) in the visible light image are projected on those in the thermal image by using the geometric transform matrix, and the searching regions for pedestrians are defined in the thermal image. Fourth, within these searching regions, the candidates from the searching image region (CSI) of pedestrians in the thermal image are detected. The final areas of pedestrians are located by combining the detected positions of the CWI and CSI of the thermal image based on OR operation. Experimental results showed that the average precision and recall of detecting pedestrians are 98.13% and 88.98%, respectively.<\/jats:p>","DOI":"10.3390\/s150510580","type":"journal-article","created":{"date-parts":[[2015,5,5]],"date-time":"2015-05-05T10:43:16Z","timestamp":1430822596000},"page":"10580-10615","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":45,"title":["Robust Pedestrian Detection by Combining Visible and Thermal Infrared Cameras"],"prefix":"10.3390","volume":"15","author":[{"given":"Ji","family":"Lee","sequence":"first","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 26 Pil-dong 3-ga, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Jong-Suk","family":"Choi","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 26 Pil-dong 3-ga, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Eun","family":"Jeon","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 26 Pil-dong 3-ga, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Yeong","family":"Kim","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 26 Pil-dong 3-ga, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Toan","family":"Le","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 26 Pil-dong 3-ga, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Kwang","family":"Shin","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 26 Pil-dong 3-ga, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Hyeon","family":"Lee","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 26 Pil-dong 3-ga, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Kang","family":"Park","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 26 Pil-dong 3-ga, Jung-gu, Seoul 100-715, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2015,5,5]]},"reference":[{"key":"ref_1","unstructured":"Lipton, A.J., Fujiyoshi, H., and Patil, R.S. (1998, January 19\u201321). Moving Target Classification and Tracking from Real-time Video. Proceedings of the IEEE Workshop on\u00a0Applications of Computer Vision, Princeton, NJ, USA."},{"key":"ref_2","unstructured":"Oren, M., Papageorgiou, C., Sinha, P., Osuna, E., and Poggio, T. (1997, January 17\u201319). Pedestrian Detection Using Wavelet Templates. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Viola, P., Jones, M.J., and Snow, D. (2003, January 13\u201316). Detecting Pedestrians Using Patterns of Motion and Appearance. Proceedings of the IEEE International Conference on Computer Vision, Nice, France.","DOI":"10.1109\/ICCV.2003.1238422"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1007\/978-3-540-24670-1_6","article-title":"Human Detection Based on a Probabilistic Assembly of Robust Part Detectors","volume":"3021","author":"Mikolajczyk","year":"2004","journal-title":"Lect. Notes Comput. Sci."},{"key":"ref_5","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of Oriented Gradients for Human Detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, W., Zhang, J., and Shen, C. (2010, January 26\u201329). Improved Human Detection and Classification in Thermal Images. Proceedings of the IEEE International Conference on Image Processing, Hong Kong, China.","DOI":"10.1109\/ICIP.2010.5649946"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Wang, W., Wang, Y., Chen, F., and Sowmya, A. (2013, January 15\u201317). A Weakly Supervised Approach for Object Detection Based on Soft-Label Boosting. Proceedings of the IEEE Workshop on Applications of Computer Vision, Tampa, FL, USA.","DOI":"10.1109\/WACV.2013.6475037"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, W., Zheng, D., Zhao, T., and Yang, M. (2012, January 29\u201331). An Effective Approach to Pedestrian Detection in Thermal Imagery. Proceedings of the International Conference on Natural Computation, Chongqing, China.","DOI":"10.1109\/ICNC.2012.6234621"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Neagoe, V.-E., Ciotec, A.-D., and Barar, A.-P. (2012, January 21\u201323). A Concurrent Neural Network Approach to Pedestrian Detection in Thermal Imagery. Proceedings of the International Conference on Communications, Bucharest, Romania.","DOI":"10.1109\/ICComm.2012.6262539"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Olmeda, D., Armingol, J.M., and Escalera, A.D.L. (2012, January 7\u201312). Discrete Features for Rapid Pedestrian Detection in Infrared Images. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal.","DOI":"10.1109\/IROS.2012.6385928"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lin, C.-F., Lin, S.-F., Hwang, C.-H., and Chen, Y.-C. (2014, January 12\u201315). Real-time Pedestrian Detection System with Novel Thermal Features at Night. Proceedings of the IEEE International Instrumentation and Measurement Technology Conference, Montevideo, Uruguay.","DOI":"10.1109\/I2MTC.2014.6860962"},{"key":"ref_12","unstructured":"Davis, J.W., and Sharma, V. (2005, January 20). Fusion-based Background-subtraction Using Contour Saliency. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition\u2014Workshops, San Diego, CA, USA."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"347","DOI":"10.3233\/ICA-130441","article-title":"Pedestrian Detection in Far Infrared Images","volume":"20","author":"Olmeda","year":"2013","journal-title":"Integr. Comput.-Aided Eng."},{"key":"ref_14","unstructured":"Bertozzi, M., Broggi, A., Rose, M.D., Felisa, M., Rakotomamonjy, A., and Suard, F. (October, January 30). A Pedestrian De-tector Using Histograms of Oriented Gradients and a Support Vector Machine Classifier. Proceedings of the IEEE Conference on Intelligent Transportation Systems, Seattle, WA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Li, Z., Zhang, J., Wu, Q., and Geers, G. (2010, January 1\u20133). Feature Enhancement Using Gradient Salience on Thermal Image. Proceedings of the International Conference on Digital Image Computing: Techniques and Applications, Sydney, Australia.","DOI":"10.1109\/DICTA.2010.99"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Chang, S.-L., Yang, F.-T., Wu, W.-P., Cho, Y.-A., and Chen, S.-W. (2011, January 8\u201310). Nighttime Pedestrian Detection Using Thermal Imaging Based on HOG Feature. Proceedings of the International Conference on System Science and Engineering, Macao, China.","DOI":"10.1109\/ICSSE.2011.5961992"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1016\/j.cviu.2006.07.016","article-title":"Pedestrian Detection by Means of Far-infrared Stereo Vision","volume":"106","author":"Bertozzi","year":"2007","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_18","unstructured":"Zhang, L., Wu, B., and Nevatia, R. (2007, January 17\u201322). Pedestrian Detection in Infrared Images Based on Local Shape Features. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, MN, USA."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Davis, J.W., and Sharma, V. (2004, January 23\u201326). Robust Detection of People in Thermal Imagery. Proceedings of the International Conference on Pattern Recognition, Cambridge, UK.","DOI":"10.1109\/ICPR.2004.1333872"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Davis, J.W., and Keck, M.A. (2005, January 5\u20137). A Two-Stage Template Approach to Person Detection in Thermal Imagery. Proceedings of the IEEE Workshop on Applications of Computer Vision, Breckenridge, CO, USA.","DOI":"10.1109\/ACVMOT.2005.14"},{"key":"ref_21","unstructured":"Latecki, L.J., Miezianko, R., and Pokrajac, D. (2005, January 15\u201316). Tracking Motion Objects in Infrared Videos. Proceedings of the IEEE International Conference on Advanced Video and Signal Based Surveillance, Como, Italy."},{"key":"ref_22","unstructured":"Dai, C., Zheng, Y., and Li, X. (2005, January 25). Layered Representation for Pedestrian Detection and Tracking in Infrared Imagery. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition\u2014Workshops, San Diego, CA, USA."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1016\/j.cviu.2006.08.009","article-title":"Pedestrian Detection and Tracking in Infrared Imagery Using Shape and Appearance","volume":"106","author":"Dai","year":"2007","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_24","unstructured":"Calafut, M. Multiple-Object Tracking in the Infrared. Available online: https:\/\/stacks.stanford.edu\/file\/druid:sg108fn0681\/Calafut_Multiple_Object_Tracking_in_Infrared.pdf."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1606","DOI":"10.4304\/jcp.5.10.1606-1613","article-title":"Real Time Pedestrian Tracking Using Thermal Infrared Imagery","volume":"5","author":"Li","year":"2010","journal-title":"J. Comput."},{"key":"ref_26","unstructured":"Zhao, J., and Cheung, S.-C.S. (October, January 27). Human Segmentation by Fusing Visible-light and Thermal Imaginary. Proceedings of the IEEE International Conference on Computer Vision Workshops, Kyoto, Japan."},{"key":"ref_27","unstructured":"Chen, Y., and Han, C. (2008, January 25\u201327). Night-time Pedestrian Detection by Visual-infrared Video Fusion. Proceedings of the World Congress on Intelligent Control and Automation, Chongqing, China."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"St-Laurent, L., Pr\u00e9vost, D., and Maldague, X. (2006, January 27\u201330). Thermal Imaging for Enhanced Foreground-background Segmentation. Proceedings of the International Conference on Quantitative InfraRed Thermography, Padova, Italy.","DOI":"10.21611\/qirt.2006.065"},{"key":"ref_29","unstructured":"ICI 7320 Scientific Specifications. Available online: http:\/\/www.infraredcamerasinc.com\/Thermal-Cameras\/Fix-Mounted-Thermal-Cameras\/ICI7320_S_fix-mounted_thermal_camera.html."},{"key":"ref_30","unstructured":"Webcam C600. Available online: https:\/\/support.logitech.com\/en_us\/product\/5869."},{"key":"ref_31","unstructured":"Gonzalez, R.C., and Woods, R.E. (2010). Digital Image Processing, Prentice Hall. [3rd ed.]."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"918","DOI":"10.1109\/TPAMI.2003.1206520","article-title":"Detecting Moving Shadows: Algorithms and Evaluation","volume":"25","author":"Prati","year":"2003","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","unstructured":"OTCBVS Benchmark Dataset Collection. Available online: http:\/\/www.cse.ohio-state.edu\/otcbvs-bench\/."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4663","DOI":"10.1109\/TIP.2014.2346013","article-title":"A Novel Video Dataset for Change Detection Benchmarking","volume":"23","author":"Goyette","year":"2014","journal-title":"IEEE Trans. Image Process."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The Pascal Visual Object Classes (VOC) Challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_36","unstructured":"Precision and recall. Available online: http:\/\/en.wikipedia.org\/wiki\/Precision_and_recall."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zivkovic, Z. (2004, January 23\u201326). Improved Adaptive Gaussian Mixture Model for Background Subtraction. Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK.","DOI":"10.1109\/ICPR.2004.1333992"},{"key":"ref_38","unstructured":"Tau 2 Uncooled Cores. Available online: http:\/\/www.flir.com\/cores\/display\/?id=54717."},{"key":"ref_39","unstructured":"Serrano-Cuerda, J., Fern\u00e1ndez-Caballero, A., and L\u00f3pez, M.T. Robust Human Detection through Fusion of Color and Infrared Video. Available online: http:\/\/elcvia.cvc.uab.es\/article\/view\/604."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"331","DOI":"10.3390\/app4030331","article-title":"Selection of a Visible-Light vs. Thermal Infrared Sensor in Dynamic Environments Based on Confidence Measures","volume":"4","year":"2014","journal-title":"Appl. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Castillo, J.C., Serrano-Cuerda, J., Sokolova, M.V., Fern\u00e1ndez-Caballero, A., Costa, A., and Novais, P. (2012, January 26\u201329). Multispectrum Video for Proactive Response in Intelligent Environments. Proceedings of the Eighth International Conference on Intelligent Environments, Guanajuato, Mexico.","DOI":"10.1109\/IE.2012.73"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Gascue\u00f1a, J.M., Serrano-Cuerda, J., Castillo, J.C., Fern\u00e1ndez-Caballero, A., and L\u00f3pez, M.T. A Multi-agent System for Infrared and Color Video Fusion. Adv. Intell. Syst. Comput., 293, 131\u2013138.","DOI":"10.1007\/978-3-319-07476-4_16"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"6666","DOI":"10.3390\/s140406666","article-title":"Thermal-Infrared Pedestrian ROI Extraction through Thermal and Motion Information Fusion","volume":"14","year":"2014","journal-title":"Sensors"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/15\/5\/10580\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:45:48Z","timestamp":1760215548000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/15\/5\/10580"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,5,5]]},"references-count":43,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2015,5]]}},"alternative-id":["s150510580"],"URL":"https:\/\/doi.org\/10.3390\/s150510580","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2015,5,5]]}}}