{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,18]],"date-time":"2026-01-18T00:57:39Z","timestamp":1768697859440,"version":"3.49.0"},"reference-count":51,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2016,6,30]],"date-time":"2016-06-30T00:00:00Z","timestamp":1467244800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Intelligent surveillance systems have been studied by many researchers. These systems should be operated in both daytime and nighttime, but objects are invisible in images captured by visible light camera during the night. Therefore, near infrared (NIR) cameras, thermal cameras (based on medium-wavelength infrared (MWIR), and long-wavelength infrared (LWIR) light) have been considered for usage during the nighttime as an alternative. Due to the usage during both daytime and nighttime, and the limitation of requiring an additional NIR illuminator (which should illuminate a wide area over a great distance) for NIR cameras during the nighttime, a dual system of visible light and thermal cameras is used in our research, and we propose a new behavior recognition in intelligent surveillance environments. Twelve datasets were compiled by collecting data in various environments, and they were used to obtain experimental results. The recognition accuracy of our method was found to be 97.6%, thereby confirming the ability of our method to outperform previous methods.<\/jats:p>","DOI":"10.3390\/s16071010","type":"journal-article","created":{"date-parts":[[2016,6,30]],"date-time":"2016-06-30T14:24:28Z","timestamp":1467296668000},"page":"1010","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Robust Behavior Recognition in Intelligent Surveillance Environments"],"prefix":"10.3390","volume":"16","author":[{"given":"Ganbayar","family":"Batchuluun","sequence":"first","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Yeong","family":"Kim","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Jong","family":"Kim","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Hyung","family":"Hong","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 100-715, Korea"}]},{"given":"Kang","family":"Park","sequence":"additional","affiliation":[{"name":"Division of Electronics and Electrical Engineering, Dongguk University, 30 Pildong-ro 1-gil, Jung-gu, Seoul 100-715, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2016,6,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"8570","DOI":"10.3390\/s150408570","article-title":"Pedestrian Detection in Far-Infrared Daytime Images Using a Hierarchical Codebook of SURF","volume":"15","author":"Besbes","year":"2015","journal-title":"Sensors"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1947","DOI":"10.1016\/j.patcog.2014.12.013","article-title":"Robust Pedestrian Detection in Thermal Infrared Imagery Using a Shape Distribution Histogram Feature and Modified Sparse Representation Classification","volume":"48","author":"Zhao","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Li, Z., Zhang, J., Wu, Q., and Geers, G. (2010, January 1\u20133). Feature Enhancement Using Gradient Salience on Thermal Image. Proceedings of the International Conference on Digital Image Computing: Techniques and Applications, Sydney, Australia.","DOI":"10.1109\/DICTA.2010.99"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chang, S.L., Yang, F.T., Wu, W.P., Cho, Y.A., and Chen, S.W. (2011, January 8\u201310). Nighttime Pedestrian Detection Using Thermal Imaging Based on HOG Feature. Proceedings of the International Conference on System Science and Engineering, Macao, China.","DOI":"10.1109\/ICSSE.2011.5961992"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3440","DOI":"10.1016\/j.patcog.2015.04.024","article-title":"Novel Outline Features for Pedestrian Detection System with Thermal Images","volume":"48","author":"Lin","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_6","unstructured":"Bertozzi, M., Broggi, A., Rose, M.D., Felisa, M., Rakotomamonjy, A., and Suard, F. (October, January 30). A Pedestrian Detector Using Histograms of Oriented Gradients and a Support Vector Machine Classifier. Proceedings of the IEEE Conference on Intelligent Transportation Systems, Seattle, WA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Li, W., Zheng, D., Zhao, T., and Yang, M. (2012, January 29\u201331). An Effective Approach to Pedestrian Detection in Thermal Imagery. Proceedings of the International Conference on Natural Computation, Chongqing, China.","DOI":"10.1109\/ICNC.2012.6234621"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, W., Wang, Y., Chen, F., and Sowmya, A. (2013, January 15\u201317). A Weakly Supervised Approach for Object Detection Based on Soft-Label Boosting. Proceedings of the IEEE Workshop on Applications of Computer Vision, Tampa, FL, USA.","DOI":"10.1109\/WACV.2013.6475037"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wang, W., Zhang, J., and Shen, C. (2010, January 26\u201329). Improved Human Detection and Classification in Thermal Images. Proceedings of the IEEE International Conference on Image Processing, Hong Kong, China.","DOI":"10.1109\/ICIP.2010.5649946"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Takeda, T., Kuramoto, K., Kobashi, S., and Haya, Y. (2013, January 22\u201324). A Fuzzy Human Detection for Security System Using Infrared Laser Camera. Proceedings of the IEEE International Symposium on Multiple-Valued Logic, Toyama, Japan.","DOI":"10.1109\/ISMVL.2013.4"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"215","DOI":"10.3233\/IFS-2012-0548","article-title":"A Fuzzy Model for Human Fall Detection in Infrared Video","volume":"24","author":"Sokolova","year":"2013","journal-title":"J. Intell. Fuzzy Syst."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Ghiass, R.S., Arandjelovi\u0107, O., Bendada, H., and Maldague, X. (2013, January 4\u20139). Infrared Face Recognition: A Literature Review. Proceedings of the International Joint Conference on Neural Networks, Dallas, TX, USA.","DOI":"10.1109\/IJCNN.2013.6707096"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/j.cviu.2006.06.010","article-title":"Background-subtraction Using Contour-based Fusion of Thermal and Visible Imagery","volume":"106","author":"Davis","year":"2007","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1801","DOI":"10.1016\/j.patcog.2009.11.023","article-title":"Thermal and Reflectance Based Personal Identification Methodology under Variable Illumination","volume":"43","author":"Hammoud","year":"2010","journal-title":"Pattern Recognit."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1007\/s00138-008-0176-5","article-title":"Pedestrian Tracking by Fusion of Thermal-Visible Surveillance Videos","volume":"21","author":"Leykin","year":"2010","journal-title":"Mach. Vis. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1007\/s11263-006-6655-0","article-title":"Multiscale Fusion of Visible and Thermal IR Images for Illumination-Invariant Face Recognition","volume":"71","author":"Kong","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Rahman, S.A., Li, L., and Leung, M.K.H. (2010, January 20\u201322). Human Action Recognition by Negative Space Analysis. Proceedings of the IEEE International Conference on Cyberworlds, Singapore.","DOI":"10.1109\/CW.2010.29"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.jvcir.2012.12.001","article-title":"Human Action Recognition Employing Negative Space Features","volume":"24","author":"Rahman","year":"2013","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1049\/iet-cvi.2011.0185","article-title":"Recognising Human Actions by Analyzing Negative Spaces","volume":"6","author":"Rahman","year":"2012","journal-title":"IET Comput. Vis."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1016\/j.eswa.2013.07.082","article-title":"Fast Action Recognition Using Negative Space Features","volume":"41","author":"Rahman","year":"2014","journal-title":"Expert Syst. Appl."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"9","DOI":"10.5121\/ijcga.2014.4202","article-title":"Performance Analysis of Chain Code Descriptor for Hand Shape Classification","volume":"4","author":"Fating","year":"2014","journal-title":"Int. J. Comput. Graph. Animat."},{"key":"ref_22","first-page":"19","article-title":"Human Shape Recognition Using Fourier Descriptor","volume":"2","author":"Tahir","year":"2009","journal-title":"J. Electr. Electron. Syst. Res."},{"key":"ref_23","unstructured":"Toth, D., and Aach, T. (2003, January 17\u201319). Detection and Recognition of Moving Objects Using Statistical Motion Detection and Fourier Descriptors. Proceedings of the IEEE International Conference on Image Analysis and Processing, Mantova, Italy."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Harding, P.R.G., and Ellis, T. (2004, January 23\u201326). Recognizing Hand Gesture Using Fourier Descriptors. Proceedings of the IEEE International Conference on Pattern Recognition, Washington, DC, USA.","DOI":"10.1109\/ICPR.2004.1334523"},{"key":"ref_25","first-page":"14","article-title":"Signature Recognition Using Multi Scale Fourier Descriptor and Wavelet Transform","volume":"7","author":"Ismail","year":"2010","journal-title":"Int. J. Comput. Sci. Inf. Secur."},{"key":"ref_26","unstructured":"Sun, X., Chen, M., and Hauptmann, A. (2009, January 20\u201326). Action Recognition via Local Descriptors and Holistic Features. Proceedings of the Workshop on Computer Vision and Pattern Recognition for Human Communicative Behavior Analysis, Miami, FL, USA."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sch\u00fcldt, C., Laptev, I., and Caputo, B. (2004, January 23\u201326). Recognizing Human Actions: A Local SVM Approach. Proceedings of the IEEE International Conference on Pattern Recognition, Cambridge, UK.","DOI":"10.1109\/ICPR.2004.1334462"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Laptev, I., and Lindeberg, T. (2003, January 13\u201316). Space-time Interest Points. Proceedings of the IEEE International Conference on Computer Vision, Nice, France.","DOI":"10.1109\/ICCV.2003.1238378"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/s11263-005-1838-7","article-title":"On Space-Time Interest Points","volume":"64","author":"Laptev","year":"2005","journal-title":"Int. J. Comput. Vis."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wang, L., Qiao, Y., and Tang, X. (2013, January 23\u201328). Motionlets: Mid-level 3D Parts for Human Motion Recognition. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Porland, OR, USA.","DOI":"10.1109\/CVPR.2013.345"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Arandjelovi\u0107, O. (2011, January 26\u201328). Contextually Learnt Detection of Unusual Motion-Based Behaviour in Crowded Public Spaces. Proceedings of the 26th International Symposium on Computer and Information Sciences, London, UK.","DOI":"10.1007\/978-1-4471-2155-8_51"},{"key":"ref_32","unstructured":"Eum, H., Lee, J., Yoon, C., and Park, M. (November, January 30). Human Action Recognition for Night Vision Using Temporal Templates with Infrared Thermal Camera. Proceedings of the International Conference on Ubiquitous Robots and Ambient Intelligence, Jeju Island, Korea."},{"key":"ref_33","unstructured":"Chunli, L., and Kejun, W. (2010, January 30\u201331). A Behavior Classification Based on Enhanced Gait Energy Image. Proceedings of the IEEE International Conference on Network and Digital Society, Wenzhou, China."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Liu, J., and Zheng, N. (2007, January 2\u20135). Gait History Image: A Novel Temporal Template for Gait Recognition. Proceedings of the IEEE International Conference on Multimedia and Expo, Beijing, China.","DOI":"10.1109\/ICME.2007.4284737"},{"key":"ref_35","first-page":"1","article-title":"Human Action Recognition Using Ordinal Measure of Accumulated Motion","volume":"2010","author":"Kim","year":"2010","journal-title":"Eur. J. Adv. Signal Process."},{"key":"ref_36","unstructured":"Han, J., and Bhanu, B. (2005, January 25). Human Activity Recognition in Thermal Infrared Imagery. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, San Diego, CA, USA."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"973","DOI":"10.1016\/j.patcog.2010.10.011","article-title":"Gait Flow Image: A Silhouette-based Gait Representation for Human Identification","volume":"44","author":"Lam","year":"2011","journal-title":"Pattern Recognit."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Wong, W.K., Lim, H.L., Loo, C.K., and Lim, W.S. (2010, January 7\u201310). Home Alone Faint Detection Surveillance System Using Thermal Camera. Proceedings of International Conference on Computer Research and Development, Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICCRD.2010.163"},{"key":"ref_39","unstructured":"Youssef, M.M. (2011). Hull Convexity Defect Features for Human Action Recognition. [Ph.D. Thesis, University of Dayton]."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhang, D., Wang, Y., and Bhanu, B. (2010, January 13\u201318). Ethnicity Classification Based on Gait Using Multi-view Fusion. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, San Francisco, CA, USA.","DOI":"10.1109\/CVPRW.2010.5544614"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Rusu, R.B., Bandouch, J., Marton, Z.C., Blodow, N., and Beetz, M. (2008, January 1\u20133). Action Recognition in Intelligent Environments Using Point Cloud Features Extracted from Silhouette Sequences. Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication, Munich, Germany.","DOI":"10.1109\/ROMAN.2008.4600677"},{"key":"ref_42","unstructured":"Kim, S.-H., Hwang, J.-H., and Jung, I.-K. (2011, January 1\u20133). Object Tracking System with Human Extraction and Recognition of Behavior. Proceedings of the International Conference on Management and Artificial Intelligence, Bali, Indonesia."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"10580","DOI":"10.3390\/s150510580","article-title":"Robust Pedestrian Detection by Combining Visible and Thermal Infrared Cameras","volume":"15","author":"Lee","year":"2015","journal-title":"Sensors"},{"key":"ref_44","unstructured":"Tau 2. Available online: http:\/\/mds-flir.com\/datasheet\/FLIR_Tau2_Family_Brochure.pdf."},{"key":"ref_45","unstructured":"Infrared Lens. Available online: http:\/\/www.irken.co.kr\/."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"6763","DOI":"10.3390\/s150306763","article-title":"Human Detection Based on the Generation of a Background Image by Using a Far-Infrared Light Camera","volume":"15","author":"Jeon","year":"2015","journal-title":"Sensors"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"2247","DOI":"10.1109\/TPAMI.2007.70711","article-title":"Actions as Space-Time Shapes","volume":"29","author":"Gorelick","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1007\/s11263-008-0142-8","article-title":"Searching for Complex Human Activities with No Visual Examples","volume":"80","author":"Ikizler","year":"2008","journal-title":"Int. J. Comput. Vis."},{"key":"ref_49","unstructured":"The LTIR Dataset v1.0. Available online: http:\/\/www.cvl.isy.liu.se\/en\/research\/datasets\/ltir\/version1.0\/."},{"key":"ref_50","unstructured":"Precision and Recall. Available online: https:\/\/en.wikipedia.org\/wiki\/Precision_and_recall."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Gehrig, D., Kuehne, H., Woerner, A., and Schultz, T. (2009, January 7\u201310). HMM-based Human Motion Recognition with Optical Flow Data. Proceedings of the IEEE-RAS International Conference on Humanoid Robots, Paris, France.","DOI":"10.1109\/ICHR.2009.5379546"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/7\/1010\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:25:09Z","timestamp":1760210709000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/7\/1010"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,6,30]]},"references-count":51,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2016,7]]}},"alternative-id":["s16071010"],"URL":"https:\/\/doi.org\/10.3390\/s16071010","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,6,30]]}}}