{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T05:54:50Z","timestamp":1790229290505,"version":"4.1.0"},"reference-count":44,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2019,6,27]],"date-time":"2019-06-27T00:00:00Z","timestamp":1561593600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Elbit Systems of America","award":["through a contract with the University of Texas at Dallas"],"award-info":[{"award-number":["through a contract with the University of Texas at Dallas"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>This paper presents a semi-supervised faster region-based convolutional neural network (SF-RCNN) approach to detect persons and to classify the load carried by them in video data captured from distances several miles away via high-power lens video cameras. For detection, a set of computationally efficient image processing steps are considered to identify moving areas that may contain a person. These areas are then passed onto a faster RCNN classifier whose convolutional layers consist of ResNet50 transfer learning. Frame labels are obtained in a semi-supervised manner for the training of the faster RCNN classifier. For load classification, another convolutional neural network classifier whose convolutional layers consist of GoogleNet transfer learning is used to distinguish a person carrying a bundle from a person carrying a long arm. Despite the challenges associated with the video dataset examined in terms of the low resolution of persons, the presence of heat haze, and the shaking of the camera, it is shown that the developed approach outperforms the faster RCNN approach.<\/jats:p>","DOI":"10.3390\/make1030044","type":"journal-article","created":{"date-parts":[[2019,6,27]],"date-time":"2019-06-27T08:19:13Z","timestamp":1561623553000},"page":"756-767","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":38,"title":["Semi-Supervised Faster RCNN-Based Person Detection and Load Classification for Far Field Video Surveillance"],"prefix":"10.3390","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0666-5551","authenticated-orcid":false,"given":"Haoran","family":"Wei","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Texas at Dallas, Richardson 75080, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nasser","family":"Kehtarnavaz","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Texas at Dallas, Richardson 75080, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,6,27]]},"reference":[{"key":"ref_1","unstructured":"(2019, February 20). Markets and Markets. Available online: https:\/\/www.marketsandmarkets.com\/Market-Reports\/video-surveillance-market-645.html."},{"key":"ref_2","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of Oriented Gradients for Human Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Dollar, P., Wojek, C., Shiele, B., and Perona, P. (2009, January 20\u201325). Pedestrian Detection: A Benchmark. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206631"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.1109\/TPAMI.2014.2300479","article-title":"Fast Feature Pyramids for Object Detection","volume":"36","author":"Dollar","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3781","DOI":"10.1007\/s11042-018-6057-7","article-title":"Combining static and dynamic features for real-time moving pedestrian detection","volume":"78","author":"Jiang","year":"2019","journal-title":"Multimed. Tools Appl."},{"key":"ref_6","unstructured":"Xiao, F., Liu, B., and Li, R. (2019). Pedestrian object detection with fusion of visual attention mechanism and semantic computation. Multimed. Tools Appl., 1\u201315."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"15229","DOI":"10.1007\/s11042-015-2455-2","article-title":"(2016) Fast multi-feature pedestrian detection algorithm based on histogram of oriented gradient using discrete wavelet transform","volume":"75","author":"Hong","year":"2016","journal-title":"Multimed. Tools Appl."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yang, Y., Liu, W., Wang, Y., and Cai, Y. (2012, January 19\u201322). Research on the algorithm of pedestrian recognition in front of the vehicle based on SVM. Proceedings of the 11th International Symposium on Distributed Computing and Applications to Business, Engineering and Science, DCABES 2012, Guilin, China.","DOI":"10.1109\/DCABES.2012.108"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1109\/TITS.2015.2479925","article-title":"Multiple Sensor Fusion and Classification for Moving Object Detection and Tracking","volume":"17","author":"Aycard","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_10","unstructured":"Wang, X., Han, T.X., and Yan, S. (October, January 29). An HOG-LBP human detector with partial occlusion handling. Proceedings of the IEEE 12th International Conference on Computer Vision, Kyoto, Japan."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Roncancio, H., Hernandes, A.C., and Becker, M. (2012, January 2\u20134). Vision-based system for pedestrian recognition using a tuned SVM classifier. Proceedings of the Workshop on Engineering Applications, Bogota, Columbia.","DOI":"10.1109\/WEA.2012.6220095"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_14","first-page":"1137","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). SSD: Single Shot MultiBox Detector. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Song, H., Choi, I.K., Ko, M.S., Bae, J., Kwak, S., and Yoo, J. (2018, January 24\u201327). Vulnerable pedestrian detection and tracking using deep learning. Proceedings of the 2018 International Conference on Electronics, Information, and Communication (ICEIC), Honolulu, HI, USA.","DOI":"10.23919\/ELINFOCOM.2018.8330547"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hou, Y.L., Song, Y., Hao, X., Shen, Y., and Qian, M. (2017, January 22\u201325). Multispectral pedestrian detection based on deep convolutional neural networks. Proceedings of the IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Xiamen, China.","DOI":"10.1109\/ICSPCC.2017.8242507"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Gonz\u00e1lez, A., Fang, Z., Socarras, Y., Serrat, J., V\u00e1zquez, D., Xu, J., and L\u00f3pez, A.M. (2016). Pedestrian Detection at Day\/Night Time with Visible and FIR Cameras: A Comparison. Sensors, 16.","DOI":"10.3390\/s16060820"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"814","DOI":"10.1109\/TPAMI.2015.2465908","article-title":"What Makes for Effective Detection Proposals?","volume":"38","author":"Hosang","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Brazil, G., Yin, X., and Liu, X. (2017, January 22\u201329). Illuminating Pedestrians via Simultaneous Detection and Segmentation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.530"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Wei, H., Laszewski, M., and Kehtarnavaz, N. (2018, January 2\u201312). Deep Learning-Based Person Detection and Classification for Far Field Video Surveillance. Proceedings of the 13th IEEE Dallas Circuits and Systems Conference, Dallas, TX, USA.","DOI":"10.1109\/DCAS.2018.8620111"},{"key":"ref_23","first-page":"743","article-title":"Pedestrian detection: An evaluation of the state of the art","volume":"34","author":"Wojek","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/j.cosrev.2014.04.001","article-title":"Traditional and recent approaches in background modeling for foreground detection: An overview","volume":"11","author":"Bouwmans","year":"2014","journal-title":"Comput. Sci. Rev."},{"key":"ref_25","unstructured":"Stauffer, C., and Grimson, W.E.L. (1999, January 23\u201325). Adaptive background mixture models for real-time tracking. Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Fort Collins, CO, USA."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Elgammal, A., Harwood, D., and Davis, L. (2000). Non-Parametric Model for Background Subtraction. Computer Vision-ECCV 2000, Springer.","DOI":"10.1007\/3-540-45053-X_48"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Heikkil\u00e4, M., Pietik\u00e4inen, M., and Heikkil\u00e4, J. (2004, January 7\u20139). A texture-basedmethod for detectingmoving objects. Proceedings of the British Machine Vision Conference (BMVC), Kingston, UK.","DOI":"10.5244\/C.18.21"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"198","DOI":"10.2197\/ipsjtcva.3.198","article-title":"Statistical Local Difference Pattern for Background Modeling","volume":"3","author":"Yoshinaga","year":"2011","journal-title":"IPSJ Trans. Comput. Vis. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1007\/s00138-018-0993-0","article-title":"Unsupervised Deep Context Prediction for Background Estimation and Foreground Segmentation","volume":"30","author":"Sultana","year":"2019","journal-title":"Mach. Vision Appl."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Minematsu, T., Shimada, A., Uchiyama, H., and Taniguchi, R.I. (2018). Analytics of Deep Neural Network-based Background Subtraction. J. Imaging, 4.","DOI":"10.3390\/jimaging4060078"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.neunet.2019.04.024","article-title":"Deep neural network concepts for background subtraction: A systematic review and comparative evaluation","volume":"117","author":"Bouwmans","year":"2019","journal-title":"Neural Netw."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1016\/j.patcog.2017.09.040","article-title":"A deep convolutional neural network for video sequence background subtraction","volume":"76","author":"Babaee","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid scene parsing network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_34","first-page":"771","article-title":"A short introduction to boosting","volume":"14","author":"Freund","year":"1999","journal-title":"J. JSAI"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Dong, P., and Wang, W. (2016, January 27\u201330). Better region proposals for pedestrian detection with R-CNN. Proceedings of the IEEE Visual Communications and Image Processing, Chengdu, China.","DOI":"10.1109\/VCIP.2016.7805452"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zitnick, C.L., and Dollar, P. (2014, January 6\u201312). Edge Boxes: Locating Object Proposals from Edges. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_26"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1007\/s11263-013-0620-5","article-title":"Selective search for object recognition","volume":"104","author":"Uijlings","year":"2013","journal-title":"Int. J. Comput. Vis."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A Survey on Transfer Learning","volume":"22","author":"Pan","year":"2010","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_40","unstructured":"(2019, February 20). ImageNet. Available online: http:\/\/www.image-net.org."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zhu, X., and Goldberg, A. (2009). Introduction to Semi-Supervised Learning. Synthesis lectures on Artificial Intelligence and Machine Learning, Morgan & Claypool.","DOI":"10.1007\/978-3-031-01548-9"},{"key":"ref_42","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G. (2012, January 3\u20136). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada, USA."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going Deeper with Convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_44","unstructured":"(2019, February 20). Mathworks. Available online: https:\/\/www.mathworks.com\/help\/vision\/examples\/object-detection-using-faster-r-cnn-deep-learning.html."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/1\/3\/44\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:01:43Z","timestamp":1760187703000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/1\/3\/44"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,27]]},"references-count":44,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2019,9]]}},"alternative-id":["make1030044"],"URL":"https:\/\/doi.org\/10.3390\/make1030044","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,6,27]]}}}