{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T08:47:20Z","timestamp":1771577240865,"version":"3.50.1"},"reference-count":49,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,2,15]],"date-time":"2020-02-15T00:00:00Z","timestamp":1581724800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the National Key Research and Development Program of China","award":["2018YFC0831003"],"award-info":[{"award-number":["2018YFC0831003"]}]},{"DOI":"10.13039\/501100001809","name":"the National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61972233"],"award-info":[{"award-number":["61972233"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"the National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61902225"],"award-info":[{"award-number":["61902225"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Although there are many algorithms to track people that are walking, existing methods mostly fail to cope with occluded bodies in the setting of multi-person tracking with one camera. In this paper, we propose a method to use people\u2019s shadows as a clue to track them instead of treating shadows as mere noise. We introduce a novel method to track multiple people by fusing shadow data from the RGB image with skeleton data, both of which are captured by a single RGB Depth (RGB-D) camera. Skeletal tracking provides the positions of people that can be captured directly, while their shadows are used to track them when they are no longer visible. Our experiments confirm that this method can efficiently handle full occlusions. It thus has substantial value in resolving the occlusion problem in multi-person tracking, even with other kinds of cameras.<\/jats:p>","DOI":"10.3390\/s20041056","type":"journal-article","created":{"date-parts":[[2020,2,20]],"date-time":"2020-02-20T03:20:03Z","timestamp":1582168803000},"page":"1056","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Employing Shadows for Multi-Person Tracking Based on a Single RGB-D Camera"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6005-1059","authenticated-orcid":false,"given":"Wei","family":"Gai","sequence":"first","affiliation":[{"name":"School of Software, Shandong University, Jinan 250101, China"}]},{"given":"Meng","family":"Qi","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China"}]},{"given":"Mingcong","family":"Ma","sequence":"additional","affiliation":[{"name":"School of Software, Shandong University, Jinan 250101, China"}]},{"given":"Lu","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Software, Shandong University, Jinan 250101, China"}]},{"given":"Chenglei","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Software, Shandong University, Jinan 250101, China"},{"name":"Engineering Research Center of Digital Media Technology, MOE, Jinan 250101, China"}]},{"given":"Juan","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Software, Shandong University, Jinan 250101, China"}]},{"given":"Yulong","family":"Bian","sequence":"additional","affiliation":[{"name":"School of Software, Shandong University, Jinan 250101, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2930-2059","authenticated-orcid":false,"given":"Gerard","family":"de Melo","sequence":"additional","affiliation":[{"name":"Department of Computer Science 110 Frelinghuysen Road Rutgers, The State University of New Jersey, Piscataway, NJ 08854-8019, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4108-1391","authenticated-orcid":false,"given":"Shijun","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Software, Shandong University, Jinan 250101, China"}]},{"given":"Xiangxu","family":"Meng","sequence":"additional","affiliation":[{"name":"School of Software, Shandong University, Jinan 250101, China"},{"name":"Engineering Research Center of Digital Media Technology, MOE, Jinan 250101, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,2,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Uson, M., Arthur, K., Whitton, M.C., Bastos, R., Steed, A., Slater, M., and Brooks, F.P.B. (1999, January 1). Walking > walking-in-place > flying, in virtual environments. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, New York, NY, USA.","DOI":"10.1145\/311535.311589"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Gai, W., Yang, C.L., Bian, Y.L., Shen, C., Meng, X.X., Wang, L., Liu, J., Dong, M.D., Niu, C.J., and Lin, C. (2017, January 6\u201311). Supporting easy physical-to-virtual creation of mobile vr maze games: A new genre. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.","DOI":"10.1145\/3025453.3025494"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Marwecki, S., Brehm, M., Wagner, L., Cheng, L.P., Mueller, F.F., and Basudisch, P. (2018, January 21\u201326). Virtual space\u2013overloading physical space with multiple virtual reality users. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/3173574.3173815"},{"key":"ref_4","unstructured":"Delden, R.V., Moreno, A., Poppe, R., Reidsma, D., and Heylen, D. (2017, January 6\u201311). A thing of beauty: Steering behavior in an interactive playground. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Sivalingam, R., Cherian, A., Fasching, J., Walczak, N., Bird, N., Morellas, V., Murphy, B., Cullen, K., Lim, K., and Sapiro, G. (2012, January 14\u201318). A multi-sensor visual tracking system for behavior monitoring of at-risk children. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6225280"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Park, J., An, K., Kim, D., and Choi, J. (November, January 30). Multiple human tracking using multiple kinects for an attendance check system of a smart class. Proceedings of the 2013 10th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Jeju, Korea.","DOI":"10.1109\/URAI.2013.6677494"},{"key":"ref_7","unstructured":"Yu, H.D., Li, H.Y., Sun, W.S., Gai, W., Cui, T.T., Wang, C.T., Guan, D.D., Yang, Y.J., Yang, C.L., and Zeng, W. (December, January 30). A two-view VR shooting theater system. Proceedings of the 13th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry, Shenzhen, China."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"982","DOI":"10.1109\/TCSVT.2018.2828606","article-title":"A robust moving object detection in multi-scenario big data for video surveillance","volume":"29","author":"Chen","year":"2018","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yilmaz, A., Javed, O., and Shah, M. (2006). Object tracking: A survey. ACM Comput. Surv. (CSUR), 38.","DOI":"10.1145\/1177352.1177355"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1109\/TCE.2012.6227420","article-title":"Employing a RGB-D sensor for real-time tracking of humans across multiple re-entries in a smart environment","volume":"58","author":"Han","year":"2012","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Nebeling, M., Teunissen, E., Husmann, M., and Norrie, M.C. (2014, January 17\u201320). XDKinect: Development framework for cross-device interaction using kinect. Proceedings of the 2014 ACM SIGCHI Symposium on Engineering Interactive Computing Systems, Rome, Italy.","DOI":"10.1145\/2607023.2607024"},{"key":"ref_12","unstructured":"Krumm, J., Harris, S., Meyers, B., Brumitt, B., Hale, M., and Shafer, S. (2000, January 1). Multi-camera multi-person tracking for easyliving. Proceedings of the Third IEEE International Workshop on Visual Surveillance, Dublin, Ireland."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.jvcir.2015.11.012","article-title":"People tracking in an environment with multiple depth cameras: A skeleton-based pairwise trajectory matching scheme","volume":"35","author":"Sun","year":"2016","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_14","first-page":"930","article-title":"Multi-user identification based on double views coupling in cooperative interaction scenarios","volume":"41","author":"Yang","year":"2015","journal-title":"J. Beijing Univ. Aeronaut. Astronaut."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Bellotto, N., Sommerlade, E., Benfold, B., Bibby, C., Reid, I., Roth, D., Fernamdez, C., van Gool, L., and Gonzalez, J. (September, January 30). A distributed camera system for multi-resolution surveillance. Proceedings of the 2009 Third ACM\/IEEE International Conference on Distributed Smart Cameras (ICDSC), Como, Italy.","DOI":"10.1109\/ICDSC.2009.5289413"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wu, C.J., Houben, S., and Marquardt, N. (2017, January 6\u201311). Eaglesense: Tracking people and devices in interactive spaces using real-time top-view depth-sensing. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.","DOI":"10.1145\/3025453.3025562"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Li, H., Zhang, P., Al Moubayed, S., Patel, S.N., and Sample, A.P. (2016, January 7\u201312). Id-match: A hybrid computer vision and rfid system for recognizing individuals in groups. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA.","DOI":"10.1145\/2858036.2858209"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Meng, R., Isenhower, J., Qin, C., and Nelakuditi, S. (2012, January 11\u201314). Can smartphone sensors enhance kinect experience. Proceedings of the Thirteenth ACM International Symposium on MOBILE Ad Hoc Networking and Computing, Hilton Head, SC, USA.","DOI":"10.1145\/2248371.2248417"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3040","DOI":"10.1109\/TIP.2014.2324292","article-title":"Exploiting long-term connectivity and visual motion in CRF-based multi-person tracking","volume":"23","author":"Heili","year":"2014","journal-title":"IEEE Trans. Image Process."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1183","DOI":"10.1109\/TMM.2018.2875360","article-title":"Deep alignment network based multi-person tracking with occlusion and motion reasoning","volume":"21","author":"Zhou","year":"2018","journal-title":"IEEE Trans. Multimed."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1186\/1687-5281-2014-12","article-title":"Shadow extraction and application in pedestrian detection","volume":"2014","author":"Wang","year":"2014","journal-title":"EURASIP J. Image Video Process."},{"key":"ref_22","unstructured":"Litomisky, K. (2012). Consumer RGB-D Cameras and Their Applications, University of California. Rapport Technique."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Chrapek, D., Beran, V., and Zemcik, P. (2015, January 26\u201329). Depth-based filtration for tracking boost. Proceedings of the International Conference on Advanced Concepts for Intelligent Vision Systems, Catania, Italy.","DOI":"10.1007\/978-3-319-25903-1_19"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Spinello, L., and Arras, K.O. (2011, January 25\u201330). People detection in RGB-D data. Proceedings of the 2011 IEEE\/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6095074"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Munaro, M., Basso, F., and Menegatti, E. (2012, January 7\u201312). Tracking people within groups with RGB-D data. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Portugal.","DOI":"10.1109\/IROS.2012.6385772"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1016\/j.robot.2015.10.004","article-title":"OpenPTrack: Open source multi-camera calibration and people tracking for RGB-D camera networks","volume":"75","author":"Munaro","year":"2016","journal-title":"Robot. Auton. Syst."},{"key":"ref_27","unstructured":"Munaro, M., Horn, A., Illum, R., Burke, J., and Rusu, R.B. (2014, January 15). OpenPTrack: People tracking for heterogeneous networks of color-depth cameras. Proceedings of the IAS-13 Workshop Proceedings: 1st Intl. Workshop on 3D Robot Perception with Point Cloud Library, Padova, Italy."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1007\/s00779-016-0997-6","article-title":"Out of sight: A toolkit for tracking occluded human joint positions","volume":"21","author":"Wu","year":"2017","journal-title":"Pers. Ubiquitous Comput."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Tseng, T.E., Liu, A.S., Hsiao, P.H., Huang, C.M., and Fu, L.C. (2014, January 14\u201318). Real-time people detection and tracking for indoor surveillance using multiple top-view depth cameras. Proceedings of the 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA.","DOI":"10.1109\/IROS.2014.6943136"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Rauter, M. (2013, January 23\u201328). Reliable human detection and tracking in top-view depth images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA.","DOI":"10.1109\/CVPRW.2013.84"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Breitenstein, M.D., Reichlin, F., Leibe, B., Koller-Meier, E., and van Gool, L. (October, January 29). Robust tracking-by-detection using a detector confidence particle filter. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.","DOI":"10.1109\/ICCV.2009.5459278"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1208","DOI":"10.1109\/TPAMI.2004.73","article-title":"Tracking multiple humans in complex situations","volume":"26","author":"Zhao","year":"2004","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.cviu.2016.05.011","article-title":"An occlusion-aware particle filter tracker to handle complex and persistent occlusions","volume":"150","author":"Meshgi","year":"2016","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_34","unstructured":"Norrie, L., and Murray-Smith, R. (September, January 30). Virtual sensors: Rapid prototyping of ubiquitous interaction with a mobile phone and a Kinect. Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, Stockholm, Sweden."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Koide, K., and Miura, J. (2016, January 9\u201314). Person identification based on the matching of foot strike timings obtained by LRFs and a smartphone. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759616"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Misu, K., and Miura, J. (2016). Specific person detection and tracking by a mobile robot using 3D LIDAR and ESPAR antenna. Intelligent Autonomous Systems 13, Springer.","DOI":"10.1007\/978-3-319-08338-4_51"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"918","DOI":"10.1109\/TPAMI.2003.1206520","article-title":"Detecting moving shadows: Algorithms and evaluation","volume":"25","author":"Prati","year":"2003","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1151","DOI":"10.1109\/JPROC.2002.801448","article-title":"Background and foreground modeling using nonparametric kernel density estimation for visual surveillance","volume":"90","author":"Elgammal","year":"2002","journal-title":"Proc. IEEE"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1337","DOI":"10.1109\/TPAMI.2003.1233909","article-title":"Detecting moving objects, ghosts, and shadows in video streams","volume":"25","author":"Cucchiara","year":"2003","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.imavis.2015.06.003","article-title":"Chromatic shadow detection and tracking for moving foreground segmentation","volume":"41","author":"Huerta","year":"2015","journal-title":"Image Vis. Comput."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Khan, S.H., Bennamoun, M., Sohel, F., and Togneri, R. (2014, January 23\u201328). Automatic feature learning for robust shadow detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.249"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"682","DOI":"10.1109\/TPAMI.2017.2691703","article-title":"Leave-one-out kernel optimization for shadow detection and removal","volume":"40","author":"Vicente","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_43","unstructured":"Mirza, M., and Osindero, S. (2014). Conditional generative adversarial nets. In Proceedings of the NIPS Deep Learning and Representation Learning Workshop. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Nguyen, V., Vicente, Y., Tomas, F., Zhao, M., Hoai, M., and Samaras, D. (2017, January 22\u201329). Shadow detection with conditional generative adversarial networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.483"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wang, J., Li, X., and Yang, J. (2018, January 18\u201323). Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00192"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. (2011, January 20\u201325). Real-time human pose recognition in parts from single depth images. Proceedings of the CVPR 2011, Providence, RI, USA.","DOI":"10.1109\/CVPR.2011.5995316"},{"key":"ref_47","unstructured":"Elgammal, A., Harwood, D., and Davis, L. (July, January 26). Non-parametric model for background subtraction. Proceedings of the European Conference on Computer Vision, Dublin, Ireland."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Cheng, J., Tsai, Y.H., Hung, W.C., Wang, S., and Yang, M.H. (2018, January 18\u201323). Fast and accurate online video object segmentation via tracking parts. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00774"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Qi, C.R., Liu, W., Wu, C., Su, H., and Guibas, L.J. (2018, January 18\u201323). Frustum pointnets for 3d object detection from RGB-D data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00102"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/4\/1056\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T08:58:06Z","timestamp":1760173086000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/4\/1056"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,15]]},"references-count":49,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,2]]}},"alternative-id":["s20041056"],"URL":"https:\/\/doi.org\/10.3390\/s20041056","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,15]]}}}