{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T10:47:27Z","timestamp":1774435647729,"version":"3.50.1"},"reference-count":57,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2023,8,7]],"date-time":"2023-08-07T00:00:00Z","timestamp":1691366400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100007053","name":"Korea Institute of Energy Technology Evaluation and Planning","doi-asserted-by":"publisher","award":["20202020800030"],"award-info":[{"award-number":["20202020800030"]}],"id":[{"id":"10.13039\/501100007053","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>As the use of construction robots continues to increase, ensuring safety and productivity while working alongside human workers becomes crucial. To prevent collisions, robots must recognize human behavior in close proximity. However, single, or RGB-depth cameras have limitations, such as detection failure, sensor malfunction, occlusions, unconstrained lighting, and motion blur. Therefore, this study proposes a multiple-camera approach for human activity recognition during human\u2013robot collaborative activities in construction. The proposed approach employs a particle filter, to estimate the 3D human pose by fusing 2D joint locations extracted from multiple cameras and applies long short-term memory network (LSTM) to recognize ten activities associated with human and robot collaboration tasks in construction. The study compared the performance of human activity recognition models using one, two, three, and four cameras. Results showed that using multiple cameras enhances recognition performance, providing a more accurate and reliable means of identifying and differentiating between various activities. The results of this study are expected to contribute to the advancement of human activity recognition and utilization in human\u2013robot collaboration in construction.<\/jats:p>","DOI":"10.3390\/s23156997","type":"journal-article","created":{"date-parts":[[2023,8,7]],"date-time":"2023-08-07T06:38:48Z","timestamp":1691390328000},"page":"6997","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["Multi-Camera-Based Human Activity Recognition for Human\u2013Robot Collaboration in Construction"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3180-7339","authenticated-orcid":false,"given":"Youjin","family":"Jang","sequence":"first","affiliation":[{"name":"Department of Civil, Construction and Environmental Engineering, North Dakota State University, Fargo, ND 58108, USA"}]},{"given":"Inbae","family":"Jeong","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, North Dakota State University, Fargo, ND 58108, USA"}]},{"given":"Moein","family":"Younesi Heravi","sequence":"additional","affiliation":[{"name":"Department of Civil, Construction and Environmental Engineering, North Dakota State University, Fargo, ND 58108, USA"}]},{"given":"Sajib","family":"Sarkar","sequence":"additional","affiliation":[{"name":"Department of Civil, Construction and Environmental Engineering, North Dakota State University, Fargo, ND 58108, USA"}]},{"given":"Hyunkyu","family":"Shin","sequence":"additional","affiliation":[{"name":"Sustainable Smart City Convergence Educational Research Center, Hanyang University ERICA, Ansan 15588, Republic of Korea"}]},{"given":"Yonghan","family":"Ahn","sequence":"additional","affiliation":[{"name":"Department of Architectural Engineering, Hanyang University ERICA, Ansan 15588, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Maeda, J., Takada, H., Abe, Y., Maeda, J., Takada, H., and Abe, Y. (2004, January 21\u201325). Applicable Possibility Studies on a Humanoid Robot to Cooperative Work on Construction Site with a Human Worker. Proceedings of the 21st International Symposium on Automation and Robotics in Construction, Jeju Island, Republic of Korea.","DOI":"10.22260\/ISARC2004\/0058"},{"key":"ref_2","unstructured":"Bock, T., Bulgalow, A., and Ashida, S. (2017, January 23\u201325). Fa\u00e7ade Cleaning Robot for the Skyscraper. Proceedings of the 19th International Symposium on Automation and Robotics in Construction (ISARC), Washington, DC, USA."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"103778","DOI":"10.1016\/j.autcon.2021.103778","article-title":"Smart Component-Oriented Method of Construction Robot Coordination for Prefabricated Housing","volume":"129","author":"Zhu","year":"2021","journal-title":"Autom. Constr."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"104812","DOI":"10.1016\/j.autcon.2023.104812","article-title":"Human\u2013Robot Collaboration for on-Site Construction","volume":"150","author":"Zhang","year":"2023","journal-title":"Autom. Constr."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Brosque, C., Galbally, E., Khatib, O., and Fischer, M. (2020, January 26\u201328). Human-Robot Collaboration in Construction: Opportunities and Challenges. Proceedings of the HORA 2020\u20142nd International Congress on Human-Computer Interaction, Optimization and Robotic Applications, Ankara, Turkey.","DOI":"10.1109\/HORA49412.2020.9152888"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Lee, S.U., Hofmann, A., and Williams, B. (2019, January 3\u20138). A Model-Based Human Activity Recognition for Human-Robot Collaboration. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.","DOI":"10.1109\/IROS40897.2019.8967650"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Roitberg, A., Perzylo, A., Somani, N., Giuliani, M., Rickert, M., and Knoll, A. (2014, January 9\u201312). Human Activity Recognition in the Context of Industrial Human-Robot Interaction. Proceedings of the 2014 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA 2014, Siem Reap, Cambodia.","DOI":"10.1109\/APSIPA.2014.7041588"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.34293\/acsjse.v1i1.1","article-title":"Human Fall Detection Using Video Surveillance","volume":"1","author":"Agrawal","year":"2021","journal-title":"ACS J. Sci. Eng."},{"key":"ref_9","unstructured":"Gupta, A., Gupta, K., Gupta, K., and Gupta, K.O. (2021, January 25\u201327). Human Activity Recognition Using Pose Estimation and Machine Learning Algorithm. Proceedings of the ISIC\u201921: International Semantic Intelligence Conference, New Delhi, India."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Nguyen, H.C., Nguyen, T.H., Scherer, R., and Le, V.H. (2023). Deep Learning for Human Activity Recognition on 3D Human Skeleton: Survey and Comparative Study. Sensors, 23.","DOI":"10.3390\/s23115121"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1049\/cit2.12014","article-title":"Deep Learning-Based Action Recognition with 3D Skeleton: A Survey","volume":"6","author":"Xing","year":"2021","journal-title":"CAAI Trans. Intell. Technol."},{"key":"ref_12","unstructured":"Ren, B., Liu, M., Ding, R., and Liu, H. (2020). A Survey on 3D Skeleton-Based Action Recognition Using Learning Method. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"33532","DOI":"10.1109\/ACCESS.2021.3061626","article-title":"Fall Detection and Activity Recognition Using Human Skeleton Features","volume":"9","author":"Ramirez","year":"2021","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1006\/cviu.2000.0878","article-title":"Reconstruction of Articulated Objects from Point Correspondences in a Single Uncalibrated Image","volume":"80","author":"Taylor","year":"2000","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_15","unstructured":"Agarwal, A., and Triggs, B. (July, January 27). 3D Human Pose from Silhouettes by Relevance Vector Regression. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/TPAMI.2006.21","article-title":"Recovering 3D Human Pose from Monocular Images","volume":"28","author":"Agarwal","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2093","DOI":"10.1109\/TVCG.2019.2898650","article-title":"Mo2Cap2: Real-Time Mobile 3D Motion Capture with a Cap-Mounted Fisheye Camera","volume":"25","author":"Xu","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_18","unstructured":"Tome, D., Peluse, P., Agapito, L., and Badino, H. (November, January 27). XR-EgoPose: Egocentric 3D Human Pose from an HMD Camera. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"6794","DOI":"10.1109\/TPAMI.2020.3029700","article-title":"SelfPose: 3D Egocentric Pose Estimation from a Headset Mounted Camera","volume":"45","author":"Tome","year":"2020","journal-title":"IEEE Trans. Pattern. Anal. Mach. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"103225","DOI":"10.1016\/j.cviu.2021.103225","article-title":"Deep 3D Human Pose Estimation: A Review","volume":"210","author":"Wang","year":"2021","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_21","first-page":"828","article-title":"Performance Capture of Interacting Characters with Handheld Kinects","volume":"Volume 7573","author":"Ye","year":"2012","journal-title":"Proceedings of the ECCV 2012: Computer Vision\u2014ECCV 2012"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Bashirov, R., Ianina, A., Iskakov, K., Kononenko, Y., Strizhkova, V., Lempitsky, V., and Vakhitov, A. (2021, January 3\u20138). Real-Time RGBD-Based Extended Body Pose Estimation. Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision, WACV, Waikoloa, HI, USA.","DOI":"10.1109\/WACV48630.2021.00285"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhang, H.B., Zhang, Y.X., Zhong, B., Lei, Q., Yang, L., Du, J.X., and Chen, D.S. (2019). A Comprehensive Survey of Vision-Based Human Action Recognition Methods. Sensors, 19.","DOI":"10.3390\/s19051005"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, J., Liu, L., Xu, W., Sarkar, K., and Theobalt, C. (2021, January 10\u201317). Estimating Egocentric 3D Human Pose in Global Space. Proceedings of the IEEE International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.01130"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Chen, L., Ai, H., Chen, R., Zhuang, Z., and Liu, S. (2020, January 13\u201319). Cross-View Tracking for Multi-Human 3D Pose Estimation at over 100 FPS. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00334"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Gholami, M., Rezaei, A., Rhodin, H., Ward, R., and Wang, Z.J. (2021). TriPose: A Weakly-Supervised 3D Human Pose Estimation via Triangulation from Video. arXiv.","DOI":"10.1016\/j.neucom.2022.02.076"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1016\/j.mechatronics.2017.12.009","article-title":"Trajectory Generation Algorithm for Safe Human-Robot Collaboration Based on Multiple Depth Sensor Measurements","volume":"55","author":"Ragaglia","year":"2018","journal-title":"Mechatronics"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1186\/s40648-020-00185-y","article-title":"Pose Estimation by Extended Kalman Filter Using Noise Covariance Matrices Based on Sensor Output","volume":"7","author":"Saito","year":"2020","journal-title":"ROBOMECH J."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Negin, F., Koperski, M., Crispim, C.F., Bremond, F., Co\u015far, S., and Avgerinakis, K. (2016, January 23\u201326). A hybrid framework for online recognition of activities of daily living in real-world settings. Proceedings of the 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Colorado Springs, CO, USA.","DOI":"10.1109\/AVSS.2016.7738021"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Liu, Y., Li, A., and Wang, M. (2014, January 14\u201316). A novel method for user-defined human posture recognition using Kinect. Proceedings of the IEEE 7th International Congress on Image and Signal Processing, Dalian, China.","DOI":"10.1109\/CISP.2014.7003875"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1016\/j.patcog.2016.08.003","article-title":"Robust human activity recognition from depth video using spatiotemporal multi-fused features","volume":"61","author":"Jalal","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1109\/MITS.2018.2806619","article-title":"Vision-based occlusion handling and vehicle classification for traffic surveillance systems","volume":"10","author":"Chang","year":"2018","journal-title":"IEEE Intell. Transp. Syst. Mag."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"107561","DOI":"10.1016\/j.patcog.2020.107561","article-title":"Sensor-based and vision-based human activity recognition: A comprehensive survey","volume":"108","author":"Dang","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Li, X., Fan, Z., Liu, Y., Li, Y., and Dai, Q. (2019). 3D Pose Detection of Closely Interactive Humans Using Multi-View Cameras. Sensors, 19.","DOI":"10.3390\/s19122831"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2703","DOI":"10.1109\/TSP.2007.893914","article-title":"Particle Filtering for Multisensor Data Fusion with Switching Observation Models: Application to Land Vehicle Positioning","volume":"55","author":"Caron","year":"2007","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1762","DOI":"10.1109\/TSP.2003.810284","article-title":"Efficient Particle Filtering for Jump Markov Systems. Application to Time-Varying Autoregressions","volume":"51","author":"Andrieu","year":"2003","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Doucet, A., Freitas, N., and Gordon, N. (2001). Sequential Monte Carlo Methods in Practice, Springer.","DOI":"10.1007\/978-1-4757-3437-9"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1007\/s10015-017-0422-x","article-title":"Deep Recurrent Neural Network for Mobile Human Activity Recognition with High Throughput","volume":"23","author":"Inoue","year":"2018","journal-title":"Artif. Life Robot"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Zhu, W., Lan, C., Xing, J., Zeng, W., Li, Y., Shen, L., and Xie, X. (2016, January 12\u201317). Co-Occurrence Feature Learning for Skeleton Based Action Recognition Using Regularized Deep LSTM Networks. Proceedings of the 30th AAAI Conference on Artificial Intelligence, AAAI, Phoenix, AR, USA.","DOI":"10.1609\/aaai.v30i1.10451"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.patcog.2017.10.033","article-title":"Convolutional Neural Networks and Long Short-Term Memory for Skeleton-Based Human Activity and Hand Gesture Recognition","volume":"76","author":"Cabido","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"04020184","DOI":"10.1061\/(ASCE)CO.1943-7862.0002001","article-title":"Automatic Recognition of Workers\u2019 Motions in Highway Construction by Using Motion Sensors and Long Short-Term Memory Networks","volume":"147","author":"Kim","year":"2020","journal-title":"J. Constr. Eng. Manag."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"6044","DOI":"10.1109\/TGRS.2020.3010826","article-title":"Target-Constrained Interference-Minimized Band Selection for Hyperspectral Target Detection","volume":"59","author":"Shang","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2256","DOI":"10.1109\/TGRS.2020.3004353","article-title":"Super-Resolution Mapping Based on Spatial-Spectral Correlation for Spectral Imagery","volume":"59","author":"Wang","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_44","unstructured":"Lugaresi, C., Tang, J., Nash, H., Mcclanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.-L., Yong, M.G., and Lee, J. (2019). MediaPipe: A Framework for Building Perception Pipelines. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"108136","DOI":"10.1016\/j.patcog.2021.108136","article-title":"Human trajectory prediction and generation using LSTM models and GANs","volume":"120","author":"Rossi","year":"2021","journal-title":"Pattern Recognit."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"103126","DOI":"10.1016\/j.autcon.2020.103126","article-title":"Effective inertial sensor quantity and locations on a body for deep learning-based worker\u2019s motion recognition","volume":"113","author":"Kim","year":"2020","journal-title":"Autom. Constr."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"04020017","DOI":"10.1061\/(ASCE)CP.1943-5487.0000898","article-title":"Vision-Based Construction Worker Activity Analysis Informed by Body Posture","volume":"34","author":"Roberts","year":"2020","journal-title":"J. Comput. Civ. Eng."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1016\/j.autcon.2016.08.015","article-title":"Smartphone-Based Construction Workers\u2019 Activity Recognition and Classification","volume":"71","author":"Akhavian","year":"2016","journal-title":"Autom. Constr."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Sam, M., Franz, B., Sey-Taylor, E., and McCarty, C. (2022, January 9\u201312). Evaluating the perception of human-robot collaboration among construction project managers. Proceedings of the Construction Research Congress 2022, Arlington, VA, USA.","DOI":"10.1061\/9780784483961.058"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1093\/tse\/tdaa007","article-title":"An integrated review of automation and robotic technologies for structural prefabrication and construction","volume":"2","author":"Chea","year":"2020","journal-title":"Transp. Saf. Environ."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1108\/IR-03-2020-0060","article-title":"The Pransky interview: Dr. Tessa Lau, Founder and CEO of Dusty Robotics","volume":"47","author":"Pransky","year":"2020","journal-title":"Ind. Robot Int. J. Robot. Res. Appl."},{"key":"ref_52","first-page":"38","article-title":"Robotic rebar-tying system uses artificial intelligence","volume":"88","author":"Cardno","year":"2018","journal-title":"Civ. Eng. Mag. Arch."},{"key":"ref_53","unstructured":"Eternal Robotics (2023, June 30). MYRO. Available online: www.eternalrobotics.com\/solutions\/#SMyro."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Putra, P.U., Shima, K., and Shimatani, K. (2018, January 10\u201313). Markerless Human Activity Recognition Method Based on Deep Neural Network Model Using Multiple Cameras. Proceedings of the 5th International Conference on Control, Decision and Information Technologies, CoDIT, Thessaloniki, Greece.","DOI":"10.1109\/CoDIT.2018.8394780"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"92300","DOI":"10.1109\/ACCESS.2021.3092403","article-title":"A Unified Approach for Patient Activity Recognition in Healthcare Using Depth Camera","volume":"9","author":"Siddiqi","year":"2021","journal-title":"IEEE Access"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"2364","DOI":"10.1016\/j.procs.2020.03.289","article-title":"A Lightweight Deep Learning Model for Human Activity Recognition on Edge Devices","volume":"167","author":"Agarwal","year":"2020","journal-title":"Procedia Comput. Sci."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"104181","DOI":"10.1016\/j.autcon.2022.104181","article-title":"Deep learning-based networks for automated recognition and classification of awkward working postures in construction using wearable insole sensor data","volume":"136","author":"Qarout","year":"2022","journal-title":"Autom. Constr."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/15\/6997\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:27:07Z","timestamp":1760128027000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/15\/6997"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,7]]},"references-count":57,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["s23156997"],"URL":"https:\/\/doi.org\/10.3390\/s23156997","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,7]]}}}