{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T05:59:47Z","timestamp":1763445587335,"version":"build-2065373602"},"reference-count":40,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,3,6]],"date-time":"2023-03-06T00:00:00Z","timestamp":1678060800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61973302, 61633020"],"award-info":[{"award-number":["61973302, 61633020"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["2019YFB1311100"],"award-info":[{"award-number":["2019YFB1311100"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>3D modeling plays a significant role in many industrial applications that require geometry information for pose measurements, such as grasping, spraying, etc. Due to random pose changes in the workpieces on the production line, demand for online 3D modeling has increased and many researchers have focused on it. However, online 3D modeling has not been entirely determined due to the occlusion of uncertain dynamic objects that disturb the modeling process. In this study, we propose an online 3D modeling method under uncertain dynamic occlusion based on a binocular camera. Firstly, focusing on uncertain dynamic objects, a novel dynamic object segmentation method based on motion consistency constraints is proposed, which achieves segmentation by random sampling and poses hypotheses clustering without any prior knowledge about objects. Then, in order to better register the incomplete point cloud of each frame, an optimization method based on local constraints of overlapping view regions and a global loop closure is introduced. It establishes constraints in covisibility regions between adjacent frames to optimize the registration of each frame, and it also establishes them between the global closed-loop frames to jointly optimize the entire 3D model. Finally, a confirmatory experimental workspace is designed and built to verify and evaluate our method. Our method achieves online 3D modeling under uncertain dynamic occlusion and acquires an entire 3D model. The pose measurement results further reflect the effectiveness.<\/jats:p>","DOI":"10.3390\/s23052871","type":"journal-article","created":{"date-parts":[[2023,3,7]],"date-time":"2023-03-07T01:43:35Z","timestamp":1678153415000},"page":"2871","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["An Online 3D Modeling Method for Pose Measurement under Uncertain Dynamic Occlusion Based on Binocular Camera"],"prefix":"10.3390","volume":"23","author":[{"given":"Xuanchang","family":"Gao","sequence":"first","affiliation":[{"name":"State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6347-572X","authenticated-orcid":false,"given":"Junzhi","family":"Yu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"State Key Laboratory for Turbulence and Complex Systems, Department of Advanced Manufacturing and Robotics, Beijing Innovation Center for Engineering Science and Advanced Technology (BIC-ESAT), College of Engineering, Peking University, Beijing 100871, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Min","family":"Tan","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,6]]},"reference":[{"key":"ref_1","first-page":"1","article-title":"Online 3-D Modeling of Complex Workpieces for the Robotic Spray Painting with Low-cost RGB-D Cameras","volume":"70","author":"Ge","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2020.3044719","article-title":"Density-Invariant Registration of Multiple Scans for Aircraft Measurement","volume":"70","author":"Wang","year":"2020","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1016\/j.rcim.2017.04.001","article-title":"Microsoft Kinect V2 Vision System in A Manufacturing Application","volume":"48","author":"Caruso","year":"2017","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chai, X., Wen, F., Cao, X., and Yuan, K. (2013, January 4\u20137). A Fast 3D Surface Reconstruction Method for Spraying Robot with Time-of-Flight Camera. Proceedings of the 2013 IEEE International Conference on Mechatronics and Automation, Takamatsu, Kagawa, Japan.","DOI":"10.1109\/ICMA.2013.6617893"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1145\/3516521","article-title":"HRBF-Fusion: Accurate 3D Reconstruction from RGB-D Data Using on-the-fly Implicits","volume":"41","author":"Xu","year":"2022","journal-title":"ACM Trans. Graph."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1007\/s00170-019-03901-0","article-title":"A Pose Estimation System Based on Deep Neural Network and ICP Registration for Robotic Spray Painting Application","volume":"104","author":"Wang","year":"2019","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"7305","DOI":"10.1109\/TSMC.2020.2980424","article-title":"Complex Workpiece Positioning System with Nonrigid Registration Method for 6-DoFs Automatic Spray Painting Robot","volume":"51","author":"Gao","year":"2020","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"103042","DOI":"10.1016\/j.cad.2021.103042","article-title":"Part-in-Whole Point Cloud Registration for Aircraft Partial Scan Automated Localization","volume":"137","author":"Xie","year":"2021","journal-title":"Comput.-Aided Des."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1109\/TII.2007.891309","article-title":"A Framework for CAD- and Sensor-based Robotic Coating Automation","volume":"3","author":"Bi","year":"2007","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"898","DOI":"10.18287\/2412-6179-2018-42-5-898-903","article-title":"Fusion of Information from Multiple Kinect Sensors for 3D Object Reconstruction","volume":"42","author":"Ruchay","year":"2018","journal-title":"Comput. Opt."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohi, P., Shotton, J., Hodges, S., and Fitzgibbon, A. (2011, January 26\u201329). KinectFusion: Real-Time Dense Surface Mapping and Tracking. Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality, Basel, Switzerland.","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lin, C.Y., Abebe, Z.A., and Chang, S.H. (2015, January 27\u201331). Advanced Spraying Task Strategy for Bicycle-Frame Based on Geometrical Data of Workpiece. Proceedings of the 2015 International Conference on Advanced Robotics (ICAR), Istanbul, Turkey.","DOI":"10.1109\/ICAR.2015.7251468"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhang, T., Zhang, H., and Li, Y. (August, January 31). Flowfusion: Dynamic Dense RGB-D SLAM based on Optical Flow. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197349"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sun, D., Yang, X., and Liu, M.Y. (2018, January 18\u201323). PWC-NET: CNNs for Optical Flow using Pyramid, Warping, and Cost Volume. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00931"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Scona, R., Jaimez, M., and Petillot, Y.R. (2018, January 21\u201325). Staticfusion: Background Reconstruction for Dense RGB-D SLAM in Dynamic Environments. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8460681"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Jaimez, M., Kerl, C., and Jimenez, J.G. (June, January 29). Fast Odometry and Scene Flow from RGB-D Cameras based on Geometric Clustering. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989459"},{"key":"ref_17","first-page":"100","article-title":"Algorithm AS 136: A K-means Clustering Algorithm","volume":"28","author":"Hartigan","year":"1979","journal-title":"J. R. Stat. Soc. Ser. C (Appl. Stat.)"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"R\u00fcnz, M., and Agapito, L. (June, January 29). Co-fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989518"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Xu, B., Li, W., and Tzoumanikas, D. (2019, January 20\u201324). Mid-Fusion: Octree-based Object-level Multi-instance Dynamic SLAM. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794371"},{"key":"ref_20","unstructured":"Zhang, T., and Nakamura, Y. Posefusion: Dense RGB-D SLAM in Dynamic Human Environments. Proceedings of the 2018 International Symposium on Experimental Robotics."},{"key":"ref_21","unstructured":"Pinheiro, P.O., Lin, T.Y., and Collobert, R. (2016). Computer Vision\u2014ECCV 2016, Springer."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., and Doll\u00e1r, P. (2017, January 22\u201329). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Runz, M., Buffier, M., and Agapito, L. (2018, January 16\u201320). Maskfusion: Real-time Recognition, Tracking and Reconstruction of Multiple Moving Objects. Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Munich, Germany.","DOI":"10.1109\/ISMAR.2018.00024"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1697","DOI":"10.1177\/0278364916669237","article-title":"ElasticFusion: Real-Time Dense SLAM and Light Source Estimation","volume":"35","author":"Whelan","year":"2016","journal-title":"Int. J. Robot. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/TPAMI.2019.2929257","article-title":"OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields","volume":"43","author":"Cao","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","unstructured":"Bouguet, J.Y. (2022, November 05). Camera Calibration Toolbox for Matlab. Caltech. Available online: http:\/\/www.vision.caltech.edu\/bouguetj\/calibdoc\/index.html."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1109\/34.601246","article-title":"In Defense of the Eight-point Algorithm","volume":"19","author":"Hartley","year":"1997","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1142\/S0218001488000285","article-title":"Motion and Structure from Motion in A Piecewise Planar Environment","volume":"2","author":"Faugeras","year":"1988","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Xu, X., Cheong, L.F., and Li, Z. (2018, January 18\u201323). Motion Segmentation by Exploiting Complementary Geometric Models. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00302"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1007\/s11263-008-0152-6","article-title":"EPnP: An Accurate O(n) Solution to the PnP Problem","volume":"81","author":"Lepetit","year":"2009","journal-title":"Int. J. Comput. Vis."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1492","DOI":"10.1126\/science.1242072","article-title":"Clustering by Fast Search and Find of Density Peaks","volume":"344","author":"Rodriguez","year":"2014","journal-title":"Science"},{"key":"ref_32","first-page":"333","article-title":"The Ordered Residual Kernel for Robust Motion Subspace Clustering","volume":"22","author":"Chin","year":"2009","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_33","unstructured":"Yamashita, N., and Fukushima, M. (2001). Topics in Numerical Analysis: With Special Emphasis on Nonlinear Problems, Springer."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A Flexible New Technique for Camera Calibration","volume":"22","author":"Zhang","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Bian, J.W., Lin, W.Y., Matsushita, Y., Zhang, L., Yeung, S.-K., and Cheng, M.-M. (2017, January 21\u201326). GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.302"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1109\/TPAMI.2007.1166","article-title":"Stereo Processing by Semiglobal Matching and Mutual Information","volume":"30","author":"Hirschmuller","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_37","unstructured":"Grisetti, G., K\u00fcmmerle, R., and Strasdat, H. (2011, January 9\u201313). g2o: A General Framework for (Hyper) Graph Optimization. Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA 2011), Shanghai, China."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"8272","DOI":"10.1364\/AO.403356","article-title":"3D Reconstruction of Line-Structured Light Based on Binocular Vision Calibration Rotary Axis","volume":"59","author":"Ye","year":"2020","journal-title":"Appl. Opt."},{"key":"ref_39","first-page":"586","article-title":"Method for Registration of 3-D Shapes","volume":"1611","author":"Besl","year":"1992","journal-title":"Int. Soc. Opt. Photonics"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Yin, P., Wang, D., and Du, S. (January, January 24). CoBigICP: Robust and Precise Point Set Registration using Correntropy Metrics and Bidirectional Correspondence. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9340857"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2871\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:49:13Z","timestamp":1760122153000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2871"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,6]]},"references-count":40,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23052871"],"URL":"https:\/\/doi.org\/10.3390\/s23052871","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,3,6]]}}}