{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,7]],"date-time":"2025-11-07T13:39:20Z","timestamp":1762522760181,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2022,11,1]],"date-time":"2022-11-01T00:00:00Z","timestamp":1667260800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61903207","2021KJ025","2020-CXY32","2019JZZY010731","2019GXRC062"],"award-info":[{"award-number":["61903207","2021KJ025","2020-CXY32","2019JZZY010731","2019GXRC062"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Youth Innovation Science and technology support plan of colleges in Shandong Province","award":["61903207","2021KJ025","2020-CXY32","2019JZZY010731","2019GXRC062"],"award-info":[{"award-number":["61903207","2021KJ025","2020-CXY32","2019JZZY010731","2019GXRC062"]}]},{"name":"Industry-university-research Collaborative Innovation Fund project","award":["61903207","2021KJ025","2020-CXY32","2019JZZY010731","2019GXRC062"],"award-info":[{"award-number":["61903207","2021KJ025","2020-CXY32","2019JZZY010731","2019GXRC062"]}]},{"name":"Key Research &amp; Development Plan of Shandong Province","award":["61903207","2021KJ025","2020-CXY32","2019JZZY010731","2019GXRC062"],"award-info":[{"award-number":["61903207","2021KJ025","2020-CXY32","2019JZZY010731","2019GXRC062"]}]},{"name":"Jinan scientific research leader studio","award":["61903207","2021KJ025","2020-CXY32","2019JZZY010731","2019GXRC062"],"award-info":[{"award-number":["61903207","2021KJ025","2020-CXY32","2019JZZY010731","2019GXRC062"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This work provides a 3D hand attitude estimation approach for fixed hand posture based on a CNN and LightGBM for dual-view RGB images to facilitate the application of hand posture teleoperation. First, using dual-view cameras and an IMU sensor, we provide a simple method for building 3D hand posture datasets. This method can quickly acquire dual-view 2D hand image sets and automatically append the appropriate three-axis attitude angle labels. Then, combining ensemble learning, which has strong regression fitting capabilities, with deep learning, which has excellent automatic feature extraction capabilities, we present an integrated hand attitude CNN regression model. This model uses a Bayesian optimization based LightGBM in the ensemble learning algorithm to produce 3D hand attitude regression and two CNNs to extract dual-view hand image features. Finally, a mapping from dual-view 2D images to 3D hand attitude angles is established using a training approach for feature integration, and a comparative experiment is run on the test set. The results of the experiments demonstrate that the suggested method may successfully solve the hand self-occlusion issue and accomplish 3D hand attitude estimation using only two normal RGB cameras.<\/jats:p>","DOI":"10.3390\/s22218410","type":"journal-article","created":{"date-parts":[[2022,11,2]],"date-time":"2022-11-02T03:44:17Z","timestamp":1667360657000},"page":"8410","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["A 3D Hand Attitude Estimation Method for Fixed Hand Posture Based on Dual-View RGB Images"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5476-8734","authenticated-orcid":false,"given":"Peng","family":"Ji","sequence":"first","affiliation":[{"name":"School of Information and Automation Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4994-0592","authenticated-orcid":false,"given":"Xianjian","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information and Automation Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, China"}]},{"given":"Fengying","family":"Ma","sequence":"additional","affiliation":[{"name":"School of Information and Automation Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1587-7457","authenticated-orcid":false,"given":"Jinxiang","family":"Feng","sequence":"additional","affiliation":[{"name":"School of Information and Automation Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, China"}]},{"given":"Chenglong","family":"Li","sequence":"additional","affiliation":[{"name":"School of Information and Automation Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan 250353, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"5989","DOI":"10.1007\/s11042-019-08274-w","article-title":"Design of hand gesture recognition system for human-computer interaction","volume":"79","author":"Tsai","year":"2020","journal-title":"Multimed. Tools Appl."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/j.procs.2017.09.092","article-title":"Hand Gesture Recognition for Human Computer Interaction","volume":"115","author":"Haria","year":"2017","journal-title":"Procedia Comput. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10462-012-9356-9","article-title":"Vision based hand gesture recognition for human computer interaction: A survey","volume":"43","author":"Rautaray","year":"2015","journal-title":"Artif. Intell. Rev. Int. Sci. Eng. J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1016\/j.media.2017.11.011","article-title":"Gaze Gesture Based Human Robot Interaction for Laparoscopic Surgery","volume":"44","author":"Fujii","year":"2018","journal-title":"Med. Image Anal."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"103996","DOI":"10.1016\/j.archger.2019.103996","article-title":"Improving gesture-based interaction between an assistive bathing robot and older adults via user training on the gestural commands-ScienceDirect","volume":"87","author":"Cwa","year":"2020","journal-title":"Arch. Gerontol. Geriatr."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, W., Yu, C., Tu, C., Lyu, Z., and Xue, Z. (2020). A Survey on Hand Pose Estimation with Wearable Sensors and Computer-Vision-Based Methods. Sensors, 20.","DOI":"10.3390\/s20041074"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.patcog.2019.04.026","article-title":"A Survey on 3D Hand Pose Estimation: Cameras, Methods, and Datasets","volume":"93","author":"Li","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1049\/iet-cvi.2011.0128","article-title":"Vision-based hand pose estimation through similarity search using the earth mover\u2019s distance","volume":"6","author":"Villiers","year":"2012","journal-title":"IET Comput. Vis."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Li, Y., Pan, L., and Cheng, T. (2021, January 26\u201328). A Camera PTZ Control Algorithm For Autonomous Mobile Inspection Robot. Proceedings of the 2021 IEEE 2nd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), Nanchang, China.","DOI":"10.1109\/ICBAIE52039.2021.9389970"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1793","DOI":"10.1109\/TPAMI.2011.33","article-title":"Model-Based 3D Hand Pose Estimation from Monocular Video","volume":"33","author":"Fleet","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"5293","DOI":"10.1109\/TCSVT.2022.3142787","article-title":"3D Hand Pose Estimation from Monocular RGB with Feature Interaction Module","volume":"32","author":"Guo","year":"2022","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Pfister, T., Charles, J., and Zisserman, A. (2015, January 7\u201313). Flowing ConvNets for Human Pose Estimation in Videos. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Washington, DC, USA.","DOI":"10.1109\/ICCV.2015.222"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zimmermann, C., and Brox, T. (2017, January 22\u201329). Learning to Estimate 3D Hand Pose from Single RGB Images. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.525"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Panteleris, P., Oikonomidis, I., and Argyros, A. (2018, January 12\u201315). Using a Single RGB Frame for Real Time 3D Hand Pose Estimation in the Wild. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.","DOI":"10.1109\/WACV.2018.00054"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Simon, T., Joo, H., Matthews, I., and Sheikh, Y. (2017, January 21\u201326). Hand Keypoint Detection in Single Images Using Multiview Bootstrapping. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.494"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kong, D., Ma, H., Chen, Y., and Xie, X. (2020, January 1\u20135). Rotation-invariant Mixed Graphical Model Network for 2D Hand Pose Estimation. Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA.","DOI":"10.1109\/WACV45572.2020.9093638"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach. Learn."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1006\/jcss.1997.1504","article-title":"A decision-theoretic generalization of on-line learning and an application to boosting","volume":"55","author":"Freund","year":"1997","journal-title":"J. Comput. Syst. Sci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining-KDD\u201916, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_21","unstructured":"Ke, G., Meng, Q., Finely, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.Y. (2017, January 4\u20139). LightGBM: A Highly Efficient Gradient Boosting Decision Tree. Proceedings of the Advances in Neural Information Processing Systems 30 (NIP 2017), Long Beach, CA, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Fan, L., Rao, H., and Yang, W. (2021). 3D Hand Pose Estimation Based on Five-Layer Ensemble CNN. Sensors, 21.","DOI":"10.3390\/s21020649"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1016\/j.jvcir.2018.04.005","article-title":"Region Ensemble Network: Towards Good Practices for Deep 3D Hand Pose Estimation","volume":"55","author":"Wang","year":"2018","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yang, S., Chen, L.F., Yan, T., Zhao, Y.H., and Fan, Y.J. (2017, January 24\u201326). An ensemble classification algorithm for convolutional neural network based on AdaBoost. Proceedings of the 2017 IEEE\/ACIS 16th International Conference on Computer and Information Science (ICIS), Wuhan, China.","DOI":"10.1109\/ICIS.2017.7960026"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"60445","DOI":"10.1109\/ACCESS.2018.2875525","article-title":"A Novel Model Based on AdaBoost and Deep CNN for Vehicle Classification","volume":"6","author":"Chen","year":"2018","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hameed, K., Chai, D., and Rassau, A. (2020). A Sample Weight and AdaBoost CNN-Based Coarse to Fine Classification of Fruit and Vegetables at a Supermarket Self-Checkout. Appl. Sci., 10.","DOI":"10.3390\/app10238667"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1007\/s10846-016-0440-2","article-title":"Egocentric-Vision based Hand Posture Control System for Reconnaissance Robots","volume":"87","author":"Ji","year":"2017","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"522","DOI":"10.1016\/j.net.2020.04.008","article-title":"ConvXGB: A new deep learning model for classification problems based on CNN and XGBoost","volume":"53","author":"Thongsuwan","year":"2020","journal-title":"Nucl. Eng. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Rahman, M., Prodhan, R., Shishir, Y., and Ripon, S. (2021, January 25\u201327). Analyzing and Evaluating Boosting-Based CNN Algorithms for Image Classification. Proceedings of the 2021 International Conference on Intelligent Technologies (CONIT), Hubballi, India.","DOI":"10.1109\/CONIT51480.2021.9498328"},{"key":"ref_30","unstructured":"Kraetzer, C., Shi, Y.Q., Dittmann, J., and Kim, H.J. (2017). A Novel Image Classification Method with CNN-XGBoost Model. International Workshop on Digital Watermarking, Springer. Lecture Notes in Computer Science."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"12755","DOI":"10.1109\/ACCESS.2018.2796722","article-title":"Image Classification Based on the Boost Convolutional Neural Network","volume":"6","author":"Lee","year":"2018","journal-title":"IEEE Access"},{"key":"ref_32","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_34","first-page":"2951","article-title":"Practical Bayesian Optimization of Machine Learning Algorithms","volume":"4","author":"Snoek","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"104461","DOI":"10.1016\/j.chemolab.2021.104461","article-title":"Prediction of f-CaO content in cement clinker: A novel prediction method based on LightGBM and Bayesian optimization","volume":"220","author":"Hao","year":"2022","journal-title":"Chemom. Intell. Lab. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"4645","DOI":"10.1016\/j.autcon.2022.104331","article-title":"Surface settlement prediction for urban tunneling using machine learning algorithms with Bayesian optimization","volume":"140","author":"Kim","year":"2022","journal-title":"Autom. Constr."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Narasimhaswamy, S., Wei, Z., Wang, Y., Zhang, J., and Nguyen, M.H. (November, January 27). Contextual Attention for Hand Detection in the Wild. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00966"},{"key":"ref_38","unstructured":"Bergstra, J., Yamins, D., and Cox, D.D. (2013, January 16\u201321). Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures. Proceedings of the 30th International Conference on International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Dong, X., Qian, L., and Huang, L. (2017, January 4\u20138). A CNN based bagging learning approach to short-term load forecasting in smart grid. Proceedings of the 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld\/SCALCOM\/UIC\/ATC\/CBDCom\/IOP\/SCI), Francisco, CA, USA.","DOI":"10.1109\/UIC-ATC.2017.8397649"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8410\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:08:59Z","timestamp":1760144939000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8410"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,1]]},"references-count":39,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["s22218410"],"URL":"https:\/\/doi.org\/10.3390\/s22218410","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,11,1]]}}}