{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T10:45:24Z","timestamp":1775040324735,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T00:00:00Z","timestamp":1775001600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Technologies Research and Development Program of China","award":["2024YFB4708805"],"award-info":[{"award-number":["2024YFB4708805"]}]},{"name":"Major Science and Technology Project of Jiangsu Province","award":["BG2025040"],"award-info":[{"award-number":["BG2025040"]}]},{"name":"Frontier Technology Research and Development Project of Jiangsu Province","award":["BF2025060"],"award-info":[{"award-number":["BF2025060"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Biomimetics"],"abstract":"<jats:p>Human-to-robot (H2R) handovers are critical in human\u2013robot interaction but are challenged by complex environments that impact robot perception. Traditional RGB-based perception methods exhibit severe performance degradation under harsh lighting (e.g., glare and darkness). Furthermore, H2R handovers occur in unstructured environments populated with fine-grained visual details, such as multi-angle hand configurations and novel object geometries, where conventional semantic segmentation and grasp generation approaches struggle to generalize. To overcome lighting disturbances, we present an H2R handover system with a dual-path perception pipeline. The system fuses perception data from a stereo RGB-D camera (eye-in-hand) and a time-of-flight (ToF) camera (fixed scene) under normal lighting, and switches to the ToF camera for reliable perception under glare and darkness. In parallel, to address the complex spatial and geometric features, we augment the Point Transformer v3 (PTv3) architecture by integrating a T-Net module and a self-attention mechanism to fuse the relative positional angle features between human and robot, enabling efficient real-time 3D semantic segmentation of both the object and the human hand. For grasp generation, we extend GraspNet with a grasp selection module optimized for H2R scenarios. We validate our approach through extensive experiments: (1) a semantic segmentation dataset with 7500 annotated point clouds covering 15 objects and 5 relative angles and tested on 750 point clouds from 15 unseen objects, where our method achieves 84.4% mIoU, outperforming Swin3D-L by 3.26 percentage points with 3.2\u00d7 faster inference; (2) 250 real-world handover trials comparing our method with the baseline across 5 objects, 5 hand postures, and 5 angles, showing an improvement of 18.4 percentage points in success rate; (3) 450 trials under controlled adverse lighting (darkness and glare), where our dual-path perception method achieves 82.7% overall success, surpassing single-camera baselines by up to 39.4 percentage points; and (4) a comparative experiment against a state-of-the-art multimodal H2R handover method under identical adverse lighting, where our system achieves 75.0% success (15\/20) versus the baseline\u2019s 15.0% (3\/20), further confirming the lighting robustness of our design. These results demonstrate the system\u2019s robustness and generalization in challenging H2R handover scenarios.<\/jats:p>","DOI":"10.3390\/biomimetics11040231","type":"journal-article","created":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T08:08:34Z","timestamp":1775030914000},"page":"231","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Robust Human-to-Robot Handover System Under Adverse Lighting"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-4533-3054","authenticated-orcid":false,"given":"Yifei","family":"Wang","sequence":"first","affiliation":[{"name":"State Key Laboratory of Digital Medical Engineering, Jiangsu Key Laboratory of Robot Perception and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0735-1498","authenticated-orcid":false,"given":"Baoguo","family":"Xu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Digital Medical Engineering, Jiangsu Key Laboratory of Robot Perception and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huijun","family":"Li","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Digital Medical Engineering, Jiangsu Key Laboratory of Robot Perception and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1982-6780","authenticated-orcid":false,"given":"Aiguo","family":"Song","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Digital Medical Engineering, Jiangsu Key Laboratory of Robot Perception and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,4,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"102432","DOI":"10.1016\/j.rcim.2022.102432","article-title":"Human\u2013robot collaboration and machine learning: A systematic review of recent research","volume":"79","author":"Semeraro","year":"2023","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Yang, W., Paxton, C., Cakmak, M., and Fox, D. (2020\u201324, January 24). Human Grasp Classification for Reactive Human-to-Robot Handovers. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341004"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Yang, W., Paxton, C., Mousavian, A., Chao, Y.W., Cakmak, M., and Fox, D. (June, January 30). Reactive Human-to-Robot Handovers of Arbitrary Objects. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561170"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Christen, S., Yang, W., P\u00e9rez-D\u2019Arpino, C., Hilliges, O., Fox, D., and Chao, Y.W. (2023, January 18\u201322). Learning Human-to-Robot Handovers from Point Clouds. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00931"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1109\/LRA.2020.3026970","article-title":"Object-Independent Human-to-Robot Handovers Using Real Time Robotic Vision","volume":"6","author":"Rosenberger","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1224","DOI":"10.1109\/TCDS.2022.3203025","article-title":"Learning Human-to-Robot Dexterous Handovers for Anthropomorphic Hand","volume":"15","author":"Duan","year":"2023","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"494","DOI":"10.1109\/TNNLS.2022.3175720","article-title":"Simulation-Aided Handover Prediction from Video Using Recurrent Image-to-Motion Networks","volume":"35","author":"Mavsar","year":"2024","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Mavsar, M., and Ude, A. (2022, January 28\u201330). RoverNet: Vision-Based Adaptive Human-to-Robot Object Handovers. Proceedings of the 2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids), Ginowan, Japan.","DOI":"10.1109\/Humanoids53995.2022.10000200"},{"key":"ref_9","first-page":"2510311","article-title":"Object transfer point predicting based on human comfort model for human-robot handover","volume":"70","author":"Liu","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3933","DOI":"10.1109\/LRA.2021.3067299","article-title":"Evaluating Guided Policy Search for Human-Robot Handovers","volume":"6","author":"Kshirsagar","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3772","DOI":"10.1109\/LRA.2023.3270745","article-title":"Fed-HANet: Federated Visual Grasping Learning for Human Robot Handovers","volume":"8","author":"Huang","year":"2023","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, Z., Chen, J., Chen, Z., Xie, P., Chen, R., and Yi, L. (2024, January 16\u201322). GenH2R: Learning generalizable human-to-robot handover via scalable simulation demonstration and imitation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.01548"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Orsula, A., B\u00f8gh, S., Olivares-Mendez, M., and Martinez, C. (2022, January 23\u201327). Learning to Grasp on the Moon from 3D Octree Observations with Deep Reinforcement Learning. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.","DOI":"10.1109\/IROS47612.2022.9981661"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Li, Z., and Stamos, I. (2023, January 1\u20135). Depth-Based 6DoF Object Pose Estimation Using Swin Transformer. Proceedings of the 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA.","DOI":"10.1109\/IROS55552.2023.10342215"},{"key":"ref_15","unstructured":"Ma, H., and Huang, D. (2022, January 14\u201318). Towards Scale Balanced 6-DoF Grasp Detection in Cluttered Scenes. Proceedings of Machine Learning Research, Proceedings of the 6th Conference on Robot Learning, Auckland, New Zealand."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhang, Y., M\u00fcller, S., Stephan, B., Gross, H.M., and Notni, G. (2021). Point Cloud Hand\u2013Object Segmentation Using Multimodal Imaging with Thermal and Color Data for Safe Robotic Object Handover. Sensors, 21.","DOI":"10.3390\/s21165676"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"103171","DOI":"10.1016\/j.rcim.2025.103171","article-title":"A two-stage framework for learning human-to-robot object handover policy from 4D spatiotemporal flow","volume":"98","author":"Zhong","year":"2026","journal-title":"Robot. Comput.-Integr. Manuf."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1425","DOI":"10.1109\/LRA.2024.3519875","article-title":"Enabling Embodied Human-Robot Co-Learning: Requirements, Method, and Test with Handover Task","volume":"10","author":"Neerincx","year":"2025","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"12008","DOI":"10.1109\/LRA.2025.3615523","article-title":"LLM-Handover: Exploiting LLMs for Task-Oriented Robot-Human Handovers","volume":"10","author":"Tulbure","year":"2025","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wu, X., Jiang, L., Wang, P.S., Liu, Z., Liu, X., Qiao, Y., Ouyang, W., He, T., and Zhao, H. (2024, January 16\u201322). Point Transformer V3: Simpler, Faster, Stronger. Proceedings of the 2024 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.00463"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"10781","DOI":"10.1038\/s41467-024-54789-8","article-title":"Embodied neuromorphic synergy for lighting-robust machine vision to see in extreme bright","volume":"15","author":"Lin","year":"2024","journal-title":"Nat. Commun."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"eady2869","DOI":"10.1126\/scirobotics.ady2869","article-title":"Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity","volume":"11","author":"Ye","year":"2026","journal-title":"Sci. Robot."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.actaastro.2025.09.004","article-title":"SLiG-Net: A joint pose optimization network for space robot grasping under low-light conditions in on-orbit operations","volume":"238","author":"Huang","year":"2025","journal-title":"Acta Astronaut."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1574","DOI":"10.1109\/TII.2025.3628649","article-title":"Efficient Six-Degrees of Freedom (6-DoF) Grasp Pose Detection in Cluttered Scenes via Multimodal Fusion and Object-Centric Receptive Fields","volume":"22","author":"Liu","year":"2026","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Eppner, C., Mousavian, A., and Fox, D. (June, January 30). ACRONYM: A Large-Scale Grasp Dataset Based on Simulation. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9560844"},{"key":"ref_26","unstructured":"Mousavian, A., Eppner, C., and Fox, D. (November, January 27). 6-DOF GraspNet: Variational Grasp Generation for Object Manipulation. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chao, Y.W., Yang, W., Xiang, Y., Molchanov, P., Handa, A., Tremblay, J., Narang, Y.S., Van Wyk, K., Iqbal, U., and Birchfield, S. (2021, January 20\u201325). DexYCB: A Benchmark for Capturing Hand Grasping of Objects. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00893"},{"key":"ref_28","first-page":"5099","article-title":"PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space","volume":"Volume 30","author":"Qi","year":"2017","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"83","DOI":"10.26599\/CVM.2025.9450383","article-title":"Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understanding","volume":"11","author":"Yang","year":"2025","journal-title":"Comput. Vis. Media"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Lai, X., Chen, Y., Lu, F., Liu, J., and Jia, J. (2023, January 18\u201322). Spherical transformer for lidar-based 3d recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01683"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3705","DOI":"10.1109\/TRO.2023.3281483","article-title":"Impedance Learning for Human-Guided Robots in Contact with Unknown Environments","volume":"39","author":"Xing","year":"2023","journal-title":"IEEE Trans. Robot."}],"container-title":["Biomimetics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-7673\/11\/4\/231\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T08:42:40Z","timestamp":1775032960000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-7673\/11\/4\/231"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,1]]},"references-count":31,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,4]]}},"alternative-id":["biomimetics11040231"],"URL":"https:\/\/doi.org\/10.3390\/biomimetics11040231","relation":{},"ISSN":["2313-7673"],"issn-type":[{"value":"2313-7673","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,1]]}}}