{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T09:06:31Z","timestamp":1779181591872,"version":"3.51.4"},"reference-count":35,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T00:00:00Z","timestamp":1779062400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Robotic waste sorting presents significant challenges, including object variability, cluttered environments, and the predominant reliance on deep learning and traditional computer vision techniques, which typically demand extensive datasets and task-specific training. This paper introduces a robotic waste sorting system that integrates the Gemini Vision\u2013Language\u2013Action (VLA) model with a KUKA LBR iiwa collaborative robot and an RGB-D camera. Our approach leverages the advanced reasoning capabilities of large, pre-trained VLA models to perform waste sorting, without requiring explicit training or dataset collection. Key contributions include the development of effective prompt engineering strategies for waste object identification, the assessment of the VLA\u2019s performance in terms of inference time and accuracy, and the development of different grasping strategies for operation in cluttered scenarios. Our experimental tests demonstrated that the system\u2019s inference time is between 2 and 4 s, which is suitable for collaborative robotic applications, and the system achieved a high overall classification accuracy of 89.64%. Crucially, we demonstrated that integration of RGB-D sensing enhanced the model\u2019s ability to perceive object heights, resolve occlusions, and make informed grasping decisions in realistic, three-dimensional settings. We further validated multiple real-world grasping strategies, demonstrating tradeoffs between system efficiency and safety in heavily cluttered scenarios. This work establishes a practical and adaptable framework for deploying VLA-driven intelligence on commercial robotic platforms, highlighting the potential of VLAs for complex manipulation tasks beyond waste sorting.<\/jats:p>","DOI":"10.3390\/robotics15050100","type":"journal-article","created":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T07:35:43Z","timestamp":1779176143000},"page":"100","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Integrating Vision\u2013Language\u2013Action Models and RGB-D Sensing for Robotic Waste Sorting on KUKA LBR iiwa"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-8513-0100","authenticated-orcid":false,"given":"Teresa","family":"Sinico","sequence":"first","affiliation":[{"name":"Department of Management and Engineering (DTG), University of Padua, Stradella S. Nicola 3, 36100 Vicenza, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-1601-2033","authenticated-orcid":false,"given":"Daniele","family":"Businaro","sequence":"additional","affiliation":[{"name":"Department of Management and Engineering (DTG), University of Padua, Stradella S. Nicola 3, 36100 Vicenza, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2452-221X","authenticated-orcid":false,"given":"Giovanni","family":"Boschetti","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering (DII), University of Padua, Via Venezia 1, 35131 Padova, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,5,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1023","DOI":"10.1109\/TASE.2022.3221969","article-title":"Challenges for Future Robotic Sorters of Mixed Industrial Waste: A Survey","volume":"21","author":"Kiyokawa","year":"2024","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2789","DOI":"10.1007\/s12008-023-01320-w","article-title":"A state-of-the-art review on robotics in waste sorting: Scope and challenges","volume":"17","author":"Satav","year":"2023","journal-title":"Int. J. Interact. Des. Manuf."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"128151","DOI":"10.1109\/ACCESS.2022.3226682","article-title":"A Survey on Waste Detection and Classification Using Deep Learning","volume":"10","author":"Abdu","year":"2022","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Bircano\u011flu, C., Atay, M., Be\u015fer, F., Gen\u00e7, \u00d6., and K\u0131zrak, M.A. (2018). RecycleNet: Intelligent waste sorting using deep neural networks. Proceedings of the 2018 Innovations in Intelligent Systems and Applications (INISTA), IEEE.","DOI":"10.1109\/INISTA.2018.8466276"},{"key":"ref_5","unstructured":"Yang, M., and Thung, G. (2016). Classification of Trash for Recyclability Status, Stanford University. CS229 Project Report."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e70025","DOI":"10.1002\/widm.70025","article-title":"Artificial Intelligence-Based Waste Management: A Review of Classification, Techniques, Issues, and Challenges","volume":"15","author":"Yevle","year":"2025","journal-title":"WIREs Data Min. Knowl. Discov."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"522","DOI":"10.1007\/978-3-031-91151-4_57","article-title":"Robots for Sustainability: Real-Time AI-Based Robotic Waste Sorting from a Moving Conveyor","volume":"Volume 179","author":"Boschetti","year":"2025","journal-title":"Proceedings of I4SDG Workshop 2025\u2013IFToMM for Sustainable Development Goals; Mechanisms and Machine Science"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"9834","DOI":"10.1109\/LRA.2022.3192582","article-title":"A 4-DoF Parallel Robot With a Built-in Gripper for Waste Sorting","volume":"7","author":"Leveziel","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Choi, J., Lim, B., and Yoo, Y. (2023). Advancing Plastic Waste Classification and Recycling Efficiency: Integrating Image Sensors and Deep Learning Algorithms. Appl. Sci., 13.","DOI":"10.3390\/app131810224"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Deng, S., Lai, Y., Zhou, L., Pei, R., Qin, H., Xiang, S., Liang, Q., Sun, W., Zhang, D., and Wang, Y. (2026). An Efficient Continual Learning Grasp Detection Method With Lightweight RGB-D Fusion and Automated Data Annotation for Kitchen Waste Sorting. IEEE\/ASME Trans. Mechatron., 1\u201312.","DOI":"10.1109\/TMECH.2025.3650331"},{"key":"ref_11","unstructured":"Lu, G., Guo, W., Zhang, C., Zhou, Y., Jiang, H., Gao, Z., Tang, Y., and Wang, Z. (2025). Vla-rl: Towards masterful and general robotic manipulation with scalable reinforcement learning. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"162467","DOI":"10.1109\/ACCESS.2025.3609980","article-title":"Vision-language-action models for robotics: A review towards real-world applications","volume":"13","author":"Kawaharazuka","year":"2025","journal-title":"IEEE Access"},{"key":"ref_13","unstructured":"Team, G.R., Abeyruwan, S., Ainslie, J., Alayrac, J.B., Arenas, M.G., Armstrong, T., Balakrishna, A., Baruch, R., Bauza, M., and Blokzijl, M. (2025). Gemini robotics: Bringing ai into the physical world. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ranganathan, C.S., S, K., Vimal, S.P., Kannagi, V., Muthulekshmi, M., and Meganathan, M. (2024). Intelligent Robotics for Sorting Clothing and Textile Waste Using Cloud-Integrated Recycling Systems and CNN. Proceedings of the 2024 International Conference on Smart Technologies for Sustainable Development Goals (ICSTSDG), IEEE.","DOI":"10.1109\/ICSTSDG61998.2024.11026397"},{"key":"ref_15","unstructured":"Moktar, M.H., Hajjaj, S., and Mohamed, H. (2024). Implementation of YOLO Algorithm and IoT in Medical Waste Sorting Machine. Selected Proceedings from the 2nd International Conference on Intelligent Manufacturing and Robotics, Suzhou, China, 22\u201323 August 2024, Springer Nature."},{"key":"ref_16","first-page":"5773","article-title":"An improved YOLO-based waste detection model and its integration to robotic gripping systems","volume":"84","author":"Wang","year":"2025","journal-title":"Comput. Mater. Contin."},{"key":"ref_17","unstructured":"Sekar, S. (2026, May 13). Waste Classification Data. Available online: https:\/\/www.kaggle.com\/datasets\/techsash\/waste-classification-data."},{"key":"ref_18","unstructured":"Proen\u00e7a, P.F., and Sim\u00f5es, P. (2020). Taco: Trash annotations in context for litter detection. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Lai, K., Bo, L., Ren, X., and Fox, D. (2011). A large-scale hierarchical multi-view RGB-D object dataset. Proceedings of the 2011 IEEE International Conference on Robotics and Automation, IEEE.","DOI":"10.1109\/ICRA.2011.5980382"},{"key":"ref_20","unstructured":"(2026, April 05). kskmar. ReSort-IT. GitHub Repository. Available online: https:\/\/github.com\/kskmar\/ReSort-IT."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/j.wasman.2021.03.017","article-title":"A deep convolutional neural network to simultaneously localize and recognize waste types in images","volume":"126","author":"Liang","year":"2021","journal-title":"Waste Manag."},{"key":"ref_22","unstructured":"Labs, D. (2026, May 13). Domestic Trash\/Garbage Dataset. Available online: https:\/\/www.kaggle.com\/datasets\/dataclusterlabs\/domestic-trash-garbage-dataset."},{"key":"ref_23","first-page":"2522810","article-title":"An Efficient Generative Intelligent Multi-Objective Grasping Model for Kitchen Waste Sorting","volume":"74","author":"Deng","year":"2025","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1016\/j.procir.2024.01.101","article-title":"Sorting of Packaging Waste: A Framework to link Gripper Technologies and Waste Classes","volume":"122","author":"Engelen","year":"2024","journal-title":"Procedia CIRP"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Petuya, V., Quaglia, G., Parikyan, T., and Carbone, G. (2023). Multibody Simulation of an Underactuated Gripper for Sustainable Waste Sorting. Proceedings of the I4SDG Workshop 2023, Springer.","DOI":"10.1007\/978-3-031-32439-0"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1007\/s42452-023-05274-2","article-title":"Underactuated embedded constraints gripper for grasping in toxic environments","volume":"5","author":"Achilli","year":"2023","journal-title":"SN Appl. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"115164","DOI":"10.1016\/j.wasman.2025.115164","article-title":"RTDRNet-lite: A lightweight real-time detection framework for robotic waste sorting","volume":"208","author":"Karim","year":"2025","journal-title":"Waste Manag."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Nurkadem, K., Karymsakova, N., and Mansurova, M. (2023). Disposal of Solid Household Waste Using Computer Vision. Proceedings of the 2023 IEEE International Conference on Smart Information Systems and Technologies (SIST), IEEE.","DOI":"10.1109\/SIST58284.2023.10223538"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Gao, J., Sarkar, B., Xia, F., Xiao, T., Wu, J., Ichter, B., Majumdar, A., and Sadigh, D. (2024). Physically grounded vision-language models for robotic manipulation. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), IEEE.","DOI":"10.1109\/ICRA57147.2024.10610090"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Jeong, H., Lee, H., Kim, C., and Shin, S. (2024). A survey of robot intelligence with large language models. Appl. Sci., 14.","DOI":"10.3390\/app14198868"},{"key":"ref_31","unstructured":"Han, B., Kim, J., and Jang, J. (2024). A dual process vla: Efficient robotic manipulation leveraging vlm. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Gbagbe, K.F., Cabrera, M.A., Alabbas, A., Alyunes, O., Lykov, A., and Tsetserukou, D. (2024). Bi-vla: Vision-language-action model-based system for bimanual robotic dexterous manipulations. Proceedings of the 2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC), IEEE.","DOI":"10.1109\/SMC54092.2024.10831380"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1043","DOI":"10.1080\/0951192X.2023.2177744","article-title":"Model-based hardware in the loop control of collaborative robots: Simulink and Python based interfaces","volume":"37","author":"Safeea","year":"2024","journal-title":"Int. J. Comput. Integr. Manuf."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1109\/MRA.2018.2877776","article-title":"KUKA Sunrise Toolbox: Interfacing Collaborative Robots With MATLAB","volume":"26","author":"Safeea","year":"2019","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Sinico, T., Boschetti, G., and Neto, P. (2025). Enhancing Physical Human-Robot Interaction: Recognizing Digits via Intrinsic Robot Tactile Sensing. Proceedings of the IECON 2025\u201351st Annual Conference of the IEEE Industrial Electronics Society, IEEE.","DOI":"10.1109\/IECON58223.2025.11221741"}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/15\/5\/100\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T08:11:57Z","timestamp":1779178317000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/15\/5\/100"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,18]]},"references-count":35,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,5]]}},"alternative-id":["robotics15050100"],"URL":"https:\/\/doi.org\/10.3390\/robotics15050100","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,18]]}}}