{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T18:21:48Z","timestamp":1781720508594,"version":"3.54.5"},"reference-count":12,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T00:00:00Z","timestamp":1761004800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>The integration of Vision-Language Models (VLMs) into autonomous systems is of growing importance for improving Human-Robot Interaction (HRI), enabling robots to operate within complex and unstructured environments and collaborate with non-expert users. For mobile robots to be effectively deployed in dynamic settings such as domestic or industrial areas, the ability to interpret and execute natural language commands is crucial. However, while VLMs offer powerful zero-shot, open-vocabulary recognition capabilities, their high computational cost presents a significant challenge for real-time performance on resource-constrained edge devices. This study provides a systematic analysis of the trade-offs involved in optimizing a real-time robotic perception pipeline on the NVIDIA Jetson AGX Orin 64GB platform. We investigate the relationship between accuracy and latency by evaluating combinations of two open-vocabulary detection models and two prompt-based segmentation models. Each pipeline is optimized using various precision levels (FP32, FP16, and Best) via NVIDIA TensorRT. We present a quantitative comparison of the mean Intersection over Union (mIoU) and latency for each configuration, offering practical insights and benchmarks for researchers and developers deploying these advanced models on embedded systems.<\/jats:p>","DOI":"10.3389\/frobt.2025.1693988","type":"journal-article","created":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T04:12:58Z","timestamp":1761019978000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Real-time open-vocabulary perception for mobile robots on edge devices: a systematic analysis of the accuracy-latency trade-off"],"prefix":"10.3389","volume":"12","author":[{"given":"Jongyoon","family":"Park","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pileun","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daeil","family":"Ko","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,10,21]]},"reference":[{"key":"B1","article-title":"RT-2: vision-Language-Action models transfer web knowledge to robotic control","volume-title":"Proceedings of the 7th Conference on Robot Learning (CoRL 2023)","author":"Brohan","year":"2023"},{"key":"B2","article-title":"EfficientViT: Multi-Scale linear attention for high-resolution dense prediction","volume-title":"Proceedings of the IEEE\/CVF international conference on computer vision (ICCV 2023)","author":"Cai","year":"2023"},{"key":"B3","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR52733.2024.01599","article-title":"YOLO-World: Real-Time open-vocabulary object detection","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR 2024)","author":"Cheng","year":"2024"},{"key":"B4","article-title":"Open-Vocabulary object detection via vision and Language knowledge distillation","volume-title":"International Conference on Learning Representations (ICLR 2022)","author":"Gu","year":"2022"},{"key":"B5","doi-asserted-by":"crossref","DOI":"10.1109\/ICCV51070.2023.00371","article-title":"Segment anything","volume-title":"Proceedings of the IEEE\/CVF international conference on computer vision (ICCV 2023)","author":"Kirillov","year":"2023"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2406.09039","article-title":"Language-Driven closed-loop grasping with model-predictive trajectory replanning","author":"Lee","year":"2024","journal-title":"arXiv Prepr. arXiv:2406.09039"},{"key":"B7","article-title":"Grounding DINO: marrying DINO with grounded pre-training for open-set object detection","volume-title":"European Conference on Computer Vision (ECCV 2024)","author":"Liu","year":"2024"},{"key":"B8","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-031-20080-9_42","article-title":"Simple open-vocabulary object detection with vision transformers","volume-title":"European conference on Computer Vision (ECCV 2022)","author":"Minderer","year":"2022"},{"key":"B9","article-title":"Learning transferable visual models from natural Language supervision","volume-title":"Proceedings of the 38th international conference on machine learning (ICML 2021)","author":"Radford","year":"2021"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2408.15857","article-title":"What is YOLOv8: an In-Depth exploration of the internal features of the next-generation object detector","author":"Yaseen","year":"2024","journal-title":"arXiv Prepr. arXiv:2408.15857"},{"key":"B11","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-46475-6_5","article-title":"Modeling context in referring expressions","volume-title":"European Conference on computer vision (ECCV 2016)","author":"Yu","year":"2016"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2306.14289","article-title":"Faster segment anything: towards lightweight SAM for mobile applications","author":"Zhang","year":"2023","journal-title":"arXiv Prepr. arXiv:2306.14289"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1693988\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T04:13:01Z","timestamp":1761019981000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1693988\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,21]]},"references-count":12,"alternative-id":["10.3389\/frobt.2025.1693988"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2025.1693988","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,21]]},"article-number":"1693988"}}