{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,7]],"date-time":"2026-01-07T23:29:48Z","timestamp":1767828588676,"version":"3.49.0"},"reference-count":21,"publisher":"Springer Science and Business Media LLC","issue":"12","license":[{"start":{"date-parts":[[2025,8,31]],"date-time":"2025-08-31T00:00:00Z","timestamp":1756598400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,31]],"date-time":"2025-08-31T00:00:00Z","timestamp":1756598400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CNS 2321531"],"award-info":[{"award-number":["CNS 2321531"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Vis"],"published-print":{"date-parts":[[2025,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Teleoperation can enable human intervention to help handle instances of failure in autonomy thus allowing for much safer deployment of autonomous vehicle technology. Successful teleoperation requires recreating the environment around the remote vehicle using camera data received over wireless communication channels. This paper develops a new predictive display system to tackle the significant time delays encountered in receiving camera data over wireless networks. First, a new high gain observer is developed for estimating the position and orientation of the ego vehicle. The novel observer is shown to perform accurate state estimation using only GNSS and gyroscope sensor readings. A vector field method which fuses the delayed camera and Lidar data is then presented. This method uses sparse 3D points obtained from Lidar and transforms them using the state estimates from the high gain observer to generate a sparse vector field for the camera image. Polynomial based interpolation is then performed to obtain the vector field for the complete image which is then remapped to synthesize images for accurate predictive display. The method is evaluated on real-world experimental data from the nuScenes and KITTI datasets. The performance of the high gain observer is also evaluated and compared with that of the EKF. The synthesized images using the vector field based predictive display are compared with ground truth images using various image metrics and offer vastly improved performance compared to delayed images.<\/jats:p>","DOI":"10.1007\/s11263-025-02550-z","type":"journal-article","created":{"date-parts":[[2025,8,31]],"date-time":"2025-08-31T12:00:38Z","timestamp":1756641638000},"page":"8315-8331","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Predictive Display for Teleoperation Based on Vector Fields Using Lidar-Camera Fusion"],"prefix":"10.1007","volume":"133","author":[{"given":"Gaurav","family":"Sharma","sequence":"first","affiliation":[]},{"given":"Jeff","family":"Calder","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9931-7419","authenticated-orcid":false,"given":"Rajesh","family":"Rajamani","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,8,31]]},"reference":[{"key":"2550_CR1","unstructured":"Chucholowski, F. (2016a). Eine vorausschauende Anzeige zur Teleoperation von Stra\u00dfenfahrzeugen. PhD thesis"},{"key":"2550_CR2","doi-asserted-by":"publisher","first-page":"80","DOI":"10.21535\/just.v3i3.38","volume":"3","author":"F Chucholowski","year":"2016","unstructured":"Chucholowski, F. (2016b). Evaluation of display methods for teleoperation of road vehicles. Journal of Unmanned System Technology, 3, 80\u201385. https:\/\/doi.org\/10.21535\/just.v3i3.38","journal-title":"Journal of Unmanned System Technology"},{"issue":"3","key":"2550_CR3","doi-asserted-by":"publisher","first-page":"590","DOI":"10.1109\/TRO.2010.2046695","volume":"26","author":"J Davis","year":"2010","unstructured":"Davis, J., Smyth, C., & McDowell, K. (2010). The effects of time lag on driving performance and possible mitigation. IEEE Transactions on Robotics, 26(3), 590\u2013593.","journal-title":"IEEE Transactions on Robotics"},{"key":"2550_CR4","doi-asserted-by":"publisher","first-page":"102536","DOI":"10.1016\/j.ijhcs.2020.102536","volume":"145","author":"H Dybvik","year":"2021","unstructured":"Dybvik, H., L\u00f8land, M., Gerstenberg, A., Sl \u030aattsveen, K. B., & Steinert, M. (2021). A low-cost predictive display for teleoperation: Investigating effects on human performance and workload. International Journal of Human-Computer Studies, 145, 102536.","journal-title":"International Journal of Human-Computer Studies"},{"issue":"2","key":"2550_CR5","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1177\/001872088803000207","volume":"30","author":"LH Frank","year":"1988","unstructured":"Frank, L. H., Casali, J. G., & Wierwille, W. W. (1988). Effects of visual display and motion system delays on operator performance and uneasiness in a driving simulator. Human Factors, 30(2), 201\u2013217.","journal-title":"Human Factors"},{"issue":"no. 4","key":"2550_CR6","doi-asserted-by":"publisher","first-page":"975","DOI":"10.1137\/S0363012991221791","volume":"32","author":"JP Gauthier","year":"1994","unstructured":"Gauthier, J. P., & Kupka, I. (1994). Observability and observers for nonlinear systems. SIAM Journal on Control and Optimization, 32(4), 975\u2013994.","journal-title":"SIAM Journal on Control and Optimization"},{"key":"2550_CR7","doi-asserted-by":"crossref","unstructured":"Graf, G., Xu, H., Schitz, D., & Xu, X. (2020) Improving the prediction accuracy of predictive displays for teleoperated autonomous vehicles. In 2020 6th International Conference on Control, Automation and Robotics (ICCAR) (pp. 440\u2013445)","DOI":"10.1109\/ICCAR49639.2020.9108011"},{"key":"2550_CR8","unstructured":"Halo launches the first 5G tele-operated driverless car service in Las Vegas. (2021). URL: https:\/\/www.fanaticalfuturist.com\/2021\/07\/halo-launches-the-first-5g-tele-operated-driverless-car-service-in-las-vegas\/"},{"key":"2550_CR9","doi-asserted-by":"crossref","unstructured":"Hu, X. et al. (2023) A dynamic multi-scale voxel flow network for video prediction. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition.","DOI":"10.1109\/CVPR52729.2023.00593"},{"key":"2550_CR10","unstructured":"Hu, A. et al. (2023). Gaia-1: A generative world model for autonomous driving. arXiv preprint arXiv:2309.17080."},{"key":"2550_CR11","unstructured":"Jocher, G., Chaurasia, A., & Qiu, J. (2023). Ultralytics YOLO (Version 8.0.0) [Computer software]. https:\/\/github.com\/ultralytics\/ultralytics"},{"key":"2550_CR12","volume-title":"Nonlinear control","author":"HK Khalil","year":"2015","unstructured":"Khalil, H. K. (2015). Nonlinear control (Vol. 406). Pearson."},{"key":"2550_CR13","unstructured":"Ming, R. et al. (2024). A survey on video prediction: from deterministic to generative approaches. arXiv preprint arXiv:2401.14718"},{"key":"2550_CR14","doi-asserted-by":"publisher","DOI":"10.1007\/s10846-022-01749-3","author":"MD Moniruzzaman","year":"2022","unstructured":"Moniruzzaman, M. D., Rassau, A., Chai, D., & Islam, S. M. S. (2022). High latency unmanned ground vehicle teleoperation enhancement by presentation of estimated future through video transformation. Journal of Intelligent & Robotic Systems. https:\/\/doi.org\/10.1007\/s10846-022-01749-3","journal-title":"Journal of Intelligent & Robotic Systems"},{"key":"2550_CR15","doi-asserted-by":"crossref","unstructured":"Neumeier, S., Gay, N., Dannheim, C., & Facchi, C. (2018). On the Way to Autonomous Vehicles Teleoperated Driving. In AmE 2018 - Automotive meets Electronics; 9th GMM-Symposium, Dortmund, Germany (pp. 1\u20136)","DOI":"10.1109\/ICCVE45908.2019.8965037"},{"key":"2550_CR16","unstructured":"Remote operation and the future of trucking. (2021). URL: https:\/\/einride.tech\/insights\/remote-operation-and-the-future-of-trucking"},{"key":"2550_CR17","doi-asserted-by":"crossref","unstructured":"Sato, Y. et al. (2021). Implementation and evaluation of latency visualization method for teleoperated vehicle. In 2021 IEEE Intelligent Vehicles Symposium (IV), (pp. 1\u20137).","DOI":"10.1109\/IV48863.2021.9575817"},{"key":"2550_CR18","doi-asserted-by":"publisher","unstructured":"Sharma, G., Manekar, R., & Rajamani, R. (2025). Predictive display for teleoperation based on 3D reconstruction using lidar-camera fusion. In IEEE Transactions on Intelligent Transportation Systems. https:\/\/doi.org\/10.1109\/TITS.2025.3566016.","DOI":"10.1109\/TITS.2025.3566016"},{"issue":"3","key":"2550_CR19","doi-asserted-by":"publisher","first-page":"4456","DOI":"10.1109\/TIV.2024.3360410","volume":"9","author":"G Sharma","year":"2024","unstructured":"Sharma, G., & Rajamani, R. (2024). Teleoperation enhancement for autonomous vehicles using estimation based predictive display. IEEE Transactions on Intelligent Vehicles, 9(3), 4456\u20134469. https:\/\/doi.org\/10.1109\/TIV.2024.3360410","journal-title":"IEEE Transactions on Intelligent Vehicles"},{"key":"2550_CR20","unstructured":"Startup Vay\u2019s Autonomy Workaround: Teledrivers to Operate Cars from Remote Location. (2021). URL: https:\/\/www.caranddriver.com\/news\/a37648114\/vay-autonomous-teledriver-startup\/"},{"issue":"6","key":"2550_CR21","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1109\/THMS.2020.3018684","volume":"50","author":"Y Zheng","year":"2020","unstructured":"Zheng, Y., Brudnak, M. J., Jayakumar, P., Stein, J. L., & Ersal, T. (2020). Evaluation of a predictor-based framework in high-speed teleoperated military UGVs. IEEE Transactions on Human-Machine Systems, 50(6), 561\u2013572.","journal-title":"IEEE Transactions on Human-Machine Systems"}],"container-title":["International Journal of Computer Vision"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-025-02550-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11263-025-02550-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-025-02550-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,6]],"date-time":"2025-12-06T16:01:21Z","timestamp":1765036881000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11263-025-02550-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,31]]},"references-count":21,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2025,12]]}},"alternative-id":["2550"],"URL":"https:\/\/doi.org\/10.1007\/s11263-025-02550-z","relation":{},"ISSN":["0920-5691","1573-1405"],"issn-type":[{"value":"0920-5691","type":"print"},{"value":"1573-1405","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,31]]},"assertion":[{"value":"20 February 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 July 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 August 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}