{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T12:46:48Z","timestamp":1788266808654,"version":"build-2803163510"},"reference-count":40,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T00:00:00Z","timestamp":1773878400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Israeli Innovation Authority","award":["81938"],"award-info":[{"award-number":["81938"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Autonomous systems are expanding rapidly, driving a demand for robust perception technologies capable of navigating challenging, unstructured environments. While urban autonomy has made significant progress, off-road environments pose unique challenges, including dynamic terrain and limited communication infrastructure. This research addresses these challenges by introducing a novel three-part solution for off-road autonomous vehicles. First, we present a large-scale off-road dataset curated to capture the visual complexity and variability of unstructured environments, providing a realistic training ground that supports improved model generalization. Second, we propose a Confusion-Aware Loss (CAL) that dynamically penalizes systematic misclassifications based on class-level confusion statistics. When combined with cross-entropy, CAL improves segmentation mean Intersection over Union (mIoU) on the off-road test set from 68.66% to 70.06% and achieves cross-domain gains of up to ~0.49% mIoU on the Cityscapes dataset. Third, leveraging semantic segmentation as an intermediate representation, we introduce a spatial overlay video encoding scheme that preserves high-fidelity RGB information in semantically critical regions while compressing non-essential background regions. Experimental results demonstrate Peak Signal-to-Noise Ratio (PSNR) improvements of up to +5 dB and Video Multi-Method Assessment Fusion (VMAF) gains of up to +40 points under lossy compression, enabling efficient and reliable off-road autonomous operation. This integrated approach provides a robust framework for real-time remote operation in bandwidth-constrained environments.<\/jats:p>","DOI":"10.3390\/s26061944","type":"journal-article","created":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T15:38:01Z","timestamp":1773934681000},"page":"1944","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Off-Road Autonomous Vehicle Semantic Segmentation and Spatial Overlay Video Assembly"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6342-5223","authenticated-orcid":false,"given":"Itai","family":"Dror","sequence":"first","affiliation":[{"name":"School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beer Sheva 8410501, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-8775-5963","authenticated-orcid":false,"given":"Omer","family":"Aviv","sequence":"additional","affiliation":[{"name":"School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beer Sheva 8410501, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6089-8401","authenticated-orcid":false,"given":"Ofer","family":"Hadar","sequence":"additional","affiliation":[{"name":"School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beer Sheva 8410501, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,3,19]]},"reference":[{"key":"ref_1","first-page":"169","article-title":"A Comprehensive Exploration of Resource Allocation Strategies within Vehicle Ad-Hoc Networks","volume":"2","author":"Sheela","year":"2023","journal-title":"Mechatron. Intell. Transp. Syst."},{"key":"ref_2","first-page":"154","article-title":"Enhancing Performance and Reducing Latency in Autonomous Systems Through Edge Computing for Real-Time Data Processing","volume":"4","author":"Pallikonda","year":"2025","journal-title":"Mechatron. Intell. Transp. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27\u201330). The Cityscapes Dataset for Semantic Urban Scene Understanding. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.patrec.2008.04.005","article-title":"Semantic object classes in video: A high-definition ground truth database","volume":"30","author":"Brostow","year":"2009","journal-title":"Pattern Recognit. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Valada, A., Oliveira, G.L., Brox, T., and Burgard, W. (2016). Deep multispectral semantic scene understanding of forested environments using multimodal fusion. International Symposium on Experimental Robotics, Springer.","DOI":"10.1007\/978-3-319-50115-4_41"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Maturana, D., Chou, P.W., Uenoyama, M., and Scherer, S. (2017). Real-time semantic mapping for autonomous off-road navigation. Field and Service Robotics: Results of the 11th International Conference, Springer.","DOI":"10.1007\/978-3-319-67361-5_22"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Min, C., Jiang, W., Zhao, D., Xu, J., Xiao, L., Nie, Y., and Dai, B. (2022, January 23\u201327). ORFD: A Dataset and Benchmark for Off-Road Freespace Detection. Proceedings of the 2022 IEEE International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9812139"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Jiang, P., Osteen, P., Wigness, M., and Saripalli, S. (June, January 30). RELLIS-3D Dataset: Data, Benchmarks and Analysis. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561251"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Viswanath, K., Singh, K., Jiang, P., Sujit, P.B., and Saripalli, S. (2021, January 23\u201327). OffSeg: A Semantic Segmentation Framework for Off-Road Driving. Proceedings of the 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE), Lyon, France.","DOI":"10.1109\/CASE49439.2021.9551643"},{"key":"ref_10","unstructured":"Shaban, A., Meng, X., Lee, J., Boots, B., and Fox, D. (2022). Semantic terrain classification for off-road autonomous driving. Conference on Robot Learning, PMLR."},{"key":"ref_11","unstructured":"Guan, T., He, Z., Song, R., Manocha, D., and Zhang, L. (July, January 27). TNS: Terrain Traversability Mapping and Navigation System for Autonomous Excavators. Proceedings of the Robotics: Science and Systems (RSS), New York, NY, USA."},{"key":"ref_12","unstructured":"Min, C., Mei, J., Zhai, H., Wang, S., Sun, T., Kong, F., Li, H., Mao, F., Liu, F., and Wang, S. (2025). Advancing Off-Road Autonomous Driving: The Large-Scale ORAD-3D Dataset and Comprehensive Benchmarks. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Medellin, A., Bhamri, A., Ma, A., Lanagri, R., Gopalswamy, S., Grabowsky, D., and Mikulski, D. (2024). Applications of Unifying Off-Road Datasets Through Ontology, SAE. Technical Report, SAE Technical Paper.","DOI":"10.4271\/2024-01-4074"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1007\/s10846-024-02114-2","article-title":"OffRoadSynth Open Dataset for Semantic Segmentation Using Synthetic-Data-Based Weight Initialization for Autonomous UGV in Off-Road Environments","volume":"110","author":"Kordecki","year":"2024","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1007\/s10846-025-02340-2","article-title":"A High-fidelity Multimodal Synthetic Dataset Generation Framework for Off-road Unstructured Terrain Navigation Training of Autonomous Robots","volume":"112","author":"Wijayathunga","year":"2026","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Xu, J., Xiong, Z., and Bhattacharyya, S.P. (2023, January 17\u201324). PIDNet: A Real-Time Semantic Segmentation Network Inspired by PID Controllers. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01871"},{"key":"ref_17","unstructured":"Hong, Y., Pan, H., Sun, W., and Jia, Y. (2021). Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., and Sang, N. (2018, January 8\u201314). BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01261-8_20"},{"key":"ref_19","first-page":"12077","article-title":"SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers","volume":"34","author":"Xie","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst. (NeurIPS)"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., and Torr, P.H.S. (2021, January 20\u201325). Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00681"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Sharma, S., Ball, J.E., Tang, B., Carruth, D.W., Doude, M., and Islam, M.A. (2019). Semantic segmentation with transfer learning for off-road autonomous driving. Sensors, 19.","DOI":"10.3390\/s19112577"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ye, S., Wu, Q., Fan, P., and Fan, Q. (2025). A survey on semantic communications in internet of vehicles. Entropy, 27.","DOI":"10.3390\/e27040445"},{"key":"ref_23","unstructured":"Lv, J., Tong, H., Pan, Q., Zhang, Z., He, X., Luo, T., and Yin, C. (2024). Importance-aware image segmentation-based semantic communication for autonomous driving. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Shrivastava, A., Gupta, A., and Girshick, R. (2016, January 27\u201330). Training Region-Based Object Detectors with Online Hard Example Mining. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.89"},{"key":"ref_25","first-page":"200629","article-title":"IndiSegNet: Real-Time Semantic Segmentation for Unstructured Road Scenes in Intelligent Transportation Systems","volume":"29","author":"Chakraborty","year":"2026","journal-title":"Intell. Syst. Appl."},{"key":"ref_26","unstructured":"Niedermayer, M., Rice, D., and Martinez, J. (2026, March 15). RFC 9043: FFV1 Video Coding Format Versions 0, 1, and 3. Available online: https:\/\/datatracker.ietf.org\/doc\/html\/rfc9043."},{"key":"ref_27","unstructured":"FFmpeg Team (2026, March 10). FFmpeg, Available online: https:\/\/www.ffmpeg.org\/."},{"key":"ref_28","unstructured":"Sithu, M. (2026, March 15). Semantic-Segmentation: A Collection of Various Semantic Segmentation Models. GitHub Repository. Available online: https:\/\/github.com\/sithu31296\/semantic-segmentation."},{"key":"ref_29","unstructured":"Azad, R., Heidari, M., Yilmaz, K., H\u00fcttemann, M., Karimijafarbigloo, S., Wu, Y., Schmeink, A., and Merhof, D. (2023). Loss Functions in the Era of Semantic Segmentation: A Survey and Outlook. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Csurka, G., Volpi, R., and Chidlovskii, B. (2023). Semantic image segmentation: Two decades of research. arXiv.","DOI":"10.1561\/9781638280774"},{"key":"ref_31","unstructured":"Loshchilov, I., and Hutter, F. (2019, January 6\u20139). Decoupled Weight Decay Regularization. Proceedings of the International Conference on Learning Representations (ICLR), New Orleans, LA, USA."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_33","unstructured":"Loshchilov, I., and Hutter, F. (2016). Sgdr: Stochastic gradient descent with warm restarts. arXiv."},{"key":"ref_34","unstructured":"Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H., and Smith, N. (2020). Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv."},{"key":"ref_35","unstructured":"Li, Z., Aaron, A., Katsavounidis, I., Anantharaman, A., and Guo, F. (2026, March 15). Toward a Practical Perceptual Video Quality Metric. Netflix Technology Blog. Available online: https:\/\/netflixtechblog.com\/toward-a-practical-perceptual-video-quality-metric-653f208b9652."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Kamtam, S.B., Lu, Q., Bouali, F., Haas, O.C., and Birrell, S. (2024). Network Latency in Teleoperation of Connected and Autonomous Vehicles: A Review of Trends, Challenges, and Mitigation Strategies. Sensors, 24.","DOI":"10.3390\/s24123957"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Adwani, N., Silvestrini-Cordero, K., Rojas-Cessa, R., Han, T., and Wang, C. (2024, January 15\u201319). A special-purpose video streaming codec for internet-based remote driving. Proceedings of the 2024 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), Boston, MA, USA.","DOI":"10.1109\/AIM55361.2024.10637227"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Nowakowski, M. (2024, January 4\u20136). Operational Environment Impact on Sensor Capabilities in Special Purpose Unmanned Ground Vehicles. Proceedings of the 2024 21st International Conference on Mechatronics-Mechatronika (ME), Brno, Czech Republic.","DOI":"10.1109\/ME61309.2024.10789716"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1109\/MNET.2024.3468328","article-title":"Semantic Communication for Edge Intelligence Enabled Autonomous Driving System","volume":"39","author":"Feng","year":"2025","journal-title":"IEEE Netw."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2568","DOI":"10.4271\/2025-01-8339","article-title":"A Review of Off-Road Datasets, Sensor Technologies and Terrain Traversability Analysis","volume":"5","author":"Musau","year":"2025","journal-title":"SAE Int. J. Adv. Curr. Pract. Mobil."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/6\/1944\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T09:48:13Z","timestamp":1774000093000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/6\/1944"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,19]]},"references-count":40,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["s26061944"],"URL":"https:\/\/doi.org\/10.3390\/s26061944","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,19]]}}}