{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T05:03:05Z","timestamp":1789707785673,"version":"4.0.1"},"reference-count":56,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T00:00:00Z","timestamp":1789516800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Supercomputing Center","award":["KSC-2025-CRE-0483"],"award-info":[{"award-number":["KSC-2025-CRE-0483"]}],"id":[{"id":"https:\/\/ror.org\/05qbsk739","id-type":"ROR","asserted-by":"crossref"}]},{"award":["KSC-2025-CRE-0483"],"award-info":[{"award-number":["KSC-2025-CRE-0483"]}],"id":[{"id":"https:\/\/ror.org\/00cs28a13","id-type":"ROR","asserted-by":"publisher"}]},{"name":"Ministry of SMEs and Startup","award":["RS-2025-16069404"],"award-info":[{"award-number":["RS-2025-16069404"]}],"id":[{"id":"https:\/\/ror.org\/022c4zk48","id-type":"ROR","asserted-by":"crossref"}]},{"award":["RS-2025-16069404"],"award-info":[{"award-number":["RS-2025-16069404"]}],"id":[{"id":"https:\/\/ror.org\/022c4zk48","id-type":"ROR","asserted-by":"publisher"}]},{"name":"Korea government","award":["RS-2026-25488194"],"award-info":[{"award-number":["RS-2026-25488194"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Biomimetics"],"abstract":"<jats:p>Semantic segmentation for autonomous driving is challenged by limited field-of-view (FoV), where occlusions and restricted viewpoints reduce available visual information. Existing approaches often incur considerable computational overhead, which is unsuitable for autonomous driving systems. To build robust and lightweight semantic segmentation models, attention-based knowledge distillation (KD) can be an efficient alternative. Nevertheless, KD remains primarily visual and provides limited information when visual cues are incomplete or ambiguous. In contrast, biological visual systems integrate visual cues with contextual and semantic knowledge to support robust scene perception. Motivated by this biomimetic principle, language-guided methods provide high-level semantic knowledge, offering another promising direction for enhancing visual understanding. However, the effectiveness of attention-based KD under limited FoV conditions and its interaction with semantic knowledge remain largely unexplored. To systematically investigate this problem, we devise Semantic-guided Attentive Feature Distillation (SAFD), a KD-based framework that combines attention-based feature distillation with semantic guidance derived from large language models (LLMs) without introducing additional inference-time computation. Through comprehensive analyses of multiple distillation strategies and textual representations using the framework, we observe that semantic guidance generally provides additional benefits when the gains from visual distillation are limited, while its effectiveness varies across different attention-based distillation strategies and textual representations. Furthermore, our analyses reveal that different attention mechanisms exploit semantic guidance in distinct ways, with consistent benefits across different pretrained text encoders and improved generalization. These findings provide practical insights into integrating linguistic semantics with visual knowledge transfer under limited FoV.<\/jats:p>","DOI":"10.3390\/biomimetics11090665","type":"journal-article","created":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T16:07:29Z","timestamp":1789574849000},"page":"665","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Exploring the Synergy Between Language Semantic Guidance and Visual Attention in Knowledge Distillation for Semantic Segmentation Under a Limited Field of View"],"prefix":"10.3390","volume":"11","author":[{"given":"Jin Hyeok","family":"Ryu","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Seoul National University of Science and Technology, Seoul 01811, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"}]},{"given":"Seung Woo","family":"Cha","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Seoul National University of Science and Technology, Seoul 01811, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"validation","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1112-4653","authenticated-orcid":false,"given":"Eun Som","family":"Jeon","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Seoul National University of Science and Technology, Seoul 01811, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"},{"role":"funding-acquisition","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1109\/TITS.2020.2972974","article-title":"Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges","volume":"22","author":"Feng","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1109\/TITS.2018.2801309","article-title":"Importance-aware semantic segmentation for autonomous vehicles","volume":"20","author":"Chen","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ranftl, R., Bochkovskiy, A., and Koltun, V. (2021). Vision transformers for dense prediction. Proceedings of the IEEE\/CVF International Conference on Computer Vision, IEEE.","DOI":"10.1109\/ICCV48922.2021.01196"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"5420","DOI":"10.1109\/TII.2025.3555982","article-title":"Local self-attention with mixing abstract tokens for urban autonomous driving","volume":"21","author":"Vo","year":"2025","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4350","DOI":"10.1109\/TITS.2019.2939832","article-title":"Restricted deformable convolution-based road scene semantic segmentation using surround view cameras","volume":"21","author":"Deng","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yogamani, S., Unger, D., Narayanan, V., and Kumar, V.R. (2024). DaF-BEVSeg: Distortion-aware Fisheye Camera based Bird\u2019s Eye View Segmentation with Occlusion Reasoning. arXiv.","DOI":"10.1109\/CVPRW63382.2024.00140"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shi, H., Li, Y., Yang, K., Zhang, J., Peng, K., Roitberg, A., Ye, Y., Ni, H., Wang, K., and Stiefelhagen, R. (2023). FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, IEEE.","DOI":"10.1109\/CVPRW59228.2023.00684"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Pro, F., Dionelis, N., Maiano, L., Le Saux, B., and Amerini, I. (2024). A semantic segmentation-guided approach for ground-to-aerial image matching. Proceedings of the IGARSS 2024-2024 IEEE International Geoscience and Remote Sensing Symposium, IEEE.","DOI":"10.1109\/IGARSS53475.2024.10642526"},{"key":"ref_9","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1789","DOI":"10.1007\/s11263-021-01453-z","article-title":"Knowledge distillation: A survey","volume":"129","author":"Gou","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_11","unstructured":"Zagoruyko, S., and Komodakis, N. (2017). Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. Proceedings of the International Conference on Learning Representations, OpenReview."},{"key":"ref_12","first-page":"7945","article-title":"Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching","volume":"35","author":"Ji","year":"2021","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Shin, S., Lee, J., Lee, J., Yu, Y., and Lee, K. (2022). Teaching where to look: Attention similarity knowledge distillation for low resolution face recognition. Proceedings of the European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-031-19775-8_37"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Shu, C., Liu, Y., Gao, J., Yan, Z., and Shen, C. (2021). Channel-wise knowledge distillation for dense prediction. Proceedings of the IEEE\/CVF International Conference on Computer Vision, IEEE.","DOI":"10.1109\/ICCV48922.2021.00526"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mansourian, A., Jalali, A., Ahmadi, R., and Kasaei, S. (2026). Attention as Geometric Transformation: Revisiting Feature Distillation for Semantic Segmentation. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, IEEE.","DOI":"10.1109\/WACVW68408.2026.00140"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"519","DOI":"10.3758\/BF03197524","article-title":"The effects of contextual scenes on the identification of objects","volume":"3","author":"Palmer","year":"1975","journal-title":"Mem. Cogn."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1016\/j.tics.2007.09.009","article-title":"The role of context in object recognition","volume":"11","author":"Oliva","year":"2007","journal-title":"Trends Cogn. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Biederman, I. (2017). On the semantics of a glance at a scene. Perceptual Organization, Routledge.","DOI":"10.4324\/9781315512372-8"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Du, T., Wang, Z., Wang, Y., Ma, M., and Li, W. (2026). Biologically Inspired Medical Multi-Modal Dataset Distillation via Contrast-Aware Alignment and Memory Compression. Biomimetics, 11.","DOI":"10.3390\/biomimetics11050314"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wu, Y., Mi, Q., and Gao, T. (2025). A comprehensive review of multimodal emotion recognition: Techniques, challenges, and future directions. Biomimetics, 10.","DOI":"10.3390\/biomimetics10070418"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, L.H., Zhang, P., Zhang, H., Yang, J., Li, C., Zhong, Y., Wang, L., Yuan, L., Zhang, L., and Hwang, J.N. (2022). Grounded language-image pre-training. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR52688.2022.01069"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Xu, J., De Mello, S., Liu, S., Byeon, W., Breuel, T., Kautz, J., and Wang, X. (2022). GroupVit: Semantic segmentation emerges from text supervision. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR52688.2022.01760"},{"key":"ref_23","unstructured":"Li, B., Weinberger, K.Q., Belongie, S., Koltun, V., and Ranftl, R. (2022). Language-driven Semantic Segmentation. Proceedings of the International Conference on Learning Representations, OpenReview."},{"key":"ref_24","unstructured":"Menon, S., and Vondrick, C. (2023). Visual Classification via Description from Large Language Models. Proceedings of the International Conference on Learning Representations, OpenReview."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Pratt, S., Covert, I., Liu, R., and Farhadi, A. (2023). What does a platypus look like? generating customized prompts for zero-shot image classification. Proceedings of the IEEE\/CVF International Conference on Computer Vision, IEEE.","DOI":"10.1109\/ICCV51070.2023.01438"},{"key":"ref_26","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning, PMLR."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Rao, Y., Zhao, W., Chen, G., Tang, Y., Zhu, Z., Huang, G., Zhou, J., and Lu, J. (2022). DenseCLIP: Language-guided dense prediction with context-aware prompting. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR52688.2022.01755"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Hoyer, L., Tan, D.J., Naeem, M.F., Van Gool, L., and Tombari, F. (2024). SemiVL: Semi-supervised semantic segmentation with vision-language guidance. Proceedings of the European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-031-72933-1_15"},{"key":"ref_29","unstructured":"Wang, Y., Wang, Y., Dai, R., Wang, Y., Liu, K., Chu, X., and Li, Y. (2026). Urban Socio-Semantic Segmentation with Vision-Language Reasoning. Proceedings of the International Conference on Learning Representations, OpenReview."},{"key":"ref_30","unstructured":"Ma, Y., Ma, J., Zhou, M., Chen, Q., Ge, T., Jiang, Y., and Lin, T. (2021). Boosting image outpainting with semantic layout prediction. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Seifi, S., and Tuytelaars, T. (2020). Attend and segment: Attention guided active semantic segmentation. Proceedings of the European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-030-58595-2_19"},{"key":"ref_32","unstructured":"Ba, L.J., and Caruana, R. (2014). Do deep nets really need to be deep?. Adv. Neural Inf. Process. Syst., 27."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Heo, B., Kim, J., Yun, S., Park, H., Kwak, N., and Choi, J.Y. (2019). A comprehensive overhaul of feature distillation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, IEEE.","DOI":"10.1109\/ICCV.2019.00201"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Chen, P., Liu, S., Zhao, H., and Jia, J. (2021). Distilling knowledge via knowledge review. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR46437.2021.00497"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Tung, F., and Mori, G. (2019). Similarity-preserving knowledge distillation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, IEEE.","DOI":"10.1109\/ICCV.2019.00145"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Park, W., Kim, D., Lu, Y., and Cho, M. (2019). Relational knowledge distillation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR.2019.00409"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Chen, L., Wang, D., Gan, Z., Liu, J., Henao, R., and Carin, L. (2021). Wasserstein contrastive representation distillation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR46437.2021.01603"},{"key":"ref_38","unstructured":"Zhou, Z., Zhuge, C., Guan, X., and Liu, W. (2020). Channel distillation: Channel-wise attention for knowledge distillation. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018). CBAM: Convolutional Block Attention Module. Proceedings of the European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_40","unstructured":"Anthropic (2026, July 19). Claude 4 Model Family Updates. Available online: https:\/\/www.anthropic.com\/claude."},{"key":"ref_41","unstructured":"Google DeepMind (2026, July 19). Gemini 3.5: Frontier Intelligence with Action. Available online: https:\/\/deepmind.google\/models\/gemini\/."},{"key":"ref_42","unstructured":"OpenAI (2025, August 07). GPT-5 System Card Large Language Model. Available online: https:\/\/openai.com\/index\/gpt-5-system-card\/."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Roth, K., Vinyals, O., and Akata, Z. (2022). Integrating language guidance into vision-based deep metric learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR52688.2022.01570"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"El Banani, M., Desai, K., and Johnson, J. (2023). Learning visual representations via language-guided sampling. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR52729.2023.01841"},{"key":"ref_45","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Brostow, G.J., Shotton, J., Fauqueur, J., and Cipolla, R. (2008). Segmentation and recognition using structure from motion point clouds. Proceedings of the European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-540-88682-2_5"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The KITTI dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_49","unstructured":"Kohavi, R. (1995, January 20\u201325). A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the IJCAI, Montreal, QC, Canada."},{"key":"ref_50","first-page":"12077","article-title":"SegFormer: Simple and efficient design for semantic segmentation with transformers","volume":"34","author":"Xie","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuhmann, C., Schmidt, L., and Jitsev, J. (2023). Reproducible scaling laws for contrastive language-image learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR52729.2023.00276"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Zheng, B., and Cheng, R. (2025). Rethinking Decoupled Knowledge Distillation: A Predictive Distribution Perspective. Proceedings of the IEEE Transactions on Neural Networks and Learning Systems, IEEE.","DOI":"10.1109\/TNNLS.2025.3639562"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Zhai, X., Mustafa, B., Kolesnikov, A., and Beyer, L. (2023). Sigmoid loss for language image pre-training. Proceedings of the IEEE\/CVF International Conference on Computer Vision, IEEE.","DOI":"10.1109\/ICCV51070.2023.01100"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics.","DOI":"10.18653\/v1\/N19-1423"},{"key":"ref_55","unstructured":"Guo, C., Pleiss, G., Sun, Y., and Weinberger, K.Q. (2017). On calibration of modern neural networks. Proceedings of the International Conference on Machine Learning, PMLR."},{"key":"ref_56","first-page":"2901","article-title":"Obtaining well calibrated probabilities using Bayesian binning","volume":"29","author":"Naeini","year":"2015","journal-title":"Proc. AAAI Conf. Artif. Intell."}],"container-title":["Biomimetics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-7673\/11\/9\/665\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T04:12:06Z","timestamp":1789704726000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-7673\/11\/9\/665"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,16]]},"references-count":56,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["biomimetics11090665"],"URL":"https:\/\/doi.org\/10.3390\/biomimetics11090665","relation":{},"ISSN":["2313-7673"],"issn-type":[{"value":"2313-7673","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,16]]}}}