{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T21:35:31Z","timestamp":1772832931666,"version":"3.50.1"},"reference-count":38,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2023,8,28]],"date-time":"2023-08-28T00:00:00Z","timestamp":1693180800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Project of Huzhou Science and Technology","award":["2023GZ68"],"award-info":[{"award-number":["2023GZ68"]}]},{"name":"Project of Huzhou Science and Technology","award":["2021RQ062"],"award-info":[{"award-number":["2021RQ062"]}]},{"name":"High-level Talents Innovation Support Program of Dalian","award":["2023GZ68"],"award-info":[{"award-number":["2023GZ68"]}]},{"name":"High-level Talents Innovation Support Program of Dalian","award":["2021RQ062"],"award-info":[{"award-number":["2021RQ062"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Semantic segmentation and depth estimation are crucial components in the field of autonomous driving for scene understanding. Jointly learning these tasks can lead to a better understanding of scenarios. However, using task-specific networks to extract global features from task-shared networks can be inadequate. To address this issue, we propose a multi-task residual attention network (MTRAN) that consists of a global shared network and two attention networks dedicated to semantic segmentation and depth estimation. The convolutional block attention module is used to highlight the global feature map, and residual connections are added to prevent network degradation problems. To ensure manageable task loss and prevent specific tasks from dominating the training process, we introduce a random-weighted strategy into the impartial multi-task learning method. We conduct experiments to demonstrate the effectiveness of the proposed method.<\/jats:p>","DOI":"10.3390\/s23177466","type":"journal-article","created":{"date-parts":[[2023,8,28]],"date-time":"2023-08-28T06:10:22Z","timestamp":1693203022000},"page":"7466","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Semantic Segmentation and Depth Estimation Based on Residual Attention Mechanism"],"prefix":"10.3390","volume":"23","author":[{"given":"Naihua","family":"Ji","sequence":"first","affiliation":[{"name":"School of Information and Control Engineering, Qingdao University of Technology, Qingdao 266033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huiqian","family":"Dong","sequence":"additional","affiliation":[{"name":"School of Information and Control Engineering, Qingdao University of Technology, Qingdao 266033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fanyun","family":"Meng","sequence":"additional","affiliation":[{"name":"School of Information and Control Engineering, Qingdao University of Technology, Qingdao 266033, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liping","family":"Pang","sequence":"additional","affiliation":[{"name":"School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Zhang, D., Zheng, Z., Wang, T., and He, Y. (2020). HROM: Learning high-resolution representation and object-aware masks for visual object tracking. Sensors, 20.","DOI":"10.3390\/s20174807"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Abdulwahab, S., Rashwan, H.A., Sharaf, N., Khalid, S., and Puig, D. (2023). Deep Monocular Depth Estimation Based on Content and Contextual Features. Sensors, 23.","DOI":"10.3390\/s23062919"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, Q., Chen, L., Shao, M., Liang, H., and Ren, J. (2023). ESAMask: Real-Time Instance Segmentation Fused with Efficient Sparse Attention. Sensors, 23.","DOI":"10.3390\/s23146446"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1612","DOI":"10.1007\/s11431-020-1582-8","article-title":"Monocular depth estimation based on deep learning: An overview","volume":"63","author":"Zhao","year":"2020","journal-title":"Sci. China Technol. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhang, H., Dana, K., Shi, J., Zhang, Z., Wang, X., Tyagi, A., and Agrawal, A. (2018, January 18\u201323). Context encoding for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00747"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Xu, D., Wang, W., Tang, H., Liu, H., Sebe, N., and Ricci, E. (2018, January 18\u201323). Structured attention guided convolutional neural fields for monocular depth estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00412"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Guizilini, V., Hou, R., Li, J., Ambrus, R., and Gaidon, A. (2020). Semantically-guided representation learning for self-supervised monocular depth. arXiv.","DOI":"10.1109\/CVPR42600.2020.00256"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Cui, Z., Xu, C., Yan, Y., Sebe, N., and Yang, J. (2019, January 15\u201320). Pattern-affinitive propagation across depth, surface normal and semantic segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00423"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Liu, S., Johns, E., and Davison, A.J. (2019, January 15\u201320). End-to-end multi-task learning with attention. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00197"},{"key":"ref_10","unstructured":"Chen, Z., Badrinarayanan, V., Lee, C.Y., and Rabinovich, A. (2018, January 10\u201315). Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. Proceedings of the International Conference on Machine Learning, PMLR, Stockholm, Sweden."},{"key":"ref_11","unstructured":"Sener, O., and Koltun, V. (2018, January 3\u20138). Multi-task learning as multi-objective optimization. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_12","unstructured":"Kendall, A., Gal, Y., and Cipolla, R. (2018, January 18\u201323). Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA."},{"key":"ref_13","unstructured":"Liu, L., Li, Y., Kuang, Z., Xue, J.-H., Chen, Y., Yang, W., Liao, Q., and Zhang, W. (2021, January 3\u20137). Towards impartial multi-task learning. Proceedings of the ICLR, Virtual Event, Austria."},{"key":"ref_14","unstructured":"Bilen, H., and Vedaldi, A. (2016, January 5\u201310). Integrated perception with recurrent multi-task neural networks. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Eigen, D., and Fergus, R. (2015, January 7\u201313). Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.304"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3506","DOI":"10.1007\/s10489-020-02042-2","article-title":"Usr-mtl: An unsupervised sentence representation learning framework with multi-task learning","volume":"51","author":"Xu","year":"2021","journal-title":"Appl. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3367","DOI":"10.1007\/s10489-020-01760-x","article-title":"Is position important? deep multi-task learning for aspect-based sentiment analysis","volume":"50","author":"Zhou","year":"2020","journal-title":"Appl. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"109884","DOI":"10.1016\/j.asoc.2022.109884","article-title":"Spatial and temporal saliency based four-stream network with multi-task learning for action recognition","volume":"132","author":"Zong","year":"2023","journal-title":"Appl. Soft Comput."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Yan, B., Jiang, Y., Wu, J., Wang, D., Luo, P., Yuan, Z., and Lu, H. (2023, January 18\u201322). Universal instance perception as object discovery and retrieval. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01471"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1016\/j.jmsy.2021.12.003","article-title":"End to end multi-task learning with attention for multi-objective fault diagnosis under small sample","volume":"62","author":"Xie","year":"2022","journal-title":"J. Manuf. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Cui, Z., Xu, C., Jie, Z., Li, X., and Yang, J. (2018, January 8\u201314). Joint task-recursive learning for semantic segmentation and depth estimation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01249-6_15"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"18167","DOI":"10.1007\/s10489-022-03401-x","article-title":"CI-Net: A joint depth estimation and semantic segmentation network using contextual information","volume":"52","author":"Gao","year":"2022","journal-title":"Appl. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201323). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., and Hu, Q. (2020, January 13\u201319). ECA-Net: Efficient channel attention for deep convolutional neural networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01155"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8\u201314). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1016\/j.neucom.2020.11.046","article-title":"CSART: Channel and spatial attention-guided residual learning for real-time object tracking","volume":"436","author":"Zhang","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., and Lu, H. (2019, January 15\u201320). Dual attention network for scene segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00326"},{"key":"ref_28","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"107893","DOI":"10.1016\/j.patcog.2021.107893","article-title":"Residual multi-task learning for facial landmark localization and expression recognition","volume":"115","author":"Chen","year":"2021","journal-title":"Pattern Recognit."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1016\/j.procs.2021.01.025","article-title":"Deep learning in image classification using residual network (ResNet) variants for detection of colorectal cancer","volume":"179","author":"Sarwinda","year":"2021","journal-title":"Procedia Comput. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ishihara, K., Kanervisto, A., Miura, J., and Hautamaki, V. (2021, January 20\u201325). Multi-task learning with attention for end-to-end autonomous driving. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPRW53098.2021.00325"},{"key":"ref_32","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (July, January 26). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Silberman, N., Hoiem, D., Kohli, P., and Fergus, R. (2012, January 7\u201313). Indoor segmentation and support inference from RGBD images. Proceedings of the Computer Vision\u2013ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy. Proceedings Part V 12.","DOI":"10.1007\/978-3-642-33715-4_54"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"3505","DOI":"10.1007\/s10489-022-03617-x","article-title":"Multi-task learning based on geometric invariance discriminative features","volume":"53","author":"Liu","year":"2023","journal-title":"Appl. Intell."},{"key":"ref_35","unstructured":"Misra, I., Shrivastava, A., Gupta, A., and Hebert, M. (July, January 26). Cross-stitch networks for multi-task learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_36","first-page":"18878","article-title":"Conflict-averse gradient descent for multi-task learning","volume":"34","author":"Liu","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_37","first-page":"5824","article-title":"Gradient surgery for multi-task learning","volume":"33","author":"Yu","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_38","first-page":"2039","article-title":"Just pick a sign: Optimizing deep multitask models with gradient sign dropout","volume":"33","author":"Chen","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/17\/7466\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:40:45Z","timestamp":1760128845000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/17\/7466"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,28]]},"references-count":38,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["s23177466"],"URL":"https:\/\/doi.org\/10.3390\/s23177466","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,28]]}}}