{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,7]],"date-time":"2025-10-07T08:28:01Z","timestamp":1759825681505,"version":"3.37.3"},"reference-count":44,"publisher":"World Scientific Pub Co Pte Ltd","issue":"03","funder":[{"DOI":"10.13039\/501100004543","name":"Chinese Scholarship Council","doi-asserted-by":"crossref","award":["202006290010"],"award-info":[{"award-number":["202006290010"]}],"id":[{"id":"10.13039\/501100004543","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Germanys Excellence Strategy","award":["390696704"],"award-info":[{"award-number":["390696704"]}]},{"name":"Germany in the programme of Souvern. Digital. Vernetzt.","award":["16KISK002"],"award-info":[{"award-number":["16KISK002"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Semantic Computing"],"published-print":{"date-parts":[[2023,9]]},"abstract":"<jats:p> In this paper, we propose a hierarchical attention-based sensor fusion strategy for depth estimation under various weather conditions. Multiple-sensor fusion is proven as a promising solution for predicting accurate depth maps in diverse weather conditions, especially for extreme weather conditions. However, most of the current studies simply fuse the information from different sensors without jointly considering the difference in their performance at the sensor level and feature level. To fill this gap, our hierarchical attention-based fusion strategy uses two attention mask-generation modules that weigh sensor data from branches (i.e. different sensors) and features. With the cooperation of these two masks, our system is able to determine the adaptive contribution of each sensor as well as the individual contribution of each feature in the sensor regarding their performance in different weather. We compare the proposed methods with the baseline, i.e. the late fusion Sparse-to-Dense model, and two extended models individually with the branch-wise-only and feature-wise-only masks. The results show a robust and superior performance of our methods even in clear environments where the baseline already performs well enough. Moreover, we investigate the performance of RGB camera, radar, and LiDAR in foggy environments comprehensively by visualizing the generated mask. Our results show a significantly increased importance of radar sensors in extreme weather conditions, e.g. dense fog. <\/jats:p>","DOI":"10.1142\/s1793351x23500022","type":"journal-article","created":{"date-parts":[[2023,6,10]],"date-time":"2023-06-10T05:51:35Z","timestamp":1686376295000},"page":"455-475","source":"Crossref","is-referenced-by-count":2,"title":["Hierarchical Attention-Based Sensor Fusion Strategy for Depth Estimation in Diverse Weather"],"prefix":"10.1142","volume":"17","author":[{"given":"Mengchen","family":"Xiong","sequence":"first","affiliation":[{"name":"Technical University of Munich, School of Computation, Information and Technology, Department of Computer Engineering, Chair of Media Technology, Munich Institute of Robotics and Machine Intelligence (MIRMI), 80333, Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiao","family":"Xu","sequence":"additional","affiliation":[{"name":"Technical University of Munich, School of Computation, Information and Technology, Department of Computer Engineering, Chair of Media Technology, Munich Institute of Robotics and Machine Intelligence (MIRMI), 80333, Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dong","family":"Yang","sequence":"additional","affiliation":[{"name":"Technical University of Munich, School of Computation, Information and Technology, Department of Computer Engineering, Chair of Media Technology, Munich Institute of Robotics and Machine Intelligence (MIRMI), 80333, Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fabian","family":"Seguel","sequence":"additional","affiliation":[{"name":"Technical University of Munich, School of Computation, Information and Technology, Department of Computer Engineering, Chair of Media Technology, Munich Institute of Robotics and Machine Intelligence (MIRMI), 80333, Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eckehard","family":"Steinbach","sequence":"additional","affiliation":[{"name":"Technical University of Munich, School of Computation, Information and Technology, Department of Computer Engineering, Chair of Media Technology, Munich Institute of Robotics and Machine Intelligence (MIRMI), 80333, Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"219","published-online":{"date-parts":[[2023,7,25]]},"reference":[{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Geiger A.","first-page":"3354","key":"S1793351X23500022BIB001"},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Sun P.","first-page":"2446","key":"S1793351X23500022BIB002"},{"volume-title":"IEEE Int. Conf. Robotics and Automation","author":"Yang D.","first-page":"8171","key":"S1793351X23500022BIB003"},{"volume-title":"Proc. IEEE\/RSJ Int. Conf. Intelligent Robots and Systems","author":"Hu G.","first-page":"1714","key":"S1793351X23500022BIB004"},{"issue":"11","key":"S1793351X23500022BIB005","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","volume":"32","author":"Geiger A.","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"S1793351X23500022BIB006","series-title":"LNSC","first-page":"746","volume-title":"European Conf. Computer Vision","volume":"7576","author":"Silberman N.","year":"2012"},{"volume-title":"Proc. IEEE\/CVF Int. Conf. Computer Vision","author":"Wang K.","first-page":"16055","key":"S1793351X23500022BIB007"},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Bijelic M.","first-page":"11682","key":"S1793351X23500022BIB008"},{"volume-title":"Proc. IEEE\/CVF Winter Conf. Applications of Computer Vision","author":"Li A.","first-page":"32","key":"S1793351X23500022BIB009"},{"volume-title":"Proc. IEEE Int. Conf. Robotics and Automation","author":"Ma F.","first-page":"4796","key":"S1793351X23500022BIB010"},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Qian K.","first-page":"444","key":"S1793351X23500022BIB011"},{"issue":"7","key":"S1793351X23500022BIB012","doi-asserted-by":"crossref","first-page":"2148","DOI":"10.3390\/s18072148","volume":"18","author":"Golovachev Y.","year":"2018","journal-title":"Sensors"},{"key":"S1793351X23500022BIB013","first-page":"472","volume-title":"Int. Conf. 3D Imaging, Modeling, Processing, Visualization & Transmission","author":"Van Baar J.","year":"2012"},{"volume-title":"Proc. IEEE\/CVF Int. Conf. Computer Vision","author":"Huynh L.","first-page":"12767","key":"S1793351X23500022BIB014"},{"issue":"3","key":"S1793351X23500022BIB015","doi-asserted-by":"crossref","first-page":"4672","DOI":"10.1109\/LRA.2021.3068712","volume":"6","author":"Choe J.","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"issue":"1","key":"S1793351X23500022BIB016","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1109\/TITS.2019.2891788","volume":"21","author":"Park K.","year":"2019","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Caesar H.","first-page":"11621","key":"S1793351X23500022BIB017"},{"issue":"9","key":"S1793351X23500022BIB018","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1002\/rob.20310","volume":"26","author":"Ryde J.","year":"2009","journal-title":"J. Field Robot."},{"key":"S1793351X23500022BIB019","first-page":"182","volume-title":"Smart Photonic and Optoelectronic Integrated Circuits XX","volume":"10536","author":"Ashraf I.","year":"2018"},{"key":"S1793351X23500022BIB020","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.isprsjprs.2022.12.021","volume":"196","author":"Zhang Y.","year":"2023","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"S1793351X23500022BIB021","first-page":"34","volume-title":"IEEE Int. Symp. Multimedia","author":"Xiong M.","year":"2022"},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Chen C.","first-page":"10542","key":"S1793351X23500022BIB022"},{"issue":"4","key":"S1793351X23500022BIB023","doi-asserted-by":"crossref","first-page":"12038","DOI":"10.1109\/LRA.2022.3210880","volume":"7","author":"Lai H.","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"volume-title":"Proc. IEEE\/CVF Int. Conf. Computer Vision","author":"Godard C.","first-page":"3828","key":"S1793351X23500022BIB024"},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Fu H.","first-page":"2002","key":"S1793351X23500022BIB025"},{"volume-title":"Proc. European Computer Vision Conf. (ECCV)","author":"Klingner M.","first-page":"582","key":"S1793351X23500022BIB026"},{"volume-title":"Proc. Int. Conf. 3D Vision","author":"Gasperini S.","first-page":"751","key":"S1793351X23500022BIB027"},{"volume-title":"Proc. IEEE\/RSJ Int. Conf. Intelligent Robots and Systems","author":"Lin J.-T.","first-page":"10233","key":"S1793351X23500022BIB028"},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Wang F.-E.","first-page":"462","key":"S1793351X23500022BIB030"},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Long Y.","first-page":"12507","key":"S1793351X23500022BIB031"},{"key":"S1793351X23500022BIB032","first-page":"2156","volume-title":"IEEE Int. Conf. Robotics and Automation","author":"Park K.","year":"2018"},{"volume-title":"Proc. IEEE Int. Conf. Image Processing (ICIP)","author":"Lo C.-C.","first-page":"3343","key":"S1793351X23500022BIB033"},{"key":"S1793351X23500022BIB034","first-page":"4573","volume-title":"Int. Conf. Pattern Recognition","author":"Chaturvedi S. S.","year":"2022"},{"volume-title":"Proc. IEEE\/CVF Winter Conf. Applications of Computer Vision","author":"Dai Y.","first-page":"3560","key":"S1793351X23500022BIB035"},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Bhat S. F.","first-page":"4009","key":"S1793351X23500022BIB036"},{"volume-title":"Proc. Asian Computer Vision Conf. Computer Vision","author":"Li R.","first-page":"663","key":"S1793351X23500022BIB037"},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Fu J.","first-page":"3146","key":"S1793351X23500022BIB038"},{"issue":"2","key":"S1793351X23500022BIB039","first-page":"1009","volume-title":"Proc. AAAI Conf. Artificial Intelligence","volume":"35","author":"Chen J."},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"Fukui H.","first-page":"10705","key":"S1793351X23500022BIB040"},{"key":"S1793351X23500022BIB041","doi-asserted-by":"crossref","first-page":"703","DOI":"10.1007\/s11263-020-01398-9","volume":"129","author":"Zhang Z.","year":"2021","journal-title":"Int. J. Comput. Vis."},{"volume-title":"Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition","author":"He K.","first-page":"770","key":"S1793351X23500022BIB042"},{"volume-title":"Proc. Fourth Int. Conf. 3D Vision","author":"Laina I.","first-page":"239","key":"S1793351X23500022BIB043"},{"issue":"2","key":"S1793351X23500022BIB045","doi-asserted-by":"crossref","first-page":"49","DOI":"10.5194\/ars-9-49-2011","volume":"9","author":"Rasshofer R. H.","year":"2011","journal-title":"Adv. Radio Sci."},{"issue":"3","key":"S1793351X23500022BIB046","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"Russakovsky O.","year":"2015","journal-title":"Int. J. Comput. Vis."}],"container-title":["International Journal of Semantic Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S1793351X23500022","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,6]],"date-time":"2023-10-06T06:26:35Z","timestamp":1696573595000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S1793351X23500022"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,25]]},"references-count":44,"journal-issue":{"issue":"03","published-print":{"date-parts":[[2023,9]]}},"alternative-id":["10.1142\/S1793351X23500022"],"URL":"https:\/\/doi.org\/10.1142\/s1793351x23500022","relation":{},"ISSN":["1793-351X","1793-7108"],"issn-type":[{"type":"print","value":"1793-351X"},{"type":"electronic","value":"1793-7108"}],"subject":[],"published":{"date-parts":[[2023,7,25]]}}}