{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T11:34:13Z","timestamp":1782473653739,"version":"3.54.5"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,11,20]],"date-time":"2023-11-20T00:00:00Z","timestamp":1700438400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Grab-NUS AI Lab"},{"name":"GrabTaxi Holdings Pte. Ltd."},{"DOI":"10.13039\/501100001352","name":"National University of Singapore","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001352","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Industrial Postgraduate Program","award":["S18-1198-IPP-II"],"award-info":[{"award-number":["S18-1198-IPP-II"]}]},{"DOI":"10.13039\/501100001446","name":"Economic Development Board of Singapore","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001446","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Singapore Ministry of Education Academic Research Fund Tier 2","award":["T2EP20221-0023"],"award-info":[{"award-number":["T2EP20221-0023"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Spatial Algorithms Syst."],"published-print":{"date-parts":[[2023,12,31]]},"abstract":"<jats:p>Automatic inference of missing road attributes (e.g., road type and speed limit) for enriching digital maps has attracted significant research attention in recent years. A number of machine learning-based approaches have been proposed to detect road attributes from GPS traces, dash-cam videos, or satellite images. However, existing solutions mostly focus on a single modality without modeling the correlations among multiple data sources. To bridge this gap, we present a multimodal road attribute detection method, which improves the robustness by performing pixel-level fusion of crowdsourced GPS traces and satellite images. A GPS trace is usually given by a sequence of location, bearing, and speed. To align it with satellite imagery in the spatial domain, we render GPS traces into a sequence of multi-channel images that simultaneously capture the global distribution of the GPS points, the local distribution of vehicles\u2019 moving directions and speeds, and their temporal changes over time, at each pixel. Unlike previous GPS-based road feature extraction methods, our proposed GPS rendering does not require map matching in the data preprocessing step. Moreover, our multimodal solution addresses single-modal challenges such as occlusions in satellite images and data sparsity in GPS traces by learning the pixel-wise correspondences among different data sources. On top of this, we observe that geographic objects and their attributes in the map are not isolated but correlated with each other. Thus, if a road is partially labeled, then the existing information can be of great help on inferring the missing attributes. To fully use the existing information, we extend our model and discuss the possibilities for further performance improvement when partially labeled map data is available. Extensive experiments have been conducted on two real-world datasets in Singapore and Jakarta. Compared with previous work, our method is able to improve the detection accuracy on road attributes by a large margin.<\/jats:p>","DOI":"10.1145\/3618108","type":"journal-article","created":{"date-parts":[[2023,9,2]],"date-time":"2023-09-02T11:31:40Z","timestamp":1693654300000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Multimodal Deep Learning for Robust Road Attribute Detection"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6525-6133","authenticated-orcid":false,"given":"Yifang","family":"Yin","sequence":"first","affiliation":[{"name":"Institute for Infocomm Research, A*STAR, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5902-2306","authenticated-orcid":false,"given":"Wenmiao","family":"Hu","sequence":"additional","affiliation":[{"name":"GrabTaxi Holdings, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4355-812X","authenticated-orcid":false,"given":"An","family":"Tran","sequence":"additional","affiliation":[{"name":"GrabTaxi Holdings, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6411-4486","authenticated-orcid":false,"given":"Ying","family":"Zhang","sequence":"additional","affiliation":[{"name":"Northwestern Polytechnical University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4224-1449","authenticated-orcid":false,"given":"Guanfeng","family":"Wang","sequence":"additional","affiliation":[{"name":"GrabTaxi Holdings, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-2616-3469","authenticated-orcid":false,"given":"Hannes","family":"Kruppa","sequence":"additional","affiliation":[{"name":"GrabTaxi Holdings, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7410-2590","authenticated-orcid":false,"given":"Roger","family":"Zimmermann","sequence":"additional","affiliation":[{"name":"Grab-NUS AI Lab, NUS, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6565-7511","authenticated-orcid":false,"given":"See-Kiong","family":"Ng","sequence":"additional","affiliation":[{"name":"Grab-NUS AI Lab, NUS, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,11,20]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"DiDi Chuxing. 2022. GAIA Open Dataset Initiative. Retrieved from https:\/\/outreach.didichuxing.com\/research\/opendata\/en\/"},{"key":"e_1_3_1_3_2","unstructured":"Mapbox. 2022. Static Images API. Retrieved from https:\/\/docs.mapbox.com\/api\/maps\/static-images\/"},{"key":"e_1_3_1_4_2","unstructured":"Kaggle. 2022. Uber Pickups in New York City. Retrieved from https:\/\/www.kaggle.com\/fivethirtyeight\/uber-pickups-in-new-york-city"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2015.10.019"},{"key":"e_1_3_1_6_2","first-page":"5:1\u20135:10","volume-title":"Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems","author":"Aly Heba","year":"2015","unstructured":"Heba Aly and Moustafa Youssef. 2015. semMatch: Road semantics-based accurate map matching for challenging positioning data. In Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 5:1\u20135:10."},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3274895.3274927"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00496"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3474717.3483651"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3450351"},{"key":"e_1_3_1_11_2","first-page":"11905","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Bastani Favyen","year":"2021","unstructured":"Favyen Bastani and Samuel Madden. 2021. Beyond road extraction: A dataset for map update using aerial images. In Proceedings of the IEEE\/CVF International Conference on Computer Vision. 11905\u201311914."},{"key":"e_1_3_1_12_2","first-page":"79","volume-title":"Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems","author":"Biagioni James","year":"2012","unstructured":"James Biagioni and Jakob Eriksson. 2012. Map inference in the face of noise and disparity. In Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 79\u201388."},{"key":"e_1_3_1_13_2","article-title":"Assigning a grade: Accurate measurement of road quality using satellite imagery","author":"Cadamuro Gabriel","year":"2018","unstructured":"Gabriel Cadamuro, Aggrey Muhebwa, and Jay Taneja. 2018. Assigning a grade: Accurate measurement of road quality using satellite imagery. Retrieved from https:\/\/arXiv:1812.01699","journal-title":"R"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2699184"},{"key":"e_1_3_1_15_2","first-page":"81","volume-title":"Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems","author":"Chen Yihua","year":"2010","unstructured":"Yihua Chen and John Krumm. 2010. Probabilistic modeling of traffic lanes from GPS traces. In Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 81\u201388."},{"key":"e_1_3_1_16_2","first-page":"6172","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201918)","author":"Christie Gordon","year":"2018","unstructured":"Gordon Christie, Neil Fendley, James Wilson, and Ryan Mukherjee. 2018. Functional map of the world. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201918). 6172\u20136180."},{"issue":"3","key":"e_1_3_1_17_2","doi-asserted-by":"crossref","first-page":"1182","DOI":"10.1109\/TNSE.2019.2913233","article-title":"Spectral alignment of graphs","volume":"7","author":"Feizi Soheil","year":"2019","unstructured":"Soheil Feizi, Gerald Quon, Mariana Recamonde-Mendoza, Muriel Medard, Manolis Kellis, and Ali Jadbabaie. 2019. Spectral alignment of graphs. IEEE Trans. Netw. Sci. Eng. 7, 3 (2019), 1182\u20131197.","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.5555\/3045776.3045780"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/MPRV.2008.80"},{"key":"e_1_3_1_20_2","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1007\/978-0-387-84858-7_6","volume-title":"The Elements of Statistical Learning","author":"Hastie Trevor","year":"2009","unstructured":"Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2009. Kernel smoothing methods. In The Elements of Statistical Learning. Springer, Berlin, 191\u2013218."},{"key":"e_1_3_1_21_2","first-page":"2080","volume-title":"Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision","author":"He Songtao","year":"2022","unstructured":"Songtao He and Hari Balakrishnan. 2022. Lane-level street map extraction from aerial imagery. In Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision. 2080\u20132089."},{"key":"e_1_3_1_22_2","first-page":"10965","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"34","author":"He Songtao","year":"2020","unstructured":"Songtao He, Favyen Bastani, Satvat Jagwani, Edward Park, Sofiane Abbar, Mohammad Alizadeh, Hari Balakrishnan, Sanjay Chawla, Samuel Madden, and Mohammad Amin Sadeghi. 2020. RoadTagger: Robust road attribute inference with graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 10965\u201310972."},{"key":"e_1_3_1_23_2","article-title":"Mobilenets: Efficient convolutional neural networks for mobile vision applications","author":"Howard Andrew G.","year":"2017","unstructured":"Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. Retrieved from https:\/\/arXiv:1704.04861","journal-title":"R"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.243"},{"key":"e_1_3_1_25_2","first-page":"1","volume-title":"Proceedings of the ACM SIGSPATIAL International Workshop on Prediction of Human Mobility","author":"Huang Xiaocheng","year":"2019","unstructured":"Xiaocheng Huang, Yifang Yin, Simon Lim, Guanfeng Wang, Bo Hu, Jagannadan Varadarajan, Shaolin Zheng, Ajay Bulusu, and Roger Zimmermann. 2019. Grab-Posisi: An extensive real-life GPS trajectory dataset in southeast asia. In Proceedings of the ACM SIGSPATIAL International Workshop on Prediction of Human Mobility. 1\u201310."},{"key":"e_1_3_1_26_2","first-page":"1","volume-title":"Proceedings of the International Conference on Image and Vision Computing New Zealand","author":"Jan Z.","year":"2018","unstructured":"Z. Jan, B. Verma, J. Affum, S. Atabak, and L. Moir. 2018. A convolutional neural network based deep learning technique for identifying road attributes. In Proceedings of the International Conference on Image and Vision Computing New Zealand. 1\u20136."},{"key":"e_1_3_1_27_2","first-page":"1097","volume-title":"Advances in Neural Information Processing Systems","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. MIT Press, 1097\u20131105."},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1111\/tgis.12072"},{"key":"e_1_3_1_29_2","volume-title":"Proceedings of the ACM SIGSPATIAL Internetional Conference on Advances in Geographical Information Systems","author":"Lou Yin","year":"2009","unstructured":"Yin Lou, Chengyang Zhang, Xing Xie, Yu Zheng, Wei Wang, and Yan Huang. 2009. Map-matching for low-sampling-rate GPS trajectories. In Proceedings of the ACM SIGSPATIAL Internetional Conference on Advances in Geographical Information Systems."},{"key":"e_1_3_1_30_2","first-page":"336","volume-title":"Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems","author":"Newson Paul","year":"2009","unstructured":"Paul Newson and John Krumm. 2009. Hidden Markov map matching through noise and sparseness. In Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. 336\u2013343."},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_3_1_32_2","first-page":"890","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"34","author":"Ruan Sijie","year":"2020","unstructured":"Sijie Ruan, Cheng Long, Jie Bao, Chunyang Li, Zisheng Yu, Ruiyuan Li, Yuxuan Liang, Tianfu He, and Yu Zheng. 2020. Learning to generate maps from trajectories. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 890\u2013897."},{"key":"e_1_3_1_33_2","first-page":"7509","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201919)","author":"Sun Tao","year":"2019","unstructured":"Tao Sun, Zonglin Di, Pengyu Che, Chun Liu, and Yin Wang. 2019. Leveraging Crowdsourced GPS data for road extraction from aerial imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201919). 7509\u20137518."},{"issue":"5","key":"e_1_3_1_34_2","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1111\/tgis.12186","article-title":"Automatic update of road attributes by mining GPS tracks","volume":"20","author":"Winden Karl Van","year":"2016","unstructured":"Karl Van Winden, Filip Biljecki, and Stefan Van der Spek. 2016. Automatic update of road attributes by mining GPS tracks. Trans. GIS 20, 5 (2016), 664\u2013683.","journal-title":"Trans. GIS"},{"issue":"1","key":"e_1_3_1_35_2","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1109\/MGRS.2020.2994107","article-title":"OpenStreetMap: Challenges and opportunities in machine learning and remote sensing","volume":"9","author":"Vargas-Munoz John E.","year":"2020","unstructured":"John E. Vargas-Munoz, Shivangi Srivastava, Devis Tuia, and Alexandre X. Falcao. 2020. OpenStreetMap: Challenges and opportunities in machine learning and remote sensing. IEEE Geosci. Remote Sens. Mag. 9, 1 (2020), 184\u2013199.","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.3390\/ijgi8030142"},{"issue":"4","key":"e_1_3_1_37_2","doi-asserted-by":"crossref","first-page":"1261","DOI":"10.3390\/s18041261","article-title":"A method for extracting road boundary information from Crowdsourcing vehicle GPS trajectories","volume":"18","author":"Yang Wei","year":"2018","unstructured":"Wei Yang, Tinghua Ai, and Wei Lu. 2018. A method for extracting road boundary information from Crowdsourcing vehicle GPS trajectories. Sensors 18, 4 (2018), 1261.","journal-title":"Sensors"},{"issue":"2","key":"e_1_3_1_38_2","article-title":"Feature-based map matching for low-sampling-rate GPS trajectories","volume":"4","author":"Yin Yifang","year":"2018","unstructured":"Yifang Yin, Rajiv Ratn Shah, Guanfeng Wang, and Roger Zimmermann. 2018. Feature-based map matching for low-sampling-rate GPS trajectories. ACM Trans. Spatial Algor. Syst. 4, 2 (2018).","journal-title":"ACM Trans. Spatial Algor. Syst."},{"key":"e_1_3_1_39_2","first-page":"7:1\u20137:10","volume-title":"Proceedings of the ACM SIGSPATIAL International Workshop on GeoStreaming","author":"Yin Yifang","year":"2016","unstructured":"Yifang Yin, Rajiv Ratn Shah, and Roger Zimmermann. 2016. A general feature-based map matching framework with trajectory simplification. In Proceedings of the ACM SIGSPATIAL International Workshop on GeoStreaming. 7:1\u20137:10."},{"key":"e_1_3_1_40_2","unstructured":"Yifang Yin An Tran Ying Zhang Wenmiao Hu Guanfeng Wang Jagannadan Varadarajan Roger Zimmermann and See-Kiong Ng. 2021. Multimodal fusion of satellite images and crowdsourced GPS traces for robust road attribute detection. In Proceedings of the 29th International SIGSPATIAL Conference on Advances in Geographic Information Systems . 107\u2013116."},{"key":"e_1_3_1_41_2","first-page":"2662","volume-title":"Proceedings of the World Wide Web Conference","author":"Yin Yifang","year":"2020","unstructured":"Yifang Yin, Jagannadan Varadarajan, Guanfeng Wang, Xueou Wang, Dhruva Sahrawat, Roger Zimmermann, and See-Kiong Ng. 2020. A multi-task learning framework for road attribute updating via joint analysis of map data and GPS traces. In Proceedings of the World Wide Web Conference. 2662\u20132668."},{"key":"e_1_3_1_42_2","article-title":"Learning to multitask","volume":"31","author":"Zhang Yu","year":"2018","unstructured":"Yu Zhang, Ying Wei, and Qiang Yang. 2018. Learning to multitask. Adv. Neural Info. Process. Syst. 31 (2018).","journal-title":"Adv. Neural Info. Process. Syst."},{"key":"e_1_3_1_43_2","first-page":"182","volume-title":"Proceedings of the Computer Vision and Pattern Recognition Workshops","author":"Zhou Lichen","year":"2018","unstructured":"Lichen Zhou, Chuang Zhang, and Ming Wu. 2018. D-LinkNet: LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction. In Proceedings of the Computer Vision and Pattern Recognition Workshops. 182\u2013186."},{"key":"e_1_3_1_44_2","first-page":"123","volume-title":"Proceedings of the IEEE Intelligent Vehicles Symposium","author":"Zinoune Cl\u00e9ment","year":"2012","unstructured":"Cl\u00e9ment Zinoune, Philippe Bonnifait, and Javier Iba\u00f1ez-Guzm\u00e1n. 2012. Detection of missing roundabouts in maps for driving assistance systems. In Proceedings of the IEEE Intelligent Vehicles Symposium. IEEE, 123\u2013128."}],"container-title":["ACM Transactions on Spatial Algorithms and Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3618108","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3618108","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:58Z","timestamp":1750178278000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3618108"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,20]]},"references-count":43,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12,31]]}},"alternative-id":["10.1145\/3618108"],"URL":"https:\/\/doi.org\/10.1145\/3618108","relation":{},"ISSN":["2374-0353","2374-0361"],"issn-type":[{"value":"2374-0353","type":"print"},{"value":"2374-0361","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,20]]},"assertion":[{"value":"2022-01-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-08-26","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}