{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T16:58:04Z","timestamp":1782925084531,"version":"3.54.5"},"reference-count":118,"publisher":"SAGE Publications","issue":"9","license":[{"start":{"date-parts":[[2020,7,7]],"date-time":"2020-07-07T00:00:00Z","timestamp":1594080000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/100006785","name":"Google","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006785","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2020,8]]},"abstract":"<jats:p>The overarching goals in image-based localization are scale, robustness, and speed. In recent years, approaches based on local features and sparse 3D point-cloud models have both dominated the benchmarks and seen successful real-world deployment. They enable applications ranging from robot navigation, autonomous driving, virtual and augmented reality to device geo-localization. Recently, end-to-end learned localization approaches have been proposed which show promising results on small-scale datasets. However, the positioning accuracy, scalability, latency, and compute and storage requirements of these approaches remain open challenges. We aim to deploy localization at a global scale where one thus relies on methods using local features and sparse 3D models. Our approach spans from offline model building to real-time client-side pose fusion. The system compresses the appearance and geometry of the scene for efficient model storage and lookup leading to scalability beyond what has been demonstrated previously. It allows for low-latency localization queries and efficient fusion to be run in real-time on mobile platforms by combining server-side localization with real-time visual\u2013inertial-based camera pose tracking. In order to further improve efficiency, we leverage a combination of priors, nearest-neighbor search, geometric match culling, and a cascaded pose candidate refinement step. This combination outperforms previous approaches when working with large-scale models and allows deployment at unprecedented scale. We demonstrate the effectiveness of our approach on a proof-of-concept system localizing 2.5 million images against models from four cities in different regions of the world achieving query latencies in the 200 ms range.<\/jats:p>","DOI":"10.1177\/0278364920931151","type":"journal-article","created":{"date-parts":[[2020,7,7]],"date-time":"2020-07-07T06:18:14Z","timestamp":1594102694000},"page":"1061-1084","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":79,"title":["Large-scale, real-time visual\u2013inertial localization revisited"],"prefix":"10.1177","volume":"39","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6421-541X","authenticated-orcid":false,"given":"Simon","family":"Lynen","sequence":"first","affiliation":[{"name":"Google Switzerland, Zurich, Switzerland"},{"name":"Autonomous Systems Lab, ETH Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bernhard","family":"Zeisl","sequence":"additional","affiliation":[{"name":"Google Switzerland, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dror","family":"Aiger","sequence":"additional","affiliation":[{"name":"Google Israel, Tel Aviv, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Bosse","sequence":"additional","affiliation":[{"name":"Computer Vision and Geometry Group, Department of Computer Science,ETH Zurich, Switzerland"},{"name":"Microsoft Research, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joel","family":"Hesch","sequence":"additional","affiliation":[{"name":"Computer Vision and Geometry Group, Department of Computer Science,ETH Zurich, Switzerland"},{"name":"Microsoft Research, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marc","family":"Pollefeys","sequence":"additional","affiliation":[{"name":"Computer Vision and Geometry Group, Department of Computer Science,ETH Zurich, Switzerland"},{"name":"Microsoft Research, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roland","family":"Siegwart","sequence":"additional","affiliation":[{"name":"Autonomous Systems Lab, ETH Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9760-4553","authenticated-orcid":false,"given":"Torsten","family":"Sattler","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Chalmers University of Technology,Gothenburg, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2020,7,7]]},"reference":[{"key":"bibr1-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.431"},{"key":"bibr2-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2012.6247715"},{"key":"bibr3-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5979650"},{"key":"bibr4-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.572"},{"key":"bibr5-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2009.5336494"},{"key":"bibr6-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2012.6248038"},{"key":"bibr7-0278364920931151","first-page":"3","volume":"1","author":"Balntas V","year":"2016","journal-title":"BMVC"},{"key":"bibr8-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/11744023_32"},{"key":"bibr9-0278364920931151","author":"Berrio JS","year":"2018","journal-title":"arXiv preprint arXiv:1809.09774"},{"key":"bibr10-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2009.07.009"},{"key":"bibr11-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.267"},{"key":"bibr12-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00489"},{"key":"bibr13-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/IVS.2013.6629525"},{"key":"bibr14-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.54"},{"key":"bibr15-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15561-1_56"},{"key":"bibr16-0278364920931151","author":"Camposeco F","year":"2018","journal-title":"arXiv preprint arXiv:1807.07512"},{"key":"bibr17-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.709"},{"key":"bibr18-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.96"},{"key":"bibr19-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.66"},{"key":"bibr20-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2007.364109"},{"key":"bibr21-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.31"},{"key":"bibr22-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33715-4_10"},{"key":"bibr23-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1177\/0278364910385483"},{"key":"bibr24-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5980267"},{"key":"bibr25-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.73"},{"key":"bibr26-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989741"},{"key":"bibr27-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2015.7353722"},{"key":"bibr28-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139575"},{"key":"bibr29-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1145\/358669.358692"},{"key":"bibr30-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2012.2197158"},{"key":"bibr31-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.240"},{"key":"bibr32-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989362"},{"key":"bibr33-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487238"},{"key":"bibr34-0278364920931151","first-page":"14","volume":"68","author":"H\u00e4ne C","year":"2017","journal-title":"IMAVIS"},{"key":"bibr35-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/BF02028352"},{"key":"bibr36-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ECMR.2013.6698820"},{"key":"bibr37-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2013.93"},{"key":"bibr38-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913509675"},{"key":"bibr39-0278364920931151","volume-title":"Proceedings of the International Conference on Computer Vision (ICCV)","author":"Hesch JA","year":"2011"},{"key":"bibr40-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2012.6239190"},{"key":"bibr41-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206587"},{"key":"bibr42-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.57"},{"key":"bibr43-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2013.6630556"},{"key":"bibr44-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2013.6631024"},{"key":"bibr45-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1177\/0278364910388963"},{"key":"bibr46-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4684-2001-2_9"},{"key":"bibr47-0278364920931151","author":"Kasyanov A","year":"2017","journal-title":"arXiv preprint arXiv:1702.02175"},{"key":"bibr48-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.694"},{"key":"bibr49-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.336"},{"key":"bibr50-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2007.4538852"},{"key":"bibr51-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.122"},{"key":"bibr52-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995464"},{"key":"bibr53-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995339"},{"key":"bibr54-0278364920931151","doi-asserted-by":"publisher","DOI":"10.5244\/C.30.45"},{"key":"bibr55-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2006.188"},{"key":"bibr56-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1177\/0278364914554813"},{"key":"bibr57-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33718-5_2"},{"key":"bibr58-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15552-9_57"},{"key":"bibr59-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1177\/0278364914561101"},{"key":"bibr60-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.260"},{"key":"bibr61-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989359"},{"key":"bibr62-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1023\/B:VISI.0000029664.99615.94"},{"key":"bibr63-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-016-0947-9"},{"key":"bibr64-0278364920931151","author":"Lynen S","year":"2015","journal-title":"Proceedings of Robotics: Science and Systems (RSS)"},{"key":"bibr65-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2012.6386186"},{"key":"bibr66-0278364920931151","author":"Maddern WP","year":"2012","journal-title":"Robotics: Science and Systems"},{"key":"bibr67-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2004.02.009"},{"key":"bibr68-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2014.25"},{"key":"bibr69-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10605-2_18"},{"key":"bibr70-0278364920931151","author":"Mishchuk A","year":"2017","journal-title":"Neural Information Processing Systems (NIPS)"},{"key":"bibr71-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2009.2012342"},{"key":"bibr72-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2009.2012342"},{"key":"bibr73-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21595"},{"key":"bibr74-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2017.2705103"},{"key":"bibr75-0278364920931151","author":"Naseer T","year":"2014","journal-title":"Proceedings of AAAI"},{"key":"bibr76-0278364920931151","volume-title":"Proceedings of the Workshop on \u201cMulti-View Geometry in Robotics\u201d at the Robotics: Science and Systems (RSS)","author":"Nerurkar ED","year":"2013"},{"key":"bibr77-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.374"},{"key":"bibr78-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2013.41"},{"key":"bibr79-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2017.09.013"},{"key":"bibr80-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46448-0_1"},{"key":"bibr81-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/11744023_34"},{"key":"bibr82-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"bibr83-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.243"},{"key":"bibr84-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.175"},{"key":"bibr85-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2611662"},{"key":"bibr86-0278364920931151","volume":"1","author":"Sattler T","year":"2018","journal-title":"Proceedings of CVPR"},{"key":"bibr87-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.654"},{"key":"bibr88-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2018.2800113"},{"key":"bibr89-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.445"},{"key":"bibr90-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.736"},{"key":"bibr91-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00721"},{"key":"bibr92-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.377"},{"key":"bibr93-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1002\/rob.20360"},{"key":"bibr94-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2003.1238663"},{"key":"bibr95-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2013.6696952"},{"key":"bibr96-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.598"},{"key":"bibr97-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2598331"},{"key":"bibr98-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.75"},{"key":"bibr99-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2015.20"},{"key":"bibr100-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00752"},{"key":"bibr101-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298790"},{"key":"bibr102-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.119"},{"key":"bibr103-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2881829"},{"key":"bibr104-0278364920931151","unstructured":"Trawny N, Roumeliotis SI (2005) Indirect Kalman filter for 3D attitude estimation. Technical Report, University of Minnesota, Department of Computer Science and Engineering."},{"key":"bibr105-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139924"},{"key":"bibr106-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8462979"},{"key":"bibr107-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299069"},{"key":"bibr108-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2018.00114"},{"key":"bibr109-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.27"},{"key":"bibr110-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.75"},{"key":"bibr111-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_3"},{"key":"bibr112-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2007.4409115"},{"key":"bibr113-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46466-4_28"},{"key":"bibr114-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5539801"},{"key":"bibr115-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15561-1_19"},{"key":"bibr116-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.310"},{"key":"bibr117-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/3DPVT.2006.80"},{"key":"bibr118-0278364920931151","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206749"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364920931151","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0278364920931151","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364920931151","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:16:32Z","timestamp":1777457792000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0278364920931151"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,7]]},"references-count":118,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2020,8]]}},"alternative-id":["10.1177\/0278364920931151"],"URL":"https:\/\/doi.org\/10.1177\/0278364920931151","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,7]]}}}