{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:25:59Z","timestamp":1760243159328,"version":"build-2065373602"},"reference-count":45,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2015,12,10]],"date-time":"2015-12-10T00:00:00Z","timestamp":1449705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Wide-area registration in outdoor environments on mobile phones is a challenging task in mobile augmented reality fields. We present a sensor-aware large-scale outdoor augmented reality system for recognition and tracking on mobile phones. GPS and gravity information is used to improve the VLAD performance for recognition. A kind of sensor-aware VLAD algorithm, which is self-adaptive to different scale scenes, is utilized to recognize complex scenes. Considering vision-based registration algorithms are too fragile and tend to drift, data coming from inertial sensors and vision are fused together by an extended Kalman filter (EKF) to achieve considerable improvements in tracking stability and robustness. Experimental results show that our method greatly enhances the recognition rate and eliminates the tracking jitters.<\/jats:p>","DOI":"10.3390\/s151229847","type":"journal-article","created":{"date-parts":[[2015,12,14]],"date-time":"2015-12-14T02:57:29Z","timestamp":1450061849000},"page":"31092-31107","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Sensor-Aware Recognition and Tracking for Wide-Area Augmented Reality on Mobile Phones"],"prefix":"10.3390","volume":"15","author":[{"given":"Jing","family":"Chen","sequence":"first","affiliation":[{"name":"School of Optoelectronics, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ruochen","family":"Cao","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yongtian","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Optoelectronics, Beijing Institute of Technology, Beijing 100081, China"},{"name":"School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2015,12,10]]},"reference":[{"key":"ref_1","unstructured":"Gao, X., Tian, J., Liang, X., and Wang, G. (2014, January 9\u201310). ARPP: An Augmented Reality 3D ping-pong game system on Android mobile platform. Proceedings of the 2014 23rd IEEE on Wireless and Optical Communication Conference (WOCC), Newark, NJ, USA."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Choi, H., Han, G.C., and Kim, I.J. (2014, January 10\u201313). Smart Booklet: Tour guide system with mobile augmented reality. Proceedings of the 2014 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA.","DOI":"10.1109\/ICCE.2014.6776038"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1016\/j.procs.2015.07.566","article-title":"Using Augmented Reality to Enhance Aetherpet, a Prototype of a Social Game","volume":"59","author":"Kwik","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.pmcj.2014.08.009","article-title":"Tourists responses to mobile augmented reality travel guides: The role of emotions on adoption behavior","volume":"18","author":"Kourouthanassis","year":"2015","journal-title":"Perv. Mob. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2174","DOI":"10.1016\/j.eswa.2013.09.016","article-title":"Mobile augmented reality based context-aware library management system","volume":"41","author":"Shatte","year":"2014","journal-title":"Expert Syst. Appl."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Pirchheim, C., Schmalstieg, D., and Reitmayr, G. (2013, January 1\u20134). Handling pure camera rotation in keyframe-based slam. Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR\u201913), Adelaide, Australia.","DOI":"10.1109\/ISMAR.2013.6671783"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1109\/TVCG.2014.27","article-title":"Global localization from monocular SLAM on a mobile phone","volume":"20","author":"Ventura","year":"2014","journal-title":"IEEE Trans. Visual. Comput. Graph."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Klein, G., and Murray, D. (2007, January 13\u201316). Parallel Tracking and Mapping for Small AR Workspaces. Proceedings of the 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR\u201907), Nara, Japan.","DOI":"10.1109\/ISMAR.2007.4538852"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1109\/TVCG.2009.99","article-title":"Real-time detection and tracking for augmented reality on mobile phones","volume":"16","author":"Wagner","year":"2010","journal-title":"IEEE Trans. Visual. Comput. Graph."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wagner, D., Mulloni, A., Langlotz, T., and Schmalstieg, D. (2010, January 20\u201324). Real-time panoramic mapping and tracking on mobile phones. Proceedings of the Virtual Reality Conference (VR), Waltham, MA, USA.","DOI":"10.1109\/VR.2010.5444786"},{"key":"ref_11","unstructured":"Yeh, T., Tollmar, K., and Darrell, T. (July, January 27). Searching the web with mobile images for location recognition. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), Washington, DC, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1007\/s11265-014-0946-4","article-title":"A Hybrid Bags-of-Feature model for Sports Scene Classification","volume":"81","author":"Park","year":"2015","journal-title":"J. Signal Process. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"5263","DOI":"10.1007\/s11042-014-2000-8","article-title":"Foodcam: A real-time food recognition system on a smartphone","volume":"74","author":"Kawano","year":"2015","journal-title":"Multimed. Tools Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.cviu.2015.03.006","article-title":"Learning representative and discriminative image representation by deep appearance and spatial coding","volume":"136","author":"Liu","year":"2015","journal-title":"Comput. Vision Image Underst."},{"key":"ref_15","unstructured":"Chen, D.M., Tsai, S.S., Chandrasekhar, V., Takacs, G., Singh, J., and Girod, B. (April, January 30). Tree histogram coding for mobile image matching. Proceedings of the Data Compression Conference, Snowbird, UT, USA."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Chen, D.M., Tsai, S.S., Chandrasekhar, V., Takacs, G., Vedantham, R., Grzeszczuk, R., and Girod, B. (2010, January 24\u201326). Inverted Index Compression for Scalable Image Matching. Proceedings of the Data Compression Conference, Snowbird, UT, USA.","DOI":"10.1109\/DCC.2010.53"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Perronnin, F., Liu, Y., S\u00e1nchez, J., and Poirier, H. (2010, January 13\u201318). Large-scale image retrieval with compressed fisher vectors. Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5540009"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1704","DOI":"10.1109\/TPAMI.2011.235","article-title":"Aggregating local image descriptors into compact codes","volume":"34","author":"Perronnin","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2316","DOI":"10.1016\/j.sigpro.2012.06.005","article-title":"Residual enhanced visual vector as a compact signature for mobile visual search","volume":"93","author":"Chen","year":"2013","journal-title":"Signal Process."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_21","unstructured":"Ke, Y., and Sukthankar, R. (July, January 27). PCA-SIFT: A more distinctive representation for local image descriptors. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cviu.2007.09.014","article-title":"Speeded-up robust features (SURF)","volume":"110","author":"Bay","year":"2008","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1688","DOI":"10.1109\/TMM.2013.2265674","article-title":"On-device mobile visual location recognition by integrating vision and inertial sensors","volume":"15","author":"Guan","year":"2013","journal-title":"IEEE Trans. Multimed."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhao, W.L., J\u00e9gou, H., and Gravier, G. (2013, January 11\u201314). Oriented pooling for dense and non-dense rotation-invariant features. Proceedings of the British Machine Vision Conference, Bristol, UK.","DOI":"10.5244\/C.27.99"},{"key":"ref_25","unstructured":"Wang, Z., Di, W., Bhardwaj, A., Jagadeesh, V., and Piramuthu, R. (2014). Geometric VLAD for large scale image search."},{"key":"ref_26","unstructured":"Layar. Available online: https:\/\/www.layar.com\/."},{"key":"ref_27","unstructured":"Junaio. Available online: http:\/\/www.junaio.com\/."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1052","DOI":"10.1109\/TPAMI.2007.1049","article-title":"MonoSLAM: Real-time single camera SLAM","volume":"29","author":"Davison","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Newcombe, R.A., Lovegrove, S.J., and Davison, A.J. (2011, January 6\u201313). DTAM: Dense tracking and mapping in real-time. Proceedings of the 2011 IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126513"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1109\/TVCG.2014.2355207","article-title":"Real-Time 3D Tracking and Reconstruction on Mobile Phones","volume":"21","author":"Prisacariu","year":"2015","journal-title":"IEEE Trans. Visual. Comput. Graph."},{"key":"ref_31","first-page":"388","article-title":"Wide area localization and tracking on camera phones for mobile augmented reality systems","volume":"21","author":"Wei","year":"2014","journal-title":"Multimed. Syst."},{"key":"ref_32","unstructured":"AR Ivaders. Available online: http:\/\/invaders.soulbit7.com\/."},{"key":"ref_33","unstructured":"AR Defender. Available online: https:\/\/itunes.apple.com\/cn\/app\/ar-defender-2\/id559729773?mt=8."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Reitmayr, G., and Drummond, T.W. (2006, January 22\u201325). Going out: Robust model-based tracking for outdoor augmented reality. Proceedings of the IEEE\/ACM International Symposium on Mixed and Augmented Reality (ISMAR 2006), Santa Barbara, CA, USA.","DOI":"10.1109\/ISMAR.2006.297801"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Oskiper, T., Samarasekera, S., and Kumar, R. (2012, January 5\u20138). Multi-sensor navigation algorithm using monocular camera, IMU and GPS for large scale augmented reality. Proceedings of the 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Atlanta, GA, USA.","DOI":"10.1109\/ISMAR.2012.6402541"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1109\/TIP.2014.2380176","article-title":"Fusing Inertial Sensor Data in an Extended Kalman Filter for 3D Camera Tracking","volume":"24","author":"Erdem","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hilsenbeck, S., Moller, A., Huitl, R., Schroth, G., Kranz, M., and Steinbach, E. (2012, January 13\u201315). Scale-preserving long-term visual odometry for indoor navigation. Proceedings of the 2012 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Sydney, Australia.","DOI":"10.1109\/IPIN.2012.6418934"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kundra, L., Ekler, P., and Charaf, H. (2013, January 2\u20135). Improving orientation estimation in mobiles with built-in camera. Proceedings of the 2013 IEEE 4th International Conference on Cognitive Infocommunications (CogInfoCom), Budapest, Hungary.","DOI":"10.1109\/CogInfoCom.2013.6719202"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.5391\/IJFIS.2013.13.1.1","article-title":"Landmark Initialization for Unscented Kalman Filter Sensor Fusion in Monocular Camera Localization","volume":"13","author":"Hartmann","year":"2013","journal-title":"Int. J. Fuzzy Logic Intell. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Li, M., Kim, B.H., and Mourikis, A. (2013, January 6\u201310). Real-time motion tracking on a cellphone using inertial sensing and a rolling-shutter camera. Proceedings of the 2013 IEEE International Conference on Robotics and Automation (ICRA), Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6631248"},{"key":"ref_41","unstructured":"Dong, Z., Zhang, G., Jia, J., and Bao, H. (2009, January 8\u201312). Keyframe-based real-time camera tracking. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"629","DOI":"10.1364\/JOSAA.4.000629","article-title":"Closed-form solution of absolute orientation using unit quaternions","volume":"4","author":"Horn","year":"1987","journal-title":"J. Opt. Soc. Am."},{"key":"ref_43","unstructured":"Zhang, Z. (1999, January 20\u201327). Flexible camera calibration by viewing a plane from unknown orientations. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece."},{"key":"ref_44","unstructured":"Bleser, G. (2009). Towards visual-inertial slam for mobile augmented reality. [Ph.D. Thesis, Technical University Kaiserslautern]."},{"key":"ref_45","unstructured":"Our database. Available online: http:\/\/pan.baidu.com\/s\/1nt1hv8t."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/15\/12\/29847\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:53:49Z","timestamp":1760216029000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/15\/12\/29847"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,12,10]]},"references-count":45,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2015,12]]}},"alternative-id":["s151229847"],"URL":"https:\/\/doi.org\/10.3390\/s151229847","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2015,12,10]]}}}