{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:03:50Z","timestamp":1760241830370,"version":"build-2065373602"},"reference-count":59,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2018,9,17]],"date-time":"2018-09-17T00:00:00Z","timestamp":1537142400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Considering the radio-based indoor positioning system pertaining to signal degradation due to the environmental factors, and rising popularity of IP (Internet Protocol) cameras in cities, a novel fusion of inertial measurement units (IMUs) with external IP cameras to determine the positions of moving users in indoor environments is presented. This approach uses a fine-tuned Faster R-CNN (Region Convolutional Neural Network) to detect users in images captured by cameras, and acquires visual measurements including ranges and angles of users with respect to the cameras based on the proposed monocular vision relatively measuring (MVRM) method. The results are determined by integrating the positions predicted by each user\u2019s inertial measurement unit (IMU) and visual measurements using an EKF (Extended Kalman Filter). The results experimentally show that the ranging accuracy is affected by both the detected bounding box\u2019s by Faster R-CNN height errors and diverse measuring distances, however, the heading accuracy is solely interfered with bounding box\u2019s horizontal biases. The indoor obstacles including stationary obstacles and a pedestrian in our tests more significantly decrease the accuracy of ranging than that of heading, and the effect of a pedestrian on the heading errors is greater than stationary obstacles on that. We implemented a positioning test for a single user and an external camera in five indoor scenarios to evaluate the performance. The robust fused IMU\/MVRM solution significantly decreases the positioning errors and shows better performance in dense multipath scenarios compared with the pure MVRM solution and ultra-wideband (UWB) solution.<\/jats:p>","DOI":"10.3390\/s18093134","type":"journal-article","created":{"date-parts":[[2018,9,17]],"date-time":"2018-09-17T10:42:20Z","timestamp":1537180940000},"page":"3134","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Integrated IMU with Faster R-CNN Aided Visual Measurements from IP Cameras for Indoor Positioning"],"prefix":"10.3390","volume":"18","author":[{"given":"Lin","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Information and Communication, School of Electronics and Information, Northwestern Polytechnical University, Xi\u2019an 710129, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Taoyun","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Information and Communication, School of Electronics and Information, Northwestern Polytechnical University, Xi\u2019an 710129, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Baowang","family":"Lian","sequence":"additional","affiliation":[{"name":"Department of Information and Communication, School of Electronics and Information, Northwestern Polytechnical University, Xi\u2019an 710129, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,9,17]]},"reference":[{"key":"ref_1","first-page":"1275","article-title":"An indoor AC magnetic positioning system","volume":"64","author":"Pasku","year":"2015","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Shi, G.W., and Ming, Y. (2016). Survey of indoor positioning systems based on ultra-wideband (UWB) technology. Wireless Communications, Networking and Applications, Springer.","DOI":"10.1007\/978-81-322-2580-5_115"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2630413","DOI":"10.1155\/2017\/2630413","article-title":"Evolution of indoor positioning technologies: A survey","volume":"2017","author":"Brena","year":"2017","journal-title":"J. Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"10092","DOI":"10.1109\/ACCESS.2018.2798918","article-title":"A multimodal fingerprint-based indoor positioning system for airports","volume":"6","author":"Molina","year":"2018","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1786","DOI":"10.1109\/TASE.2017.2692961","article-title":"Process mining to discover shoppers\u2019 pathways at a fashion retail store using a WiFi-base indoor positioning system","volume":"14","author":"Hwang","year":"2017","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Mashuk, M.S., Pinchin, J., Siebers, P.O., and Moore, T. (2018, January 23\u201326). A smart phone based multi-floor indoor positioning system for occupancy detection. Proceedings of the 2018 IEEE\/ION Position, Location and Navigation Symposium (PLANS), Monterey, CA, USA.","DOI":"10.1109\/PLANS.2018.8373384"},{"key":"ref_7","unstructured":"Van der Ham, M.F.S., Zlatanova, S., Verbree, E., and Voute, R. (2016, January 7\u20139). Real time localization of assets in hospitals using quuppa indoor positioning technology. Proceedings of the First International Conference on Smart Data and Smart Cities (30th UDMS), Split, Croatia."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1982","DOI":"10.1109\/TMC.2015.2451641","article-title":"Evaluation of two WiFi positioning systems based on autonomous crowdsourcing of handheld devices for indoor navigation","volume":"15","author":"Zhuang","year":"2016","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_9","unstructured":"Faragher, R., and Harle, R. (2014, January 8\u201312). An analysis of the accuracy of Bluetooth low energy for indoor positioning applications. Proceedings of the 27th International Technical Meeting of the Satellite Division of the Institute of Navigation (ION GNSS 2014), Tampa, FL, USA."},{"key":"ref_10","unstructured":"Lin, X.Y., Ho, T.W., Fang, C.C., Yen, Z.S., Yang, B.D., and Lai, F.P. (2015, January 25\u201329). A mobile indoor positioning system based on iBeacon technology. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"728","DOI":"10.1109\/TIM.2014.2347691","article-title":"Real-time RFID indoor positioning system based on Kalman-filter drift removal and Heron-bilateration location estimation","volume":"64","author":"Huang","year":"2015","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/j.neucom.2015.05.120","article-title":"RFID-enabled indoor positioning method for a real-time manufacturing execution system using OS-ELM","volume":"174","author":"Yang","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yang, D., Xu, B., Rao, K.Y., and Sheng, W.H. (2018). Passive infrared (PIR)-based indoor position tracking for smart homes using accessibility maps and a-star algorithm. Sensors, 18.","DOI":"10.3390\/s18020332"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Wu, C.X., Mu, Q., Zhang, Z.B., Jin, Y.F., Wang, Z.Y., and Shi, G.Y. (2016, January 19\u201322). Indoor positioning system based on inertial mems sensors: Design and realization. Proceedings of the 2016 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), Chengdu, China.","DOI":"10.1109\/CYBER.2016.7574852"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"30319","DOI":"10.3390\/s151229799","article-title":"Towards a decentralized magnetic indoor positioning system","volume":"15","author":"Kasmi","year":"2015","journal-title":"Sensors"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Alarifi, A., Al-Salman, A., Alsaleh, M., Alnafessah, A., Al-Hadhrami, S., Al-Ammar, M.A., and Al-Khalifa, H.S. (2016). Ultra-wideband indoor positioning technologies: Analysis and recent advances. Sensors, 16.","DOI":"10.3390\/s16050707"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4467","DOI":"10.1007\/s11277-017-4734-x","article-title":"Precise indoor positioning using UWB: A review of methods, algorithms and implementations","volume":"97","author":"Mazhar","year":"2017","journal-title":"Wirel. Pers. Commun."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1983","DOI":"10.1109\/TMC.2011.216","article-title":"Received-signal-strength-based indoor positioning using compressive sensing","volume":"11","author":"Feng","year":"2012","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Janicka, J., and Rapinski, J. (2016, January 2\u20134). Application of RSSI based navigation in indoor positioning. Proceedings of the 2016 Baltic Geodetic Congress (Bgc Geomatics), Gdansk, Poland.","DOI":"10.1109\/BGC.Geomatics.2016.17"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Xia, S.X., Liu, Y., Yuan, G., Zhu, M.J., and Wang, Z.H. (2017). Indoor fingerprint positioning based on Wi-Fi: An overview. ISPRS Int. J. Geo-Inf., 6.","DOI":"10.3390\/ijgi6050135"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5311","DOI":"10.3390\/s150305311","article-title":"Fast fingerprint database maintenance for indoor positioning based on UGV SLAM","volume":"15","author":"Tang","year":"2015","journal-title":"Sensors"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Alsudani, A. (2018). NLOS mitigation and ranging accuracy for building indoor positioning system in UWB using commercial radio modules. AIP Conf. Proc., 1968.","DOI":"10.1063\/1.5039243"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1281","DOI":"10.1109\/SURV.2012.121912.00075","article-title":"A survey of indoor inertial positioning systems for pedestrians","volume":"15","author":"Harle","year":"2013","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Correa, A., Barcelo, M., Morell, A., and Vicario, J.L. (2017). A review of pedestrian indoor positioning systems for mass market applications. Sensors, 17.","DOI":"10.3390\/s17081927"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"24862","DOI":"10.3390\/s151024862","article-title":"A Bluetooth\/PDR integration algorithm for an indoor positioning system","volume":"15","author":"Li","year":"2015","journal-title":"Sensors"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"347","DOI":"10.3390\/mi6030347","article-title":"Smartphone-based indoor integrated WiFi\/MEMS positioning algorithm in a multi-floor environment","volume":"6","author":"Tian","year":"2015","journal-title":"Micromachines"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1293","DOI":"10.1109\/TVT.2015.2396640","article-title":"Indoor positioning using ultra-wideband and inertial measurements","volume":"64","author":"Kok","year":"2015","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Anup, S., Goel, A., and Padmanabhan, S. (2017, January 5\u20138). Visual positioning system for automated indoor\/outdoor navigation. Proceedings of the TENCON 2017\u20142017 IEEE Region 10 Conference, Penang, Malaysia.","DOI":"10.1109\/TENCON.2017.8228008"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2518","DOI":"10.1109\/TIM.2015.2415031","article-title":"Accurate 3-D position and orientation method for indoor mobile robot navigation based on photoelectric scanning","volume":"64","author":"Huang","year":"2015","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Endo, Y., Sato, K., Yamashita, A., and Matsubayashi, K. (2017, January 15\u201317). Indoor positioning and obstacle detection for visually impaired navigation system based on LSD-SLAM. Proceedings of the 2017 International Conference on Biometrics and Kansei Engineering (ICBAKE), Kyoto, Japan.","DOI":"10.1109\/ICBAKE.2017.8090635"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Saputra, M.R.U., Markham, A., and Trigoni, N. (2018). Visual SLAM and structure from motion in dynamic environments: A survey. ACM Comput. Surv., 51.","DOI":"10.1145\/3177853"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1289","DOI":"10.1080\/01691864.2015.1057616","article-title":"A review of visual inertial odometry from filtering and optimisation perspectives","volume":"29","author":"Gui","year":"2015","journal-title":"Adv. Robot."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Schmid, K., and Hirschmuller, H. (2013, January 6\u201310). Stereo vision and IMU based real-time Ego-motion and depth image computation on a handheld device. Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6631242"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"690","DOI":"10.1177\/0278364913481251","article-title":"High-precision, consistent EKF-based visual-inertial odometry","volume":"32","author":"Li","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1177\/0278364914554813","article-title":"Keyframe-based visual-inertial odometry using nonlinear optimization","volume":"34","author":"Leutenegger","year":"2015","journal-title":"Int. J. Robot. Res."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TRO.2018.2853729","article-title":"Vins-mono: A robust and versatile monocular visual-inertial state estimator","volume":"34","author":"Qin","year":"2018","journal-title":"IEEE Trans. Robot."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Antigny, N., Servieres, M., and Renaudin, V. (2016, January 4\u20137). Hybrid visual and inertial position and orientation estimation based on known urban 3D models. Proceedings of the 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Alcala de Henares, Spain.","DOI":"10.1109\/IPIN.2016.7743619"},{"key":"ref_38","unstructured":"Bertozzi, M., Broggi, A., Fascioli, A., Tibaldi, A., Chapuis, R., and Chausse, F. (2004, January 14\u201317). Pedestrian localization and tracking system with Kalman filtering. Proceedings of the 2004 IEEE Intelligent Vehicles Symposium, Parma, Italy."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"864","DOI":"10.1109\/TIM.2013.2283546","article-title":"Flexible indoor localization and tracking based on a wearable platform and sensor data fusion","volume":"63","author":"Colombo","year":"2014","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.neucom.2017.07.069","article-title":"Method of predicting human mobility patterns using deep learning","volume":"280","author":"Kim","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"874","DOI":"10.1109\/TMM.2016.2636750","article-title":"Fusion of magnetic and visual sensors for indoor localization: Infrastructure-free and more effective","volume":"19","author":"Liu","year":"2017","journal-title":"IEEE Trans. Multimed."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Jiao, J.C., Li, F., Deng, Z.L., and Ma, W.J. (2017). A smartphone camera-based indoor positioning algorithm of crowded scenarios with the assistance of deep CNN. Sensors, 17.","DOI":"10.3390\/s17040704"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.neucom.2018.01.092","article-title":"Computer vision and deep learning techniques for pedestrian detection and tracking: A survey","volume":"300","author":"Brunetti","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Chen, X.G., Wei, P.X., Ke, W., Ye, Q.X., and Jiao, J.B. (2015). Pedestrian detection with deep convolutional neural network. Computer Vision\u2014ACCV 2014 Workshops, Pt I, Springer.","DOI":"10.1007\/978-3-319-16628-5_26"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Park, E., del Pobil, A.P., and Kwon, S.J. (2018). The role of internet of things (IoT) in smart cities: Technology roadmap-oriented approaches. Sustainability, 10.","DOI":"10.3390\/su10051388"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Leong, C.Y., Perumal, T., Yaakob, R., and Peng, K.W. (2017, January 14\u201315). Enhancing indoor positioning service for location based internet of things (IoT) a source selecting approach with error compensation. Proceedings of the 2017 IEEE International Symposium on Consumer Electronics (ISCE), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ISCE.2017.8355546"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Fathy, Y., Barnaghi, P., and Tafazolli, R. (2018). Large-scale indexing, discovery, and ranking for the internet of things (IoT). ACM Comput. Surv., 51.","DOI":"10.1145\/3154525"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.pmcj.2014.09.003","article-title":"Accurate smartphone indoor positioning using a WSN infrastructure and non-invasive audio for TDOA estimation","volume":"20","author":"Lopes","year":"2015","journal-title":"Pervasive Mob. Comput."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"S295","DOI":"10.3233\/THC-171333","article-title":"Differential barometric-based positioning technique for indoor elevation measurement in IoT medical applications","volume":"25","author":"Wang","year":"2017","journal-title":"Technol. Health Care"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1007\/s11276-016-1309-9","article-title":"Sala: Smartphone-assisted localization algorithm for positioning indoor IoT devices","volume":"24","author":"Jeong","year":"2018","journal-title":"Wirel. Netw."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"De Angelis, G., De Angelis, A., Pasku, V., Moschitta, A., and Carbone, P. (2015, January 28\u201330). A hybrid outdoor\/indoor positioning system for IoT applications. Proceedings of the 2015 IEEE International Symposium on Systems Engineering (ISSE) Proceedings, Rome, Italy.","DOI":"10.1109\/SysEng.2015.7302503"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Liu, R., Yuen, C., Do, T.-N., Jiao, D., Liu, X., and Tan, U.X. (June, January 29). Cooperative relative positioning of mobile users by fusing IMU inertial and UWB ranging information. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA 2017), Singapore.","DOI":"10.1109\/ICRA.2017.7989660"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1007\/978-3-319-10590-1_53","article-title":"Visualizing and understanding convolutional networks","volume":"Volume 8689","author":"Fleet","year":"2014","journal-title":"Computer Vision\u2014ECCV 2014, Pt I"},{"key":"ref_57","unstructured":"Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., and Zisserman, A. (2018, September 16). The Pascal Visual Object Classes Challenge 2007 (VOC 2007) Results. Available online: http:\/\/host.robots.ox.ac.uk\/pascal\/VOC\/voc2007\/results\/index.shtml."},{"key":"ref_58","unstructured":"(2018, July 14). The Pascal Visual Object Classes Challenge 2012 (VOC 2012) Development Kit. Available online: http:\/\/host.robots.ox.ac.uk\/pascal\/VOC\/voc2012\/htmldoc\/devkit_doc.html#SECTION00044000000000000000."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A flexible new technique for camera calibration","volume":"22","author":"Zhang","year":"2000","journal-title":"IEEE Trans. Pattern Anal."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/3134\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:20:59Z","timestamp":1760196059000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/3134"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,17]]},"references-count":59,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2018,9]]}},"alternative-id":["s18093134"],"URL":"https:\/\/doi.org\/10.3390\/s18093134","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2018,9,17]]}}}