{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T09:51:23Z","timestamp":1762077083526,"version":"build-2065373602"},"reference-count":36,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2022,11,15]],"date-time":"2022-11-15T00:00:00Z","timestamp":1668470400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["61871054"],"award-info":[{"award-number":["61871054"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, a WiFi and visual fingerprint localization model based on low-rank fusion (LRF-WiVi) is proposed, which makes full use of the complementarity of heterogeneous signals by modeling both the signal-specific actions and interaction of location information in the two signals end-to-end. Firstly, two feature extraction subnetworks are designed to extract the feature vectors containing location information of WiFi channel state information (CSI) and multi-directional visual images respectively. Then, the low-rank fusion module efficiently aggregates the specific actions and interactions of the two feature vectors while maintaining low computational complexity. The fusion features obtained are used for position estimation; In addition, for the CSI feature extraction subnetwork, we designed a novel construction method of CSI time-frequency characteristic map and a double-branch CNN structure to extract features. LRF-WiVi jointly learns the parameters of each module under the guidance of the same loss function, making the whole model more consistent with the goal of fusion localization. Extensive experiments are conducted in a complex laboratory and an open hall to verify the superior performance of LRF-WiVi in utilizing WiFi and visual signal complementarity. The results show that our method achieves more advanced positioning performance than other methods in both scenarios.<\/jats:p>","DOI":"10.3390\/s22228821","type":"journal-article","created":{"date-parts":[[2022,11,16]],"date-time":"2022-11-16T04:39:03Z","timestamp":1668573543000},"page":"8821","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["LRF-WiVi: A WiFi and Visual Indoor Localization Method Based on Low-Rank Fusion"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6450-1969","authenticated-orcid":false,"given":"Wen","family":"Liu","sequence":"first","affiliation":[{"name":"School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Changyan","family":"Qin","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhongliang","family":"Deng","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haoyue","family":"Jiang","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,15]]},"reference":[{"key":"ref_1","unstructured":"Antsfeld, L., Chidlovskii, B., and Sansano-Sansano, E. (2020). Deep smartphone sensors-WiFi fusion for indoor positioning and tracking. arXiv."},{"key":"ref_2","first-page":"59","article-title":"A Review of Hybrid Indoor Positioning Systems Employing WLAN Fingerprinting and Image Processing","volume":"10","author":"Firdaus","year":"2019","journal-title":"Int. J. Electr. Comput. Eng. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Fukui, A., Park, D.H., Yang, D., Rohrbach, A., Darrell, T., and Rohrbach, M. (2016). Multimodal compact bilinear pooling for visual question answering and visual grounding. arXiv.","DOI":"10.18653\/v1\/D16-1044"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zadeh, A., Chen, M., Poria, S., Cambria, E., and Morency, L.P. (2017). Tensor fusion network for multimodal sentiment analysis. arXiv.","DOI":"10.18653\/v1\/D17-1115"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wang, Y., Zhou, G., Xiang, C., Zhang, S., and Xu, S. (2020, January 7\u201311). Joint visual and wireless signal feature based approach for high-precision indoor localization. Proceedings of the GLOBECOM 2020-2020 IEEE Global Communications Conference, Taipei, Taiwan.","DOI":"10.1109\/GLOBECOM42002.2020.9322074"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Almansoub, Y.A., Zhong, M., Hu, Z., Huang, G., Al-qaness, M.A., and Abbasi, A.A. (2020, January 24\u201325). Multi-scale Vehicle Localization in Underground Parking Lots by Integration of Dead Reckoning, Wi-Fi and Vision. Proceedings of the 2020 6th International Conference on Big Data Computing and Communications (BIGCOM), Deqing, China.","DOI":"10.1109\/BigCom51056.2020.00012"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Wang, Z., Sokliep, P., Xu, C., Huang, J., Lu, L., and Shi, Z. (2019, January 7\u20139). Indoor position algorithm based on the fusion of wifi and image. Proceedings of the 2019 Eleventh International Conference on Advanced Computational Intelligence (ICACI), Guilin, China.","DOI":"10.1109\/ICACI.2019.8778542"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"6748","DOI":"10.1109\/JIOT.2020.2974928","article-title":"WiFi and vision-integrated fingerprint for smartphone-based self-localization in public indoor scenes","volume":"7","author":"Huang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_9","first-page":"1109","article-title":"Image and wlan bimodal integration for indoor user localization","volume":"19","author":"Laoudias","year":"2019","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Levchev, P., Krishnan, M.N., Yu, C., Menke, J., and Zakhor, A. (2014, January 27\u201330). Simultaneous fingerprinting and mapping for multimodal image and WiFi indoor positioning. Proceedings of the 2014 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Busan, Korea.","DOI":"10.1109\/IPIN.2014.7275515"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Alahi, A., Haque, A., and Fei-Fei, L. (2015, January 7\u201313). RGB-W: When vision meets wireless. Proceedings of the Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.376"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1507","DOI":"10.1007\/s11277-018-5725-2","article-title":"Smart fusion of multi-sensor ubiquitous signals of mobile device for localization in GNSS-denied scenarios","volume":"116","author":"Jiao","year":"2021","journal-title":"Wirel. Pers. Commun."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhang, H., and Li, Y. (2021). Lightgbm indoor positioning method based on merged wi-fi and image fingerprints. Sensors, 21.","DOI":"10.3390\/s21113662"},{"key":"ref_14","unstructured":"Liu, W., Chen, H., Deng, Z., Qin, C., and Jia, M. (December, January 29). An Indoor Fusion Fingerprint Localization Based on Channel State Information and Images. Proceedings of the IPIN-WiP, Lloret de Mar, Spain."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1550147718757664","DOI":"10.1177\/1550147718757664","article-title":"A hybrid fusion of wireless signals and RGB image for indoor positioning","volume":"14","author":"Jiao","year":"2018","journal-title":"Int. J. Distrib. Sens. Netw."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liu, W., Cheng, Q., Deng, Z., Chen, H., Fu, X., Zheng, X., Zheng, S., Chen, C., and Wang, S. (October, January 30). Survey on CSI-based indoor positioning systems and recent advances. Proceedings of the 2019 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Pisa, Italy.","DOI":"10.1109\/IPIN.2019.8911774"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.comcom.2017.10.013","article-title":"Exploiting distribution of channel state information for accurate wireless indoor localization","volume":"114","author":"Xiao","year":"2017","journal-title":"Comput. Commun."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"18066","DOI":"10.1109\/ACCESS.2017.2749516","article-title":"ConFi: Convolutional neural networks based indoor Wi-Fi localization using channel state information","volume":"5","author":"Chen","year":"2017","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Cai, C., Deng, L., and Li, S. (2018, January 7\u201310). CSI-based device-free indoor localization using convolutional neural networks. Proceedings of the 2018 IEEE 4th International Conference on Computer and Communications (ICCC), Chengdu, China.","DOI":"10.1109\/CompComm.2018.8780855"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"106864","DOI":"10.1016\/j.comnet.2019.106864","article-title":"Convolutional neural network and dual-factor enhanced variational Bayes adaptive Kalman filter based indoor localization with Wi-Fi","volume":"162","author":"Zhao","year":"2019","journal-title":"Comput. Netw."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wang, X., Wang, X., and Mao, S. (2017, January 21\u201325). CiFi: Deep convolutional neural networks for indoor localization with 5 GHz Wi-Fi. Proceedings of the 2017 IEEE International Conference on Communications (ICC), Paris, France.","DOI":"10.1109\/ICC.2017.7997235"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"250","DOI":"10.23919\/JCC.2019.09.019","article-title":"Convolutional neural networks based indoor Wi-Fi localization with a novel kind of CSI images","volume":"16","author":"Li","year":"2019","journal-title":"China Commun."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"211","DOI":"10.5194\/isprsarchives-XL-4-W5-211-2015","article-title":"Image-Based Localization for Indoor Environment Using Mobile Phone","volume":"XL-4\/W5","author":"Huang","year":"2015","journal-title":"Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Xia, Y., Xiu, C., and Yang, D. (2018, January 22\u201323). Visual indoor positioning method using image database. Proceedings of the 2018 Ubiquitous Positioning, Indoor Navigation and Location-Based Services (UPINLBS), Wuhan, China.","DOI":"10.1109\/UPINLBS.2018.8559714"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, L., Xia, H., Liu, Q., Wei, C., Fu, D., and Qiao, Y. (2021). Visual Positioning in Indoor Environments Using RGB-D Images and Improved Vector of Local Aggregated Descriptors. ISPRS Int. J. -Geo-Inf., 10.","DOI":"10.3390\/ijgi10040195"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"59750","DOI":"10.1109\/ACCESS.2020.2981620","article-title":"Surfcnn: A descriptor accelerated convolutional neural network for image-based indoor localization","volume":"8","author":"Elmoogy","year":"2020","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wozniak, P., Afrisal, H., Esparza, R.G., and Kwolek, B. (2018, January 17\u201319). Scene recognition for indoor localization of mobile robots using deep CNN. Proceedings of the International Conference on Computer Vision and Graphics, Warsaw, Poland.","DOI":"10.1007\/978-3-030-00692-1_13"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1247","DOI":"10.1162\/089976600300015349","article-title":"Separating style and content with bilinear models","volume":"12","author":"Tenenbaum","year":"2000","journal-title":"Neural Comput."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., RoyChowdhury, A., and Maji, S. (2015, January 7\u201313). Bilinear CNN models for fine-grained visual recognition. Proceedings of the Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.170"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Hu, G., Hua, Y., Yuan, Y., Zhang, Z., Lu, Z., Mukherjee, S.S., Hospedales, T.M., Robertson, N.M., and Yang, Y. (2017, January 22\u201329). Attribute-enhanced face recognition with neural tensor fusion networks. Proceedings of the Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.404"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Liu, Z., Shen, Y., Lakshminarasimhan, V.B., Liang, P.P., Zadeh, A., and Morency, L.P. (2018). Efficient low-rank multimodal fusion with modality-specific factors. arXiv.","DOI":"10.18653\/v1\/P18-1209"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wu, C., Yang, Z., Zhou, Z., Qian, K., Liu, Y., and Liu, M. (May, January 26). PhaseU: Real-time LOS identification with WiFi. Proceedings of the 2015 IEEE Conference on Computer Communications (INFOCOM), Hong Kong, China.","DOI":"10.1109\/INFOCOM.2015.7218588"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Cheng, L., Li, Y., Zhang, M., and Wang, C. (2018, January 8\u201311). A fingerprint localization method based on weighted KNN algorithm. Proceedings of the 2018 IEEE 18th International Conference on Communication Technology (ICCT), Chongqing, China.","DOI":"10.1109\/ICCT.2018.8600210"},{"key":"ref_34","unstructured":"Bahl, P., and Padmanabhan, V.N. (2000, January 26\u201330). RADAR: An in-building RF-based user location and tracking system. Proceedings of the IEEE INFOCOM 2000. Conference on Computer communications, Nineteenth Annual joint Conference of the IEEE Computer and Communications Societies (Cat. No. 00CH37064), Tel Aviv, Israel."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1550147717749817","DOI":"10.1177\/1550147717749817","article-title":"Fingerprint-based Wi-Fi indoor localization using map and inertial sensors","volume":"13","author":"Wang","year":"2017","journal-title":"Int. J. Distrib. Sens. Netw."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13638-019-1371-y","article-title":"A novel CSI-based fingerprinting for localization with a single AP","volume":"2019","author":"Zhang","year":"2019","journal-title":"EURASIP J. Wirel. Commun. Netw."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/22\/8821\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:18:25Z","timestamp":1760145505000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/22\/8821"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,15]]},"references-count":36,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["s22228821"],"URL":"https:\/\/doi.org\/10.3390\/s22228821","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,11,15]]}}}