{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T16:44:37Z","timestamp":1771519477764,"version":"3.50.1"},"reference-count":48,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2023,6,17]],"date-time":"2023-06-17T00:00:00Z","timestamp":1686960000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ANRT (Agence Nationale Recherche Technologie; French Association for Research and Technology)"},{"name":"DATAHERTZ Company"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Indoor location-based services constitute an important part of our daily lives, providing position and direction information about people or objects in indoor spaces. These systems can be useful in security and monitoring applications that target specific areas such as rooms. Vision-based scene recognition is the task of accurately identifying a room category from a given image. Despite years of research in this field, scene recognition remains an open problem due to the different and complex places in the real world. Indoor environments are relatively complicated because of layout variability, object and decoration complexity, and multiscale and viewpoint changes. In this paper, we propose a room-level indoor localization system based on deep learning and built-in smartphone sensors combining visual information with smartphone magnetic heading. The user can be room-level localized while simply capturing an image with a smartphone. The presented indoor scene recognition system is based on direction-driven convolutional neural networks (CNNs) and therefore contains multiple CNNs, each tailored for a particular range of indoor orientations. We present particular weighted fusion strategies that improve system performance by properly combining the outputs from different CNN models. To meet users\u2019 needs and overcome smartphone limitations, we propose a hybrid computing strategy based on mobile computation offloading compatible with the proposed system architecture. The implementation of the scene recognition system is split between the user\u2019s smartphone and a server, which aids in meeting the computational requirements of CNNs. Several experimental analysis were conducted, including to assess performance and provide a stability analysis. The results obtained on a real dataset show the relevance of the proposed approach for localization, as well as the interest in model partitioning in hybrid mobile computation offloading. Our extensive evaluation demonstrates an increase in accuracy compared to traditional CNN scene recognition, indicating the effectiveness and robustness of our approach.<\/jats:p>","DOI":"10.3390\/s23125672","type":"journal-article","created":{"date-parts":[[2023,6,19]],"date-time":"2023-06-19T02:29:19Z","timestamp":1687141759000},"page":"5672","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Indoor Scene Recognition Mechanism Based on Direction-Driven Convolutional Neural Networks"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-8076-005X","authenticated-orcid":false,"given":"Andrea","family":"Daou","sequence":"first","affiliation":[{"name":"Univ Rouen Normandie, INSA Rouen Normandie, Universit\u00e9 Le Havre Normandie, Normandie Univ, LITIS UR 4108, F-76000 Rouen, France"},{"name":"Department of Research and Development, DATAHERTZ, 10000 Troyes, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jean-Baptiste","family":"Pothin","sequence":"additional","affiliation":[{"name":"Department of Research and Development, DATAHERTZ, 10000 Troyes, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3042-183X","authenticated-orcid":false,"given":"Paul","family":"Honeine","sequence":"additional","affiliation":[{"name":"Univ Rouen Normandie, INSA Rouen Normandie, Universit\u00e9 Le Havre Normandie, Normandie Univ, LITIS UR 4108, F-76000 Rouen, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Abdelaziz","family":"Bensrhair","sequence":"additional","affiliation":[{"name":"Univ Rouen Normandie, INSA Rouen Normandie, Universit\u00e9 Le Havre Normandie, Normandie Univ, LITIS UR 4108, F-76000 Rouen, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"109041","DOI":"10.1016\/j.comnet.2022.109041","article-title":"A Comprehensive Review of Indoor\/Outdoor Localization Solutions in IoT era: Research Challenges and Future Perspectives","volume":"212","author":"Asaad","year":"2022","journal-title":"Comput. Netw."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/98.878533","article-title":"GPS-less low-cost outdoor localization for very small devices","volume":"7","author":"Bulusu","year":"2000","journal-title":"IEEE Pers. Commun."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Low, R., Tekler, Z.D., and Cheah, L. (2021). An end-to-end point of interest (POI) conflation framework. ISPRS Int. J. Geo-Inf., 10.","DOI":"10.3390\/ijgi10110779"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1372","DOI":"10.1049\/iet-com.2019.1059","article-title":"Survey on WiFi-based indoor positioning techniques","volume":"14","author":"Liu","year":"2020","journal-title":"IET Commun."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"106681","DOI":"10.1016\/j.buildenv.2020.106681","article-title":"A scalable Bluetooth Low Energy approach to identify occupancy patterns and profiles in office spaces","volume":"171","author":"Tekler","year":"2020","journal-title":"Build. Environ."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Cheng, S., Wang, S., Guan, W., Xu, H., and Li, P. (2020). 3DLRA: An RFID 3D indoor localization method based on deep learning. Sensors, 20.","DOI":"10.3390\/s20092731"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"109689","DOI":"10.1016\/j.buildenv.2022.109689","article-title":"Occupancy prediction using deep learning approaches across multiple space types: A minimum sensing strategy","volume":"226","author":"Tekler","year":"2022","journal-title":"Build. Environ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1007\/s11263-006-8614-1","article-title":"Semantic modeling of natural scenes for content-based image retrieval","volume":"72","author":"Vogel","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Liu, S., and Tian, G. (2019). An indoor scene classification method for service robot Based on CNN feature. J. Robot., 2019.","DOI":"10.1155\/2019\/8591035"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1186\/s40537-019-0212-5","article-title":"Intelligent video surveillance: A review through deep learning techniques for crowd analysis","volume":"6","author":"Sreenu","year":"2019","journal-title":"J. Big Data"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ma, W., Xiong, H., Dai, X., Zheng, X., and Zhou, Y. (2018). An indoor scene recognition-based 3D registration mechanism for real-time AR-GIS visualization in mobile applications. ISPRS Int. J. Geo-Inf., 7.","DOI":"10.3390\/ijgi7030112"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Morar, A., Moldoveanu, A., Mocanu, I., Moldoveanu, F., Radoi, I.E., Asavei, V., Gradinaru, A., and Butean, A. (2020). A comprehensive survey of indoor localization methods based on computer vision. Sensors, 20.","DOI":"10.3390\/s20092641"},{"key":"ref_13","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20138). Imagenet classification with deep convolutional networks. Proceedings of the 25th International Conference on Neural Information Processing Systems, NIPS\u201912, Lake Tahoe, CA, USA."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1452","DOI":"10.1109\/TPAMI.2017.2723009","article-title":"Places: A 10 million image database for scene recognition","volume":"40","author":"Zhou","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1016\/j.neucom.2016.11.023","article-title":"G-MS2F: GoogLeNet based multi-stage feature fusion of deep CNN for scene recognition","volume":"225","author":"Tang","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_16","unstructured":"Zeng, D., Liao, M., Tavakolian, M., Guo, Y., Zhou, B., Hu, D., Pietik\u00e4inen, M., and Liu, L. (2021). Deep Learning for Scene Classification: A Survey. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"AlShamaa, D., Chehade, F., Honeine, P., and Chkeir, A. (2020). An Evidential Framework for Localization of Sensors in Indoor Environments. Sensors, 20.","DOI":"10.3390\/s20010318"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1109\/JSEN.2017.2766630","article-title":"Tracking of Mobile Sensors Using Belief Functions in Indoor Wireless Networks","volume":"18","author":"AlShamaa","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Guo, W., Wu, R., Chen, Y., and Zhu, X. (2018). Deep learning scene recognition method based on localization enhancement. Sensors, 18.","DOI":"10.3390\/s18103376"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1655","DOI":"10.1109\/JPROC.2019.2921977","article-title":"Deep learning with edge computing: A review","volume":"107","author":"Chen","year":"2019","journal-title":"Proc. IEEE"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1224","DOI":"10.1109\/TPAMI.2017.2709749","article-title":"SIFT meets CNN: A decade survey of instance retrieval","volume":"40","author":"Zheng","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_23","unstructured":"Hussain, M., Bird, J.J., and Faria, D.R. (2019). Advances in Computational Intelligence Systems: Contributions Presented at the 18th UK Workshop on Computational Intelligence, Nottingham, UK, 5\u20137 September 2018, Springer."},{"key":"ref_24","unstructured":"Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50\u00d7 fewer parameters and <0.5 MB model size. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18\u201323). Shufflenet: An extremely efficient convolutional neural network for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00716"},{"key":"ref_26","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18\u201323). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Lowe, D.G. (1999, January 20\u201325). Object recognition from local scale-invariant features. Proceedings of the Seventh IEEE International Conference on Computer Vision, Corfu, Greece.","DOI":"10.1109\/ICCV.1999.790410"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cviu.2007.09.014","article-title":"Speeded-up robust features (SURF)","volume":"110","author":"Bay","year":"2008","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Konlambigue, S., Pothin, J.B., Honeine, P., and Bensrhair, A. (2019, January 10\u201312). Performance Evaluation of State-of-the-art Filtering Criteria Applied to SIFT Features. Proceedings of the 19th IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Ajman, United Arab Emirates.","DOI":"10.1109\/ISSPIT47144.2019.9001815"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Garg, S., Fischer, T., and Milford, M. (2021, January 21). Where Is Your Place, Visual Place Recognition?. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21). International Joint Conferences on Artificial Intelligence, Montreal, QC, Canada.","DOI":"10.24963\/ijcai.2021\/603"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Quattoni, A., and Torralba, A. (2009, January 20\u201325). Recognizing indoor scenes. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206537"},{"key":"ref_33","unstructured":"Lazebnik, S., Schmid, C., and Ponce, J. (2006, January 17\u201322). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201906), New York, NY, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., and Torralba, A. (2010, January 13\u201318). Sun database: Large-scale scene recognition from abbey to zoo. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539970"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Patterson, G., and Hays, J. (2012, January 16\u201321). Sun attribute database: Discovering, annotating, and recognizing scene attributes. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6247998"},{"key":"ref_36","unstructured":"Schubert, S., and Neubert, P. (2021). What makes visual place recognition easy or hard?. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Ashraf, I., Hur, S., and Park, Y. (2020). Smartphone sensor based indoor positioning: Current status, opportunities, and future challenges. Electronics, 9.","DOI":"10.3390\/electronics9060891"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"109349","DOI":"10.1016\/j.measurement.2021.109349","article-title":"Smartphone-based indoor localization techniques: State-of-the-art and classification","volume":"179","author":"Tiglao","year":"2021","journal-title":"Measurement"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"874","DOI":"10.1109\/TMM.2016.2636750","article-title":"Fusion of magnetic and visual sensors for indoor localization: Infrastructure-free and more effective","volume":"19","author":"Liu","year":"2016","journal-title":"IEEE Trans. Multimed."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Ashraf, I., Hur, S., and Park, Y. (2019). Application of deep convolutional neural networks and smartphone sensors for indoor localization. Appl. Sci., 9.","DOI":"10.3390\/app9112337"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Reyes Leiva, K.M., Ja\u00e9n-Vargas, M., Codina, B., and Serrano Olmedo, J.J. (2021). Inertial measurement unit sensors in assistive technologies for visually impaired people, a review. Sensors, 21.","DOI":"10.3390\/s21144767"},{"key":"ref_42","unstructured":"Chien, C. (2013). The Hall Effect and ITS applications, Springer Science & Business Media."},{"key":"ref_43","unstructured":"Liu, D. (2019). Mobile Data and Computation Offloading in Mobile Cloud Computing. [Ph.D. Thesis, Universit\u00e9 de Montr\u00e9al]."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1109\/COMST.2020.2970550","article-title":"Convergence of edge computing and deep learning: A comprehensive survey","volume":"22","author":"Wang","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_45","first-page":"170","article-title":"Machine learning at the network edge: A survey","volume":"54","author":"Murshed","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"615","DOI":"10.1145\/3093337.3037698","article-title":"Neurosurgeon: Collaborative intelligence between the cloud and mobile edge","volume":"45","author":"Kang","year":"2017","journal-title":"ACM SIGARCH Comput. Archit. News"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1145\/3368305","article-title":"DNNTune: Automatic benchmarking DNN models for mobile-cloud computing","volume":"16","author":"Xia","year":"2019","journal-title":"ACM Trans. Archit. Code Optim."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"H\u00f6lzl, M., Neumeier, R., and Ostermayer, G. (2013, January 10\u201315). Analysis of compass sensor accuracy on several mobile devices in an industrial environment. Proceedings of the International Conference on Computer Aided Systems Theory, Las Palmas de Gran Canaria, Spain.","DOI":"10.1007\/978-3-642-53862-9_49"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/12\/5672\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:57:01Z","timestamp":1760126221000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/12\/5672"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,17]]},"references-count":48,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["s23125672"],"URL":"https:\/\/doi.org\/10.3390\/s23125672","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,17]]}}}