{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T14:54:46Z","timestamp":1781621686021,"version":"3.54.5"},"reference-count":44,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2022,1,30]],"date-time":"2022-01-30T00:00:00Z","timestamp":1643500800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001711","name":"Swiss National Science Foundation","doi-asserted-by":"publisher","award":["195964"],"award-info":[{"award-number":["195964"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]},{"name":"University of Applied Sciences and Arts Western Switzerland","award":["Mesures de soutien aux carri\u00e8res ralenties en raison de la crise du Covid"],"award-info":[{"award-number":["Mesures de soutien aux carri\u00e8res ralenties en raison de la crise du Covid"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Despite the great attention that the research community has paid to the creation of novel indoor positioning methods, a rather limited volume of works has focused on the confidence that Indoor Positioning Systems (IPS) assign to the position estimates that they produce. The concept of estimating, dynamically, the accuracy of the position estimates provided by an IPS has been sporadically studied in the literature of the field. Recently, this concept has started being studied as well in the context of outdoor positioning systems of Internet of Things (IoT) based on Low-Power Wide-Area Networks (LPWANs). What is problematic is that the consistent comparison of the proposed methods is quasi nonexistent: new methods rarely use previous ones as baselines; often, a small number of evaluation metrics are reported while different metrics are reported among different relevant publications, the use of open data is rare, and the publication of open code is absent. In this work, we present an open-source, reproducible benchmarking framework for evaluating and consistently comparing various methods of Dynamic Accuracy Estimation (DAE). This work reviews the relevant literature, presenting in a consistent terminology commonalities and differences and discussing baselines and evaluation metrics. Moreover, it evaluates multiple methods of DAE using open data, open code, and a rich set of relevant evaluation metrics. This is the first work aiming to establish the state of the art of methods of DAE determination in IPS and in LPWAN positioning systems, through an open, transparent, holistic, reproducible, and consistent evaluation of the methods proposed in the relevant literature.<\/jats:p>","DOI":"10.3390\/s22031088","type":"journal-article","created":{"date-parts":[[2022,1,31]],"date-time":"2022-01-31T01:46:21Z","timestamp":1643593581000},"page":"1088","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Can I Trust This Location Estimate? Reproducibly Benchmarking the Methods of Dynamic Accuracy Estimation of Localization"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8643-7427","authenticated-orcid":false,"given":"Grigorios G.","family":"Anagnostopoulos","sequence":"first","affiliation":[{"name":"Geneva School of Business Administration (DMML Group), HES-SO, 1227 Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6282-0686","authenticated-orcid":false,"given":"Alexandros","family":"Kalousis","sequence":"additional","affiliation":[{"name":"Geneva School of Business Administration (DMML Group), HES-SO, 1227 Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1080\/17489725.2018.1508763","article-title":"Location based services: Ongoing evolution and research agenda","volume":"12","author":"Huang","year":"2018","journal-title":"J. Locat. Based Serv."},{"key":"ref_2","unstructured":"(2022, January 01). Android Development\u2013Location Class\u2013getAccuracy\u2013Definition. Available online: https:\/\/developer.android.com\/reference\/android\/location\/Location.html#getAccuracy()."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Moghtadaiee, V., Dempster, A.G., and Li, B. (2012, January 23\u201326). Accuracy indicator for fingerprinting localization systems. Proceedings of the 2012 IEEE\/ION Position, Location and Navigation Symposium, Myrtle Beach, SC, USA.","DOI":"10.1109\/PLANS.2012.6236976"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zou, D., Meng, W., and Han, S. (2014, January 18\u201320). An Accuracy Estimation Algorithm for Fingerprint Positioning System. Proceedings of the 2014 Fourth International Conference on Instrumentation and Measurement, Computer, Communication and Control, Harbin, China.","DOI":"10.1109\/IMCCC.2014.123"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Nikitin, A., Laoudias, C., Chatzimilioudis, G., Karras, P., and Zeinalipour-Yazti, D. (June, January 29). Indoor Localization Accuracy Estimation from Fingerprint Data. Proceedings of the 2017 18th IEEE International Conference on Mobile Data Management (MDM), Daejeon, Korea.","DOI":"10.1109\/MDM.2017.34"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Krumm, J., Abowd, G.D., Seneviratne, A., and Strang, T. (2007). An Exploration of Location Error Estimation. UbiComp 2007: Ubiquitous Computing, Springer.","DOI":"10.1007\/978-3-540-74853-3"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Choudhury, T., Quigley, A., Strang, T., and Suginuma, K. (2009). Error Estimation for Indoor 802.11 Location Fingerprinting. Location and Context Awareness, Springer.","DOI":"10.1007\/978-3-642-01721-6"},{"key":"ref_8","first-page":"1","article-title":"Dynamic nearest neighbors and online error estimation for SMARTPOS","volume":"6","author":"Marcus","year":"2013","journal-title":"Int. J. Adv. Internet Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"33652","DOI":"10.1109\/ACCESS.2019.2903880","article-title":"Regression-Based Estimation of Individual Errors in Fingerprinting Localization","volume":"7","author":"Lemic","year":"2019","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3585","DOI":"10.1109\/JIOT.2018.2889303","article-title":"Toward Robust Crowdsourcing-Based Localization: A Fingerprinting Accuracy Indicator Enhanced Wireless\/Magnetic\/Inertial Integration Approach","volume":"6","author":"Li","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_11","unstructured":"Elbakly, R., and Youssef, M. (2016). CONE: Zero-Calibration Accurate Confidence Estimation for Indoor Localization Systems. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Anagnostopoulos, G.G., and Kalousis, A. (2021, January 14\u201323). Analysing the Data-Driven Approach of Dynamically Estimating Positioning Accuracy. Proceedings of the ICC 2021\u2014IEEE International Conference on Communications, Montreal, QC, Canada.","DOI":"10.1109\/ICC42927.2021.9500369"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lin, K., Kansal, A., Lymberopoulos, D., and Zhao, F. Energy-Accuracy Trade-off for Continuous Mobile Device Location. Proceedings of the 8th International Conference on Mobile Systems, Applications, and Services, MobiSys \u201910, San Francisco, CA, USA, 15\u201318 June 2010.","DOI":"10.1145\/1814433.1814462"},{"key":"ref_14","unstructured":"Zou, D., Meng, W., and Han, S. (2013, January 7\u201310). Euclidean distance based handoff algorithm for fingerprint positioning of WLAN system. Proceedings of the 2013 IEEE Wireless Communications and Networking Conference (WCNC), Shanghai, China."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Anagnostopoulos, G.G., and Deriaz, M. (2015, January 13\u201316). Automatic switching between indoor and outdoor position providers. Proceedings of the 2015 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Banff, AB, Canada.","DOI":"10.1109\/IPIN.2015.7346948"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Lemic, F., Handziski, V., and Famaey, J. (May, January 28). Toward Regression-Based Estimation of Localization Errors in Fingerprinting-Based Localization. Proceedings of the 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring), Kuala Lumpur, Malaysia.","DOI":"10.1109\/VTCSpring.2019.8746700"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Lemic, F., and Famaey, J. (2020, January 10\u201313). Artificial Neural Network-based Estimation of Individual Localization Errors in Fingerprinting. Proceedings of the 2020 IEEE 17th Annual Consumer Communications Networking Conference (CCNC), Las Vegas, NV, USA.","DOI":"10.1109\/CCNC46108.2020.9045648"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Torres-Sospedra, J., Jim\u00e9nez, A., Moreira, A., Lungenstrass, T., Lu, W.C., Knauth, S., Mendoza-Silva, G., Seco, F., P\u00e9rez-Navarro, A., and Nicolau, M. (2018). Off-Line Evaluation of Mobile-Centric Indoor Positioning Systems: The Experiences from the 2017 IPIN Competition. Sensors, 18.","DOI":"10.3390\/s18020487"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Aernouts, M., Berkvens, R., Van Vlaenderen, K., and Weyn, M. (2018). Sigfox and LoRaWAN Datasets for Fingerprint Localization in Large Urban and Rural Areas. Data, 3.","DOI":"10.20944\/preprints201803.0139.v1"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Berkvens, R., Weyn, M., and Peremans, H. (2016, January 4\u20137). Position error and entropy of probabilistic Wi-Fi fingerprinting in the UJIIndoorLoc dataset. Proceedings of the 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Alcala de Henares, Spain.","DOI":"10.1109\/IPIN.2016.7743691"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Berkvens, R., Peremans, H., and Weyn, M. (2016). Conditional Entropy and Location Error in Indoor Localization Using Probabilistic Wi-Fi Fingerprinting. Sensors, 16.","DOI":"10.3390\/s16101636"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Khandker, S., Mondal, R., and Ristaniemi, T. (October, January 30). Positioning Error Prediction and Training Data Evaluation in RF Fingerprinting Method. Proceedings of the 2019 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Pisa, Italy.","DOI":"10.1109\/IPIN.2019.8911821"},{"key":"ref_23","unstructured":"Grigorios, A., and Alexandros, K. (2021, January 01). Can I Trust This Location Estimate? Reproducibly Benchmarking the Methods of Dynamic Accuracy Estimation of Localization (code). Available online: https:\/\/zenodo.org\/record\/5589651#.YfOqHC1Q0UE."},{"key":"ref_24","unstructured":"King, T., Kopf, S., Haenselmann, T., Lubberger, C., and Effelsberg, W. (2022, January 01). CRAWDAD Dataset Mannheim\/Compass (v. 2008-04-11). Available online: https:\/\/crawdad.org\/mannheim\/compass\/20080411."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"King, T., Haenselmann, T., and Effelsberg, W. (2008, January 23\u201326). On-demand fingerprint selection for 802.11-based positioning systems. Proceedings of the 2008 International Symposium on a World of Wireless, Mobile and Multimedia Networks, Newport Beach, CA, USA.","DOI":"10.1109\/WOWMOM.2008.4594839"},{"key":"ref_26","unstructured":"Moreira, A., Silva, I., and Torres-Sospedra, J. (2022, January 01). The DSI Dataset for Wi-Fi Fingerprinting Using Mobile Devices. Available online: https:\/\/zenodo.org\/record\/3778646#.YfNHtOpBxPY."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Beder, C., McGibney, A., and Klepal, M. (2011, January 21\u201323). Predicting the expected accuracy for fingerprinting based WiFi localisation systems. Proceedings of the 2011 International Conference on Indoor Positioning and Indoor Navigation, Guimaraes, Portugal.","DOI":"10.1109\/IPIN.2011.6071939"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"de la Osa, C.M., Anagnostopoulos, G.G., Togneri, M., Deriaz, M., and Konstantas, D. (2016, January 4\u20137). Positioning evaluation and ground truth definition for real life use cases. Proceedings of the 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Alcala de Henares, Spain.","DOI":"10.1109\/IPIN.2016.7743650"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Moayeri, N., Li, C., and Shi, L. (2018, January 24\u201327). PerfLoc (Part 2): Performance Evaluation of the Smartphone Indoor Localization Apps. Proceedings of the 2018 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Nantes, France.","DOI":"10.1109\/IPIN.2018.8533860"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1109\/MSP.2017.2713817","article-title":"The Microsoft Indoor Localization Competition: Experiences and Lessons Learned","volume":"34","author":"Lymberopoulos","year":"2017","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Anagnostopoulos, G.G., de la Osa, C.M., Nunes, T., Hammoud, A., Deriaz, M., and Konstantas, D. (2016, January 2\u20134). Practical evaluation and tuning methodology for indoor positioning systems. Proceedings of the 2016 Fourth International Conference on Ubiquitous Positioning, Indoor Navigation and Location Based Services (UPINLBS), Shanghai, China.","DOI":"10.1109\/UPINLBS.2016.7809961"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Anagnostopoulos, G.G., Deriaz, M., and Konstantas, D. (2017, January 18\u201321). A multiobjective optimization methodology of tuning indoor positioning systems. Proceedings of the 2017 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Sapporo, Japan.","DOI":"10.1109\/IPIN.2017.8115908"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Montoliu, R., Sansano, E., Torres-Sospedra, J., and Belmonte, O. (2017, January 18\u201321). IndoorLoc platform: A public repository for comparing and evaluating indoor positioning systems. Proceedings of the 2017 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Sapporo, Japan.","DOI":"10.1109\/IPIN.2017.8115940"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Adler, S., Schmitt, S., Wolter, K., and Kyas, M. (2015, January 13\u201316). A survey of experimental evaluation in indoor localization research. Proceedings of the 2015 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Banff, AB, Canada.","DOI":"10.1109\/IPIN.2015.7346749"},{"key":"ref_35","unstructured":"Anagnostopoulos, G.G., and Kalousis, A. (December, January 29). Towards Reproducible Indoor Positioning Research. Proceedings of the 2021 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Lloret de Mar, Spain."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"100235","DOI":"10.1016\/j.iot.2020.100235","article-title":"Benchmarking RSS-based localization algorithms with LoRaWAN","volume":"11","author":"Janssen","year":"2020","journal-title":"Internet Things"},{"key":"ref_37","unstructured":"Torres-Sospedra, J., Richter, P., Moreira, A., Mendoza-Silva, G., Lohan, E.S., Trilles, S., Matey-Sanz, M., and Huerta, J. (2020). A Comprehensive and Reproducible Comparison of Clustering and Optimization Rules in Wi-Fi Fingerprinting. IEEE Trans. Mob. Comput."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"9263","DOI":"10.1016\/j.eswa.2015.08.013","article-title":"Comprehensive analysis of distance and similarity measures for Wi-Fi fingerprinting indoor positioning systems","volume":"42","author":"Montoliu","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_39","unstructured":"Aernouts, M., Berkvens, R., Van Vlaenderen, K., and Weyn, M. (2022, January 01). Sigfox and LoRaWAN Datasets for Fingerprint Localization in Large Urban and Rural Areas. Available online: https:\/\/zenodo.org\/record\/3904158#.YfNIfOpBxPY."},{"key":"ref_40","unstructured":"Grigorios, A., and Alexandros, K. (2022, January 01). Analysing the Data-Driven approach of Dynamically Estimating Positioning Accuracy (Data). Available online: https:\/\/zenodo.org\/record\/4117818#.YfNIfOpBxPY."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Anagnostopoulos, G.G., and Kalousis, A. (October, January 30). A Reproducible Analysis of RSSI Fingerprinting for Outdoor Localization Using Sigfox: Preprocessing and Hyperparameter Tuning. Proceedings of the 2019 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Pisa, Italy.","DOI":"10.1109\/IPIN.2019.8911792"},{"key":"ref_42","unstructured":"Grigorios, A., and Alexandros, K. (2022, January 01). A Reproducible Analysis of RSSI Fingerprinting for Outdoors Localization Using Sigfox: Preprocessing and Hyperparameter Tuning (datasets). Available online: https:\/\/zenodo.org\/record\/3228744#.YfNIfOpBxPY."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Li, Y., Barthelemy, J., Sun, S., Perez, P., and Moran, B. (2021). Urban vehicle localization in public LoRaWan network. IEEE Internet Things J.","DOI":"10.1109\/JIOT.2021.3121778"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Perez-Navarro, A. (December, January 29). Accuracy of a single point in kNN applying error propagation theory. Proceedings of the 2021 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Lloret de Mar, Spain.","DOI":"10.1109\/IPIN51156.2021.9662571"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/3\/1088\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:11:31Z","timestamp":1760134291000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/3\/1088"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,30]]},"references-count":44,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["s22031088"],"URL":"https:\/\/doi.org\/10.3390\/s22031088","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,30]]}}}