{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:21:30Z","timestamp":1760235690987,"version":"build-2065373602"},"reference-count":32,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2021,9,13]],"date-time":"2021-09-13T00:00:00Z","timestamp":1631491200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Bayerisches Staatsministerium f\u00fcr Wirtschaft, Landesentwicklung und Energie","award":["5GQMON"],"award-info":[{"award-number":["5GQMON"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Network"],"abstract":"<jats:p>Crowdsourced network measurements (CNMs) are becoming increasingly popular as they assess the performance of a mobile network from the end user\u2019s perspective on a large scale. Here, network measurements are performed directly on the end-users\u2019 devices, thus taking advantage of the real-world conditions end-users encounter. However, this type of uncontrolled measurement raises questions about its validity and reliability. The problem lies in the nature of this type of data collection. In CNMs, mobile network subscribers are involved to a large extent in the measurement process, and collect data themselves for the operator. The collection of data on user devices in arbitrary locations and at uncontrolled times requires means to ensure validity and reliability. To address this issue, our paper defines concepts and guidelines for analyzing the precision of CNMs; specifically, the number of measurements required to make valid statements. In addition to the formal definition of the aspect, we illustrate the problem and use an extensive sample data set to show possible assessment approaches. This data set consists of more than 20.4 million crowdsourced mobile measurements from across France, measured by a commercial data provider.<\/jats:p>","DOI":"10.3390\/network1020013","type":"journal-article","created":{"date-parts":[[2021,9,14]],"date-time":"2021-09-14T23:34:11Z","timestamp":1631662451000},"page":"215-232","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Valid Statements by the Crowd: Statistical Measures for Precision in Crowdsourced Mobile Measurements"],"prefix":"10.3390","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0356-6291","authenticated-orcid":false,"given":"Florian","family":"Wamser","sequence":"first","affiliation":[{"name":"Chair of Communication Networks, University of W\u00fcrzburg, 97074 W\u00fcrzburg, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3329-9910","authenticated-orcid":false,"given":"Anika","family":"Seufert","sequence":"additional","affiliation":[{"name":"Chair of Communication Networks, University of W\u00fcrzburg, 97074 W\u00fcrzburg, Germany"}]},{"given":"Andrew","family":"Hall","sequence":"additional","affiliation":[{"name":"Tutela Technologies, Ltd., Victoria, BC V8W 1H8, Canada"}]},{"given":"Stefan","family":"Wunderer","sequence":"additional","affiliation":[{"name":"Nokia Networks, 89081 Ulm, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0173-595X","authenticated-orcid":false,"given":"Tobias","family":"Ho\u00dffeld","sequence":"additional","affiliation":[{"name":"Chair of Communication Networks, University of W\u00fcrzburg, 97074 W\u00fcrzburg, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,13]]},"reference":[{"key":"ref_1","unstructured":"Ho\u00dffeld, T., Wunderer, S., Beyer, A., Hall, A., Schwind, A., Gassner, C., Guillemin, F., Wamser, F., Wascinski, K., and Hirth, M. (2020). White Paper on Crowdsourced Network and QoE Measurements\u2014Definitions, Use Cases and Challenges, University of Wuerzburg. Technical Report 10.25972\/OPUS-20232."},{"key":"ref_2","unstructured":"Howe, J. (Wired Magazine, 2006). The Rise of Crowdsourcing, Wired Magazine."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.comnet.2015.07.003","article-title":"Crowdsourced Network Measurements: Benefits and Best Practices","volume":"90","author":"Hirth","year":"2015","journal-title":"Comput. Netw."},{"key":"ref_4","unstructured":"International Organization for Standardization (1994). Accuracy (Trueness and Precision) of Measurement Methods and Results\u2014Part 1: General Principles and Definitions (ISO 5725-1:1994), International Organization for Standardization."},{"key":"ref_5","unstructured":"Dano, M. (2021, January 01). Who Won the Super Bowl Speed Game? It Depends on Who You Ask. Available online: https:\/\/www.lightreading.com\/testing\/who-won-the-super-bowl-speed-game-it-depends-on-who-you-ask\/d\/d-id\/749241."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Midoglu, C., and Svoboda, P. (2016, January 13\u201315). Opportunities and challenges of using crowdsourced measurements for mobile network benchmarking a case study on RTR open data. Proceedings of the 2016 SAI Computing Conference (SAI), London, UK.","DOI":"10.1109\/SAI.2016.7556101"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Seufert, A., Wamser, F., Wunderer, S., Hall, A., and Ho\u00dffeld, T. (2021, January 8\u201311). Trust but Verify: Crowdsourced Mobile Network Measurements and Statistical Validity Measures. Proceedings of the 2021 Joint European Conference on Networks and Communications & 6G Summit (EuCNC\/6G Summit), Porto, Portugal.","DOI":"10.1109\/EuCNC\/6GSummit51104.2021.9482428"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1109\/TMM.2013.2291663","article-title":"Best Practices for QoE Crowdtesting: QoE Assessment with Crowdsourcing","volume":"16","author":"Hossfeld","year":"2013","journal-title":"IEEE Trans. Multimed."},{"key":"ref_9","unstructured":"Ho\u00dffeld, T., Hirth, M., Redi, J., Mazza, F., Korshunov, P., Naderi, B., Seufert, M., Gardlo, B., Egger, S., and Keimel, C. (2014). Best Practices and Recommendations for Crowdsourced QoE-Lessons Learned from the Qualinet Task Force \u201cCrowdsourcing\u201d, Hindustan Aeronautics Limited. Technical Report hal-01078761."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Neidhardt, E., Uzun, A., Bareth, U., and K\u00fcpper, A. (2013, January 23\u201325). Estimating Locations and Coverage Areas of Mobile Network Cells based on Crowdsourced Data. Proceedings of the 6th Joint IFIP Wireless and Mobile Networking Conference (WMNC), Dubai, United Arab Emirates.","DOI":"10.1109\/WMNC.2013.6549010"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Li, Z., Nika, A., Zhang, X., Zhu, Y., Yao, Y., Zhao, B.Y., and Zheng, H. (2017, January 3\u20137). Identifying Value in Crowdsourced Wireless Signal Measurements. Proceedings of the 26th International Conference on World Wide Web, Perth, Australia.","DOI":"10.1145\/3038912.3052563"},{"key":"ref_12","unstructured":"Wamser, F., Seufert, M., Casas, P., Irmer, R., Tran-Gia, P., and Schatz, R. (July, January 29). YoMoApp: A tool for Analyzing QoE of YouTube HTTP Adaptive Streaming in Mobile Networks. Proceedings of the European Conference on Networks and Communications (EuCNC), Paris, France."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"e2058","DOI":"10.1002\/nem.2058","article-title":"Dissecting the Performance of YouTube Video Streaming in Mobile Networks","volume":"30","author":"Schwind","year":"2020","journal-title":"Int. J. Netw. Manag."},{"key":"ref_14","unstructured":"Schwind, A., Wamser, F., Wunderer, S., Gassner, C., and Ho\u00dffeld, T. (2019). Mobile Internet Experience: Urban vs. Rural\u2014Saturation vs. Starving?. arXiv."},{"key":"ref_15","unstructured":"Schwind, A., Wamser, F., Hossfeld, T., Wunderer, S., Tarnvik, E., and Hall, A. (2020, January 23\u201324). Crowdsourced Network Measurements in Germany: Mobile Internet Experience from End User Perspective. Proceedings of the Broadband Coverage in Germany (14. ITG Symposium), Berlin, Germany."},{"key":"ref_16","unstructured":"4Gmark (2018, February 28). Data Mobile Barometer. White Paper., Available online: https:\/\/www.5gmark.com\/news\/2017\/4Gmark-Barometer-2017-DE.pdf."},{"key":"ref_17","unstructured":"Tutela Technologies (2019). Germany Mobile Experience Report\u2014Country-Level Mobile Experience and Usage Results from Tutela\u2019s Crowdsourced Mobile Network Testing (May\u2013July 2019), Tutela Technologies. Technical Report."},{"key":"ref_18","unstructured":"P3 Communications (2019, January 15). The Great 2019 Mobile Network Test. Available online: http:\/\/p3-networkanalytics.com\/wp-content\/uploads\/2018\/12\/Network-Test-2019-connect-2019-01-English_sp.pdf."},{"key":"ref_19","unstructured":"Opensignal (2019, November 15). Germany\u2014Erfahrungsbericht mit mobilem Netzwerk (November 2019). Available online: https:\/\/www.opensignal.com\/de\/reports\/2019\/11\/germany\/mobile-network-experience."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bischof, Z.S., Otto, J.S., S\u00e1nchez, M.A., Rula, J.P., Choffnes, D.R., and Bustamante, F.E. (2011). Crowdsourcing ISP Characterization to the Network Edge. First ACM SIGCOMM Workshop on Measurements up the Stack, ACM.","DOI":"10.1145\/2018602.2018617"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"154899","DOI":"10.1109\/ACCESS.2019.2949051","article-title":"Crowdsensed performance benchmarking of mobile networks","volume":"7","author":"Raida","year":"2019","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Rich, R., Brians, C., and Willnat, L. (2018). Empirical Political Analysis: Quantitative and Qualitative Research Methods, Routledge.","DOI":"10.4324\/9781315109664"},{"key":"ref_23","unstructured":"Eid, M., and Schmidt, K. (2014). Testtheorie und Testkonstruktion, Hogrefe."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chow, S., and Liu, J. (2004). Design and Analysis of Clinical Trials: Concepts and Methodologies, Wiley.","DOI":"10.1002\/0471473286"},{"key":"ref_25","first-page":"i-208","article-title":"Validity","volume":"1987","author":"Messick","year":"1987","journal-title":"ETS Res. Rep. Ser."},{"key":"ref_26","unstructured":"Stuart, A. (1976). Basic Ideas of Scientific Sampling, Hafner Press. Griffin\u2019s Statistical Monographs."},{"key":"ref_27","unstructured":"Deutsches Institut f\u00fcr Normung (2001). Qualit\u00e4tsmanagement und Statistik: Normen. Begriffe, Beuth. Number Bd. 1 in DIN-Taschenbuch."},{"key":"ref_28","unstructured":"Wirtz, M., and Strohmer, J. (2016). Dorsch-Lexikon der Psychologie, Dorsch\u2014Lexikon der Psychologie, Hogrefe."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1017\/S0003055401003100","article-title":"Measurement Validity: A Shared Standard for Qualitative and Quantitative Research","volume":"95","author":"Adcock","year":"2001","journal-title":"Am. Political Sci."},{"key":"ref_30","unstructured":"Law, A.M., and Kelton, W.D. (2000). Simulation Modeling and Analysis, McGraw-Hill."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Efron, B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans, SIAM.","DOI":"10.1137\/1.9781611970319"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Efron, B. (1992). Bootstrap methods: Another look at the jackknife. Breakthroughs in Statistics, Springer.","DOI":"10.1007\/978-1-4612-4380-9_41"}],"container-title":["Network"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2673-8732\/1\/2\/13\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:02:00Z","timestamp":1760166120000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2673-8732\/1\/2\/13"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,13]]},"references-count":32,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["network1020013"],"URL":"https:\/\/doi.org\/10.3390\/network1020013","relation":{},"ISSN":["2673-8732"],"issn-type":[{"type":"electronic","value":"2673-8732"}],"subject":[],"published":{"date-parts":[[2021,9,13]]}}}