{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T21:35:49Z","timestamp":1784237749651,"version":"3.55.0"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,11,30]],"date-time":"2022-11-30T00:00:00Z","timestamp":1669766400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Urban Innovative Action Healthy Outdoor Premises for Everyone","award":["UIA03-240"],"award-info":[{"award-number":["UIA03-240"]}]},{"name":"MegaSense program","award":["324576"],"award-info":[{"award-number":["324576"]}]},{"name":"Foundations of Pervasive Sensing Systems","award":["339614"],"award-info":[{"award-number":["339614"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Sen. Netw."],"published-print":{"date-parts":[[2022,11,30]]},"abstract":"<jats:p>We contribute a novel model evaluation technique that divides available measurements into training and testing sets in a way that adheres to the requirements imposed on professional monitoring stations. We perform extensive and systematic experiments with a wide range of state-of-the-art calibration models to demonstrate that our approach provides accurate insights about the performance of calibration models in real-world deployments, while at the same time highlighting issues with evaluation techniques used in previous works. Among others, our results show that although trained and tested in the same location, calibration errors can exhibit deviation up to 116% depending on the evaluation protocol that is being adopted. We also demonstrate that models trained with continuous data can suffer up to 76% greater error when tested with data coming from diverse environmental conditions. In contrast, when models are trained and tested with our method, the variability of errors is significantly reduced and the robustness of calibration models is significantly improved. The overall performance improvements depend on pollutant concentration, ranging from 10% for low concentrations to 90% for high concentrations that represent conditions that are most dangerous for human health.<\/jats:p>","DOI":"10.1145\/3512889","type":"journal-article","created":{"date-parts":[[2022,4,19]],"date-time":"2022-04-19T12:37:58Z","timestamp":1650371878000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":26,"title":["Evaluation of Low-cost Air Quality Sensor Calibration Models"],"prefix":"10.1145","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4387-7876","authenticated-orcid":false,"given":"Kasimir","family":"Aula","sequence":"first","affiliation":[{"name":"University of Helsinki, Helsinki, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eemil","family":"Lagerspetz","sequence":"additional","affiliation":[{"name":"University of Helsinki, Helsinki, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8262-6434","authenticated-orcid":false,"given":"Petteri","family":"Nurmi","sequence":"additional","affiliation":[{"name":"University of Helsinki, Helsinki, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sasu","family":"Tarkoma","sequence":"additional","affiliation":[{"name":"University of Helsinki, Helsinki, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,12,22]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.comnet.2020.107447"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.3390\/atmos11050492"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2011.12.028"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.atmosenv.2016.09.050"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.atmosenv.2018.08.028"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/APCCAS.2018.8605619"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/2668332.2668346"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3446005"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.snb.2018.04.021"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2020.04.032"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.snb.2020.127869"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.atmosenv.2008.07.020"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aaa9375"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.snb.2016.03.038"},{"key":"e_1_3_2_16_2","volume-title":"Guide To The Demonstration of Equivalence of Ambient Air Monitoring Methods","author":"Agency European Environment","year":"2010","unstructured":"European Environment Agency. 2010. Guide To The Demonstration of Equivalence of Ambient Air Monitoring Methods. Retrieved from http:\/\/ec.europa.eu\/environment\/air\/quality\/legislation\/pdf\/equivalence.pdf."},{"key":"e_1_3_2_17_2","volume-title":"Air Quality in Europe\u20142018 Report","author":"Agency European Environment","year":"2018","unstructured":"European Environment Agency. 2018. Air Quality in Europe\u20142018 Report. Retrieved from https:\/\/www.eea.europa.eu\/publications\/air-quality-in-europe-2018."},{"issue":"1","key":"e_1_3_2_18_2","first-page":"40","article-title":"Toward large-scale autonomous marine pollution monitoring","volume":"4","author":"Flores Huber","year":"2021","unstructured":"Huber Flores, Naser Hossein Motlagh, Agustin Zuniga, Mohan Liyanage, Monica Passananti, Sasu Tarkoma, Moustafa Youssef, and Petteri Nurmi. 2021. Toward large-scale autonomous marine pollution monitoring. IEEE IoT Mag. 4, 1 (2021), 40\u201345.","journal-title":"IEEE IoT Mag."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/1882471.1882479"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM.2016.7524478"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2018.2793950"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.5194\/amt-11-315-2018"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/2750858.2807551"},{"key":"e_1_3_2_24_2","first-page":"1","volume-title":"Proceedings of the 2nd International Workshop on Mobile Sensing","author":"Hasenfratz David","year":"2012","unstructured":"David Hasenfratz, Olga Saukh, Silvan Sturzenegger, and Lothar Thiele. 2012. Participatory air pollution monitoring using smartphones. In Proceedings of the 2nd International Workshop on Mobile Sensing. Academic Press, 1\u20135."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.5194\/amt-7-1121-2014"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/0004-6981(79)90272-5"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.atmosenv.2018.04.019"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.atmosenv.2004.02.026"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.envpol.2017.09.042"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.3390\/s20133617"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1038\/535029a"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3191750"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.3390\/atmos10020041"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.5555\/2893711.2893735"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2018.2853660"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3191756"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.5194\/amt-12-903-2019"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.001.1900515"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trd.2021.102981"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.240"},{"key":"e_1_3_2_41_2","unstructured":"Sebastian Raschka. 2020. Model evaluation model selection and algorithm selection in machine learning. arXiv:1811.12808 [cs.LG]. Retrieved from https:\/\/arxiv.org\/abs\/1811.12808."},{"key":"e_1_3_2_42_2","series-title":"Proceedings of the 35th International Conference on Machine Learning, ICML","first-page":"4331","volume":"80","author":"Ren Mengye","year":"2018","unstructured":"Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In Proceedings of the 35th International Conference on Machine Learning, ICML(Proceedings of Machine Learning Research, Vol. 80), Jennifer G. Dy and Andreas Krause (Eds.). PMLR, 4331\u20134340. http:\/\/proceedings.mlr.press\/v80\/ren18a.html."},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1021\/acs.est.7b00891"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.5194\/amt-14-37-2021"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.5194\/amt-13-1693-2020"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.snb.2015.03.031"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.snb.2016.07.036"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.3390\/environments4010021"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.atmosenv.2019.04.048"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.5194\/amt-12-4211-2019"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.snb.2007.09.060"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.snb.2009.08.041"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2020.2978596"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSEN.2020.3010316"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3241539.3241570"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2884647"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.5194\/amt-11-291-2018"}],"container-title":["ACM Transactions on Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3512889","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3512889","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:12:27Z","timestamp":1750191147000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3512889"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,30]]},"references-count":57,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,11,30]]}},"alternative-id":["10.1145\/3512889"],"URL":"https:\/\/doi.org\/10.1145\/3512889","relation":{},"ISSN":["1550-4859","1550-4867"],"issn-type":[{"value":"1550-4859","type":"print"},{"value":"1550-4867","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,30]]},"assertion":[{"value":"2021-05-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-01-19","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-12-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}