{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:45:08Z","timestamp":1740149108091,"version":"3.37.3"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2021,6,7]],"date-time":"2021-06-07T00:00:00Z","timestamp":1623024000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,7]],"date-time":"2021-06-07T00:00:00Z","timestamp":1623024000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"publisher","award":["ERC-2017-ADG#787576"],"award-info":[{"award-number":["ERC-2017-ADG#787576"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"publisher","award":["ERC-2017-ADG#787576"],"award-info":[{"award-number":["ERC-2017-ADG#787576"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"publisher","award":["ERC-2017-ADG#787576"],"award-info":[{"award-number":["ERC-2017-ADG#787576"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Forschungszentrum J\u00fclich GmbH"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Earth Sci Inform"],"published-print":{"date-parts":[[2021,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We present context-aware benchmarking and performance engineering of a mature TByte-scale air quality database system which was created by the Tropospheric Ozone Assessment Report (TOAR) and contains one of the world\u2019s largest collections of near-surface air quality measurements. A special feature of our data service<jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/join.fz-juelich.de\">https:\/\/join.fz-juelich.de<\/jats:ext-link>is on-demand processing of several air quality metrics directly from the TOAR database. As a service that is used by more than 350 users of the international air quality research community, our web service must be easily accessible and functionally flexible, while delivering good performance. The current on-demand calculations of air quality metrics outside the database together with the necessary transfer of large volume raw data are identified as the major performance bottleneck. In this study, we therefore explore and benchmark in-database approaches for the statistical processing, which results in performance enhancements of up to 32%.<\/jats:p>","DOI":"10.1007\/s12145-021-00631-4","type":"journal-article","created":{"date-parts":[[2021,6,7]],"date-time":"2021-06-07T04:03:31Z","timestamp":1623038611000},"page":"1597-1607","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Context aware benchmarking and tuning of a TByte-scale air quality database and web service"],"prefix":"10.1007","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1347-5297","authenticated-orcid":false,"given":"Clara","family":"Betancourt","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1528-0933","authenticated-orcid":false,"given":"Bj\u00f6rn","family":"Hagemeier","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0309-8010","authenticated-orcid":false,"given":"Sabine","family":"Schr\u00f6der","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3455-774X","authenticated-orcid":false,"given":"Martin G.","family":"Schultz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,6,7]]},"reference":[{"key":"631_CR1","doi-asserted-by":"publisher","unstructured":"Bereta K, Caumont H, Daniels U, Dirk D, Goor E, Koubarakis M, Pantazi DA, Stamoulis G, Ubels S, Venus V et al (2019) The copernicus app lab project: Easy access to copernicus data. In: Proceedings of the 22nd International Conference on Extending Database Technology, EDBT, p 11 https:\/\/doi.org\/10.5441\/002%2Fedbt.2019.46","DOI":"10.5441\/002%2Fedbt.2019.46"},{"issue":"1","key":"631_CR2","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1021\/acs.est.5b03709","volume":"50","author":"M Brauer","year":"2016","unstructured":"Brauer M, Freedman G, Frostad J, Van Donkelaar A, Martin RV, Dentener F, Rv Dingenen, Estep K, Amini H, Apte JS et al (2016) Ambient air pollution exposure estimation for the global burden of disease 2013. Environmental science & technology 50(1):79\u201388","journal-title":"Environmental science & technology"},{"key":"631_CR3","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1525\/elementa.243","volume":"5","author":"KL Chang","year":"2017","unstructured":"Chang KL, Petropavlovskikh I, Copper OR, Schultz MG, Wang T (2017) Regional trend analysis of surface ozone observations from monitoring networks in eastern north america, europe and east asia. Elem Sci Anth 5:50. https:\/\/doi.org\/10.1525\/elementa.243","journal-title":"Elem Sci Anth"},{"key":"631_CR4","unstructured":"Directorate-General for Communication EC (2018) The DIAS: User-friendly access to copernicus data and information. https:\/\/ec.europa.eu\/docsroom\/documents\/30162"},{"key":"631_CR5","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1007\/s007790170019","volume":"5","author":"A Dey","year":"2001","unstructured":"Dey A (2001) Understanding and using context. Pers Ubiquit Comput 5:4\u20137. https:\/\/doi.org\/10.1007\/s007790170019","journal-title":"Pers Ubiquit Comput"},{"issue":"12","key":"631_CR6","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.14778\/3352063.3352102","volume":"12","author":"JV D\u2019Silva","year":"2019","unstructured":"D\u2019Silva JV, De Moor F, Kemme B (2019) Making an RDBMS data scientist friendly: advanced in-database interactive analytics with visualization support. Proceedings of the VLDB Endowment 12(12):1930\u20131933. https:\/\/doi.org\/10.14778\/3352063.3352102","journal-title":"Proceedings of the VLDB Endowment"},{"issue":"1","key":"631_CR7","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1007\/s41748-021-00202-z","volume":"5","author":"A Farahat","year":"2021","unstructured":"Farahat A, Chauhan A, Al Otaibi M, Singh RP (2021) Air quality over major cities of Saudi Arabia during hajj periods of 2019 and 2020. Earth Systems and Environment 5(1):101\u2013114","journal-title":"Earth Systems and Environment"},{"issue":"1","key":"631_CR8","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1525\/elementa.273","volume":"6","author":"ZL Fleming","year":"2018","unstructured":"Fleming ZL, Doherty RM, Von Schneidemesser E, Malley CS, Cooper OR, Pinto JP, Colette A, Xu X, Simpson D, Schultz MG et al (2018) Tropospheric ozone assessment report: Present-day ozone distribution and trends relevant to human health. Elem Sci Anth 6(1):12. https:\/\/doi.org\/10.1525\/elementa.273","journal-title":"Elem Sci Anth"},{"issue":"1","key":"631_CR9","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1525\/elementa.291","volume":"6","author":"A Gaudel","year":"2018","unstructured":"Gaudel A, Cooper OR, Ancellet G, Barret B, Boynard A, Burrows JP, Clerbaux C, Coheur PF, Cuesta J, Cuevas E et al (2018) Tropospheric ozone assessment report: Present-day distribution and trends of tropospheric ozone relevant to climate and global atmospheric chemistry model evaluation. Elem Sci Anth 6(1):39. https:\/\/doi.org\/10.1525\/elementa.291","journal-title":"Elem Sci Anth"},{"key":"631_CR10","doi-asserted-by":"crossref","unstructured":"Gkatzelis GI, Gilman JB, Brown SS, Eskes H, Gomes AR, Lange AC, McDonald BC, Peischl J, Petzold A, Thompson CR et al (2021) The global impacts of covid-19 lockdowns on urban air pollutiona critical review and recommendations. Elementa:, Science of the Anthropocene, vol 9(1)","DOI":"10.1525\/elementa.2021.00176"},{"key":"631_CR11","doi-asserted-by":"publisher","unstructured":"Gray J, Szalay A (2002) The world-wide telescope. Communications of the ACM 45(11):50\u201355 https:\/\/doi.org\/10.1145\/581571.581572, http:\/\/portal.acm.org\/citation.cfm?doid=581571.581572","DOI":"10.1145\/581571.581572"},{"key":"631_CR12","unstructured":"InfluxData, Hajek V, Pur A, Kudibal I (2018) Benchmarking influxDB vs. cassandra for time-series data, metrics & management. Technical report InfluxData, San Francisco, CA"},{"key":"631_CR13","doi-asserted-by":"publisher","unstructured":"Ireland C, Bowers D, Newton M, Waugh K (2009) A classification of object-relational impedance mismatch. https:\/\/doi.org\/10.1109\/DBKDA.2009.11","DOI":"10.1109\/DBKDA.2009.11"},{"key":"631_CR14","doi-asserted-by":"publisher","unstructured":"J\u00fclich Supercomputing Centre (2019a) HDF Cloud \u2013 helmholtz data federation cloud resources at the j\u00fclich supercomputing centre. Journal of large-scale research facilities, vol 5(A137), https:\/\/doi.org\/10.17815\/jlsrf-5-173","DOI":"10.17815\/jlsrf-5-173"},{"key":"631_CR15","doi-asserted-by":"publisher","unstructured":"J\u00fclich Supercomputing Centre (2019b) JUST: Large-scale Multi-tier storage infrastructure at the j\u00fclich supercomputing centre. Journal of large-scale research facilities, vol 5(A136), https:\/\/doi.org\/10.17815\/jlsrf-5-172","DOI":"10.17815\/jlsrf-5-172"},{"issue":"12","key":"631_CR16","doi-asserted-by":"publisher","first-page":"1474","DOI":"10.14778\/3402755.3402799","volume":"4","author":"ML Kersten","year":"2011","unstructured":"Kersten ML, Idreos S, Manegold S, Liarou E (2011) The researcher\u2019s guide to the data deluge: Querying a scientific database in just a few seconds. Proceedings of the VLDB Endowment 4(12):1474\u20131477","journal-title":"Proceedings of the VLDB Endowment"},{"key":"631_CR17","doi-asserted-by":"publisher","unstructured":"Lefohn AS, Hazucha MJ, Shadwick D, Adams WC (2010) An alternative form and level of the human health ozone standard. Inhalation Toxicology 22(12):999\u20131011, https:\/\/doi.org\/10.3109\/08958378.2010.505253","DOI":"10.3109\/08958378.2010.505253"},{"issue":"1","key":"631_CR18","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1525\/elementa.279","volume":"6","author":"AS Lefohn","year":"2018","unstructured":"Lefohn AS, Malley CS, Smith L, Wells B, Hazucha M, Simon H, Naik V, Mills G, Schultz MG, Paoletti E et al (2018) Tropospheric ozone assessment report: Global ozone metrics for climate change, human health, and crop\/ecosystem research. Elem Sci Anth 6(1):28. https:\/\/doi.org\/10.1525\/elementa.279","journal-title":"Elem Sci Anth"},{"key":"631_CR19","doi-asserted-by":"publisher","unstructured":"Lu J, Chen Y, Herodotou H, Babu S (2019) Speedup your analytics: automatic parameter tuning for databases and big data systems. Proceedings of the VLDB Endowment 12(12):1970\u20131973, https:\/\/doi.org\/10.14778\/3352063.3352112","DOI":"10.14778\/3352063.3352112"},{"key":"631_CR20","doi-asserted-by":"publisher","unstructured":"McKinney W. (2010) Data structures for statistical computing in python. In: Walt St\u00e9fan van der, Millman Jarrod (eds) Proceedings of the 9th Python in Science Conference, SCIPY2010, pp 56-61. https:\/\/doi.org\/10.25080\/Majora-92bf1922-00a","DOI":"10.25080\/Majora-92bf1922-00a"},{"key":"631_CR21","unstructured":"Mills G, Harmens H, Hayes F, Pleijel H, B\u00fcker P, Gonz\u00e1lez I, Alonso R, Bender J, Bergmann E, Bermejo V et al (2017) Mapping critical levels for vegetation revised chapter 3 of the manual on methodologies and criteria for modelling and mapping critical loads and levels and air pollution effects, risks and trends. https:\/\/www.researchgate.net\/publication\/329101282_MAPPING_CRITICAL_LEVELS_FOR_VEGETATION_Revised_Chapter_3_of_the_Manual_on_Methodologies_and_Criteria_for_Modelling_and_Mapping_Critical_Loads_and_Levels_and_Air_Pollution_Effects_Risks_and_Trends"},{"issue":"1","key":"631_CR22","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1525\/elementa.302","volume":"6","author":"G Mills","year":"2018","unstructured":"Mills G, Pleijel H, Malley CS, Sinha B, Cooper OR, Schultz MG, Neufeld HS, Simpson D, Sharps K, Feng Z et al (2018) Tropospheric ozone assessment report: Present-day tropospheric ozone distribution and trends relevant to vegetation. Elem Sci Anth 6(1):47. https:\/\/doi.org\/10.1525\/elementa.302","journal-title":"Elem Sci Anth"},{"issue":"10","key":"631_CR23","doi-asserted-by":"publisher","first-page":"1850","DOI":"10.35940\/ijitee.J9225.0881019","volume":"8","author":"M Nasar","year":"2019","unstructured":"Nasar M, Kausar MA (2019) Suitability of influxdb database for iot applications. International Journal of Innovative Technology and Exploring Engineering 8(10):1850\u20131857","journal-title":"International Journal of Innovative Technology and Exploring Engineering"},{"key":"631_CR24","doi-asserted-by":"publisher","unstructured":"Nimalasena A, Getov V (2014) Performance tuning of database systems using a context-aware approach. In: 9Th international conference on computer engineering & systems, ICES, 2014, pp 98-103. https:\/\/doi.org\/10.1109\/ICCES.2014.7030936","DOI":"10.1109\/ICCES.2014.7030936"},{"issue":"12","key":"631_CR25","doi-asserted-by":"publisher","first-page":"1854","DOI":"10.14778\/3352063.3352083","volume":"12","author":"SS Sandha","year":"2019","unstructured":"Sandha SS, Cabrera W, Al-Kateb M, Nair S, Srivastava M (2019) In-database distributed machine learning: demonstration using teradata SQL engine. Proceedings of the VLDB Endowment 12 (12):1854\u20131857. https:\/\/doi.org\/10.14778\/3352063.3352083","journal-title":"Proceedings of the VLDB Endowment"},{"key":"631_CR26","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1525\/elementa.244","volume":"5","author":"MG Schultz","year":"2017","unstructured":"Schultz MG, Schr\u00f6der S, Lyapina O, Cooper O, Galbally I, Petropavlovskikh I, Von Schneidemesser E, Tanimoto H, Elshorbany Y, Naja M et al (2017) Tropospheric ozone assessment report: Database and metrics data of global surface ozone observations. Elem Sci Anth 5:58. https:\/\/doi.org\/10.1525\/elementa.244","journal-title":"Elem Sci Anth"},{"key":"631_CR27","doi-asserted-by":"crossref","unstructured":"Shasha D, Bonnet P (2004) Database tuning principles, experiments, and troubleshooting techniques. Morgan Publishing Directors, Elsevier Science, 340 Pine Street San Francisco, CA, USA","DOI":"10.1145\/1024694.1024720"},{"key":"631_CR28","doi-asserted-by":"publisher","unstructured":"Stanaway JD, Afshin A, Gakidou E, Lim SS, Abate D, Abate KH, Abbafati C, Abbasi N, Abbastabar H, Abd-Allah F, et al. (2018) Global, regional, and national comparative risk assessment of 84 behavioural, environmental and occupational, and metabolic risks or clusters of risks for 195 countries and territories, 1990\u20132017: a systematic analysis for the global burden of disease study 2017. The Lancet 392(10159):1923\u20131994, appendix https:\/\/doi.org\/10.1016\/S0140-6736(18)32225-6","DOI":"10.1016\/S0140-6736(18)32225-6"},{"issue":"1","key":"631_CR29","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1525\/elementa.376","volume":"7","author":"D Tarasick","year":"2019","unstructured":"Tarasick D, Galbally IE, Cooper OR, Schultz MG, Ancellet G, Leblanc T, Wallington TJ, Ziemke J, Liu X, Steinbacher M et al (2019) Tropospheric ozone assessment report: Tropospheric ozone from 1877 to 2016, observed levels, trends and uncertainties. Elem Sci Anth 7(1):39. https:\/\/doi.org\/10.1525\/elementa.376","journal-title":"Elem Sci Anth"},{"key":"631_CR30","doi-asserted-by":"publisher","unstructured":"Thalheim B, Tropmann-Frick M (2011) Performance forecasting for performance critical huge databases. In: Proceedings of the 2011 conference on Information Modelling and Knowledge Bases XXII, EJC 2010, 225, pp 226-236. https:\/\/doi.org\/10.3233\/978-1-60750-689-8-206","DOI":"10.3233\/978-1-60750-689-8-206"},{"key":"631_CR31","unstructured":"The PostgreSQL Global Development Group (2015) Postgresql 10.12 documentation. https:\/\/www.postgresql.org\/docs\/10\/index.html"},{"key":"631_CR32","unstructured":"Federal Register US (2015) National ambient air quality standards for ozone. 40 CFR Part 50, 51, 52, 53, and 58, pp 65292\u201365468"},{"key":"631_CR33","doi-asserted-by":"publisher","unstructured":"van der Walt S, Colbert SC, Varoquaux G (2011) The numpy array: A structure for efficient numerical computation. Computing in Science & Engineering 13(2):22\u201330, https:\/\/doi.org\/10.1109\/MCSE.2011.37","DOI":"10.1109\/MCSE.2011.37"},{"key":"631_CR34","doi-asserted-by":"publisher","unstructured":"Virtanen P, Gommers R, Oliphant TE, Haberland M, Reddy T, Cournapeau D, Burovski E, Peterson P, Weckesser W, Bright J et al (2020) SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods 17:261\u2013272 https:\/\/doi.org\/10.1038\/s41592-019-0686-2","DOI":"10.1038\/s41592-019-0686-2"},{"issue":"1","key":"631_CR35","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1080\/17538947.2017.1351583","volume":"11","author":"J Wagemann","year":"2018","unstructured":"Wagemann J, Clements O, Marco Figuera R, Rossi AP, Mantovani S (2018) Geospatial web services pave new ways for server-based on-demand access and processing of big earth data. International Journal of Digital Earth 11(1):7\u201325. https:\/\/doi.org\/10.1080\/17538947.2017.1351583","journal-title":"International Journal of Digital Earth"},{"issue":"5","key":"631_CR36","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1016\/0306-4573(92)90034-W","volume":"28","author":"J Westland","year":"1992","unstructured":"Westland J (1992) Economic incentives for database normalization. Information Processing & Management 28(5):647\u2013662. https:\/\/doi.org\/10.1016\/0306-4573(92)90034-W","journal-title":"Information Processing & Management"},{"issue":"1","key":"631_CR37","doi-asserted-by":"publisher","first-page":"160018","DOI":"10.1038\/sdata.2016.18","volume":"3","author":"MD Wilkinson","year":"2016","unstructured":"Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, Blomberg N, Boiten JW, da Silva Santos LB, Bourne PE et al (2016) The FAIR guiding principles for scientific data management and stewardship. Scientific Data 3(1):160018. https:\/\/doi.org\/10.1038\/sdata.2016.18","journal-title":"Scientific Data"},{"key":"631_CR38","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1525\/elementa.409","volume":"7","author":"X Xu","year":"2020","unstructured":"Xu X, Lin W, Xu W, Jin J, Wang Y, Zhang G, Zhang X, Ma Z, Dong Y, Ma Q et al (2020) Tropospheric ozone assessment report: Long-term changes of regional ozone in china: implications for human health and ecosystem impacts. Elem Sci Anth 7:13. https:\/\/doi.org\/10.1525\/elementa.409","journal-title":"Elem Sci Anth"},{"issue":"1","key":"631_CR39","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1525\/elementa.265","volume":"6","author":"PJ Young","year":"2018","unstructured":"Young PJ, Naik V, Fiore AM, Gaudel A, Guo J, Lin MY, Neu JL, Parrish DD, Rieder HE, Schnell JL et al (2018) Tropospheric ozone assessment report: Assessment of global-scale model performance for global and regional ozone distributions, variability, and trends. Elem Sci Anth 6(1):10. https:\/\/doi.org\/10.1525\/elementa.265","journal-title":"Elem Sci Anth"}],"container-title":["Earth Science Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12145-021-00631-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12145-021-00631-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12145-021-00631-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,30]],"date-time":"2022-12-30T03:17:24Z","timestamp":1672370244000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12145-021-00631-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,7]]},"references-count":39,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,9]]}},"alternative-id":["631"],"URL":"https:\/\/doi.org\/10.1007\/s12145-021-00631-4","relation":{},"ISSN":["1865-0473","1865-0481"],"issn-type":[{"type":"print","value":"1865-0473"},{"type":"electronic","value":"1865-0481"}],"subject":[],"published":{"date-parts":[[2021,6,7]]},"assertion":[{"value":"27 August 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 May 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 June 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}