{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:14:46Z","timestamp":1750220086518,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":49,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,7,6]],"date-time":"2022-07-06T00:00:00Z","timestamp":1657065600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,7,6]]},"DOI":"10.1145\/3538712.3538715","type":"proceedings-article","created":{"date-parts":[[2022,8,23]],"date-time":"2022-08-23T10:14:41Z","timestamp":1661249681000},"page":"1-12","source":"Crossref","is-referenced-by-count":2,"title":["Northlight: Declarative and Optimized Analysis of Atmospheric Datasets in SparkSQL"],"prefix":"10.1145","author":[{"given":"Justus","family":"Henneberg","sequence":"first","affiliation":[{"name":"Johannes Gutenberg-University, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Felix","family":"Schuhknecht","sequence":"additional","affiliation":[{"name":"Johannes Gutenberg-University, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Philipp","family":"Reutter","sequence":"additional","affiliation":[{"name":"Johannes Gutenberg-University, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nils","family":"Brast","sequence":"additional","affiliation":[{"name":"Johannes Gutenberg-University, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Spichtinger","sequence":"additional","affiliation":[{"name":"Johannes Gutenberg-University, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,8,23]]},"reference":[{"volume-title":"Encyclopedia of GIS","key":"e_1_3_2_1_1_1","unstructured":"2017. Oracle Spatial GeoRaster . In Encyclopedia of GIS , Shashi Shekhar, Hui Xiong, and Xun Zhou (Eds.). Springer , 1522. https:\/\/doi.org\/10.1007\/978-3-319-17885-1_100917 2017. Oracle Spatial GeoRaster. In Encyclopedia of GIS, Shashi Shekhar, Hui Xiong, and Xun Zhou (Eds.). Springer, 1522. https:\/\/doi.org\/10.1007\/978-3-319-17885-1_100917"},{"key":"e_1_3_2_1_2_1","unstructured":"2021. ClimateSpark Codebase. https:\/\/github.com\/feihugis\/ClimateSpark  2021. ClimateSpark Codebase. https:\/\/github.com\/feihugis\/ClimateSpark"},{"key":"e_1_3_2_1_3_1","unstructured":"[\n  3\n  ]  2022. https:\/\/www.unidata.ucar.edu\/software\/netcdf-java\/  [3] 2022. https:\/\/www.unidata.ucar.edu\/software\/netcdf-java\/"},{"key":"e_1_3_2_1_4_1","unstructured":"[\n  4\n  ]  2022. https:\/\/unidata.github.io\/netcdf4-python\/  [4] 2022. https:\/\/unidata.github.io\/netcdf4-python\/"},{"key":"e_1_3_2_1_5_1","unstructured":"[\n  5\n  ]  2022. https:\/\/docs.h5py.org\/en\/latest\/vds.html  [5] 2022. https:\/\/docs.h5py.org\/en\/latest\/vds.html"},{"key":"e_1_3_2_1_6_1","unstructured":"2022. Alliance for High Performance Computing in Rhineland Palatinate. www.ahrp.info  2022. Alliance for High Performance Computing in Rhineland Palatinate. www.ahrp.info"},{"key":"e_1_3_2_1_7_1","unstructured":"2022. Iris. https:\/\/scitools-iris.readthedocs.io\/en\/latest\/  2022. Iris. https:\/\/scitools-iris.readthedocs.io\/en\/latest\/"},{"key":"e_1_3_2_1_8_1","unstructured":"2022. Lustre File System. https:\/\/www.lustre.org  2022. Lustre File System. https:\/\/www.lustre.org"},{"key":"e_1_3_2_1_9_1","unstructured":"Apache Software Foundation. 2022. Apache Spark - Unified Analytics Engine for Big Data. http:\/\/spark.apache.org\/  Apache Software Foundation. 2022. Apache Spark - Unified Analytics Engine for Big Data. http:\/\/spark.apache.org\/"},{"key":"e_1_3_2_1_10_1","unstructured":"Apache Software Foundation. 2022. QueryPlanConstraints.scala. https:\/\/github.com\/apache\/spark\/blob\/master\/sql\/catalyst\/src\/main\/scala\/org\/apache\/spark\/sql\/catalyst\/plans\/logical\/QueryPlanConstraints.scala  Apache Software Foundation. 2022. QueryPlanConstraints.scala. https:\/\/github.com\/apache\/spark\/blob\/master\/sql\/catalyst\/src\/main\/scala\/org\/apache\/spark\/sql\/catalyst\/plans\/logical\/QueryPlanConstraints.scala"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742797"},{"key":"e_1_3_2_1_12_1","unstructured":"Michael Armbrust Wenchen Fan Reynold Xin and Matei Zaharia. 2022. Introducing Apache Spark Datasets - The Databricks Blog. https:\/\/databricks.com\/blog\/2016\/01\/04\/introducing-apache-spark-datasets.html  Michael Armbrust Wenchen Fan Reynold Xin and Matei Zaharia. 2022. Introducing Apache Spark Datasets - The Databricks Blog. https:\/\/databricks.com\/blog\/2016\/01\/04\/introducing-apache-spark-datasets.html"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/331697.331732"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-020-00399-2"},{"key":"e_1_3_2_1_15_1","volume-title":"Second Biennial Conference on Innovative Data Systems Research, CIDR 2005, Asilomar, CA, USA, January 4-7, 2005, Online Proceedings. www.cidrdb.org, 225\u2013237","author":"Boncz A.","year":"2005","unstructured":"Peter\u00a0 A. Boncz , Marcin Zukowski , and Niels Nes . 2005 . MonetDB\/X100: Hyper-Pipelining Query Execution . In Second Biennial Conference on Innovative Data Systems Research, CIDR 2005, Asilomar, CA, USA, January 4-7, 2005, Online Proceedings. www.cidrdb.org, 225\u2013237 . http:\/\/cidrdb.org\/cidr2005\/papers\/P19.pdf Peter\u00a0A. Boncz, Marcin Zukowski, and Niels Nes. 2005. MonetDB\/X100: Hyper-Pipelining Query Execution. In Second Biennial Conference on Innovative Data Systems Research, CIDR 2005, Asilomar, CA, USA, January 4-7, 2005, Online Proceedings. www.cidrdb.org, 225\u2013237. http:\/\/cidrdb.org\/cidr2005\/papers\/P19.pdf"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687553.1687584"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3078597.3078599"},{"key":"e_1_3_2_1_18_1","unstructured":"European Centre for Medium-Range Weather Forecasts. 2022. ERA5. https:\/\/www.ecmwf.int\/en\/forecasts\/datasets\/reanalysis-datasets\/era5  European Centre for Medium-Range Weather Forecasts. 2022. ERA5. https:\/\/www.ecmwf.int\/en\/forecasts\/datasets\/reanalysis-datasets\/era5"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.5194\/acp-21-87-2021"},{"volume-title":"Encyclopedia of Database Systems","author":"Gupta Amarnath","key":"e_1_3_2_1_20_1","unstructured":"Amarnath Gupta . 2018. Data Types in Scientific Data Management . In Encyclopedia of Database Systems , Second Edition, Ling Liu and M.\u00a0Tamer \u00d6zsu (Eds.). Springer . https:\/\/doi.org\/10.1007\/978-1-4614-8265-9_1277 Amarnath Gupta. 2018. Data Types in Scientific Data Management. In Encyclopedia of Database Systems, Second Edition, Ling Liu and M.\u00a0Tamer \u00d6zsu (Eds.). Springer. https:\/\/doi.org\/10.1007\/978-1-4614-8265-9_1277"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/qj.3803"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.5334\/jors.148"},{"key":"e_1_3_2_1_23_1","volume-title":"ClimateSpark: An in-memory distributed computing framework for big climate data analytics. Computers & Geosciences 115","author":"Fei Hu","year":"2018","unstructured":"Fei Hu 2018. ClimateSpark: An in-memory distributed computing framework for big climate data analytics. Computers & Geosciences 115 ( 2018 ). https:\/\/doi.org\/10.1016\/j.cageo.2018.03.011 Fei Hu 2018. ClimateSpark: An in-memory distributed computing framework for big climate data analytics. Computers & Geosciences 115 (2018). https:\/\/doi.org\/10.1016\/j.cageo.2018.03.011"},{"key":"e_1_3_2_1_24_1","first-page":"9","article-title":"Experiences in Exascale Scientific Data Management","volume":"43","author":"Lassnig Mario","year":"2020","unstructured":"Mario Lassnig , Martin Barisits , and Dimitrios Christidis . 2020 . Experiences in Exascale Scientific Data Management . IEEE Data Eng. Bull. 43 , 1 (2020), 9 \u2013 22 . http:\/\/sites.computer.org\/debull\/A20mar\/p9.pdf Mario Lassnig, Martin Barisits, and Dimitrios Christidis. 2020. Experiences in Exascale Scientific Data Management. IEEE Data Eng. Bull. 43, 1 (2020), 9\u201322. http:\/\/sites.computer.org\/debull\/A20mar\/p9.pdf","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_3_2_1_25_1","unstructured":"Jialin Liu 2016. H5Spark: Bridging the I\/O Gap between Spark and Scientific Data Formats on HPC Systems.  Jialin Liu 2016. H5Spark: Bridging the I\/O Gap between Spark and Scientific Data Formats on HPC Systems."},{"key":"e_1_3_2_1_26_1","unstructured":"Johannes Gutenberg\u00a0University Mainz. 2022. Supercomputer Mogon Johannes Gutenberg University Mainz. hpc.uni-mainz.de  Johannes Gutenberg\u00a0University Mainz. 2022. Supercomputer Mogon Johannes Gutenberg University Mainz. hpc.uni-mainz.de"},{"key":"e_1_3_2_1_27_1","volume-title":"Research on Scientific Data Management in Big Data Era. In CSAE 2020: The 4th International Conference on Computer Science and Application Engineering","author":"Man Rui","year":"2020","unstructured":"Rui Man , Guomin Zhou , and Jingchao Fan . 2020 . Research on Scientific Data Management in Big Data Era. In CSAE 2020: The 4th International Conference on Computer Science and Application Engineering , Sanya, China , October 20-22, 2020, Ali Emrouznejadand Jui-Sheng\u00a0Rayson Chou (Eds.). ACM, 32:1\u201332:6. https:\/\/doi.org\/10.1145\/3424978.3425010 Rui Man, Guomin Zhou, and Jingchao Fan. 2020. Research on Scientific Data Management in Big Data Era. In CSAE 2020: The 4th International Conference on Computer Science and Application Engineering, Sanya, China, October 20-22, 2020, Ali Emrouznejadand Jui-Sheng\u00a0Rayson Chou (Eds.). ACM, 32:1\u201332:6. https:\/\/doi.org\/10.1145\/3424978.3425010"},{"key":"e_1_3_2_1_28_1","volume-title":"CF Conventions Home Page. http:\/\/cfconventions.org\/","author":"Harris Matthew","year":"2022","unstructured":"Matthew Harris . 2022 . CF Conventions Home Page. http:\/\/cfconventions.org\/ Matthew Harris. 2022. CF Conventions Home Page. http:\/\/cfconventions.org\/"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1256\/qj.04.94"},{"key":"e_1_3_2_1_30_1","unstructured":"NASA. 2022. Panoply. https:\/\/www.giss.nasa.gov\/tools\/panoply\/  NASA. 2022. Panoply. https:\/\/www.giss.nasa.gov\/tools\/panoply\/"},{"key":"e_1_3_2_1_31_1","unstructured":"NASA Jet Propulsion Laboratory. 2022. SciSpark. https:\/\/github.com\/SciSpark\/SciSpark\/blob\/master\/src\/main\/java\/org\/dia\/HDFSRandomAccessFile.java  NASA Jet Propulsion Laboratory. 2022. SciSpark. https:\/\/github.com\/SciSpark\/SciSpark\/blob\/master\/src\/main\/java\/org\/dia\/HDFSRandomAccessFile.java"},{"key":"e_1_3_2_1_32_1","unstructured":"Oracle. 2022. Raster Algebra Language. https:\/\/www.oracle.com\/a\/tech\/docs\/georaster-2021.pdf  Oracle. 2022. Raster Algebra Language. https:\/\/www.oracle.com\/a\/tech\/docs\/georaster-2021.pdf"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2015.7363983"},{"key":"e_1_3_2_1_34_1","unstructured":"The pandas\u00a0development team. 2022. pandas-dev\/pandas: Pandas. https:\/\/pandas.pydata.org\/  The pandas\u00a0development team. 2022. pandas-dev\/pandas: Pandas. https:\/\/pandas.pydata.org\/"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.5194\/acp-20-787-2020"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.5194\/acp-15-10939-2015"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2017.8258301"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2010.5496972"},{"key":"e_1_3_2_1_39_1","volume-title":"Fourth Biennial Conference on Innovative Data Systems Research, CIDR","author":"Michael Stonebraker","year":"2009","unstructured":"Michael Stonebraker 2009. Requirements for Science Data Bases and SciDB . In Fourth Biennial Conference on Innovative Data Systems Research, CIDR 2009 , Asilomar, CA , USA, January 4-7, 2009, Online Proceedings . www.cidrdb.org. http:\/\/www-db.cs.wisc.edu\/cidr\/cidr2009\/Paper_26.pdf Michael Stonebraker 2009. Requirements for Science Data Bases and SciDB. In Fourth Biennial Conference on Innovative Data Systems Research, CIDR 2009, Asilomar, CA, USA, January 4-7, 2009, Online Proceedings. www.cidrdb.org. http:\/\/www-db.cs.wisc.edu\/cidr\/cidr2009\/Paper_26.pdf"},{"key":"e_1_3_2_1_40_1","unstructured":"The HDF Group. 2022. Hierarchical data format version 5. http:\/\/www.hdfgroup.org\/HDF5  The HDF Group. 2022. Hierarchical data format version 5. http:\/\/www.hdfgroup.org\/HDF5"},{"key":"e_1_3_2_1_41_1","unstructured":"Unidata. 2022. A Convention for Coordinates: Coordinate Variables. https:\/\/www.unidata.ucar.edu\/software\/netcdf\/workshops\/2010\/datamodels\/NcCVars.html  Unidata. 2022. A Convention for Coordinates: Coordinate Variables. https:\/\/www.unidata.ucar.edu\/software\/netcdf\/workshops\/2010\/datamodels\/NcCVars.html"},{"key":"e_1_3_2_1_42_1","unstructured":"University Corporation for Atmospheric Research. 2022. Network Common Data Form (NetCDF). https:\/\/www.unidata.ucar.edu\/software\/netcdf\/  University Corporation for Atmospheric Research. 2022. Network Common Data Form (NetCDF). https:\/\/www.unidata.ucar.edu\/software\/netcdf\/"},{"key":"e_1_3_2_1_43_1","unstructured":"University Corporation for Atmospheric Research. 2022. Unidata\u2019s Common Data Model Version 4. https:\/\/www.unidata.ucar.edu\/software\/netcdf-java\/v4.6\/CDM\/index.html  University Corporation for Atmospheric Research. 2022. Unidata\u2019s Common Data Model Version 4. https:\/\/www.unidata.ucar.edu\/software\/netcdf-java\/v4.6\/CDM\/index.html"},{"key":"e_1_3_2_1_44_1","volume-title":"The airborne experiment on natural cirrus and contrail cirrus with the high-altitude long-range research aircraft HALO. Bulletin of the American Meteorological Society","author":"Voigt Christiane","year":"2016","unstructured":"Christiane Voigt , Ulrich Schumann , Andreas Minikin , Ahmed Abdelmonem , Armin Afchine , 2016. ML-CIRRUS : The airborne experiment on natural cirrus and contrail cirrus with the high-altitude long-range research aircraft HALO. Bulletin of the American Meteorological Society ( 2016 ), BAMS\u2013D\u201315\u201300213.1. https:\/\/doi.org\/10.1175\/BAMS-D-15-00213.1 12.01.02; LK 01. Christiane Voigt, Ulrich Schumann, Andreas Minikin, Ahmed Abdelmonem, Armin Afchine, 2016. ML-CIRRUS : The airborne experiment on natural cirrus and contrail cirrus with the high-altitude long-range research aircraft HALO. Bulletin of the American Meteorological Society (2016), BAMS\u2013D\u201315\u201300213.1. https:\/\/doi.org\/10.1175\/BAMS-D-15-00213.1 12.01.02; LK 01."},{"key":"e_1_3_2_1_45_1","volume-title":"SparkArray: An Array-Based Scientific Data Management System Built on Apache Spark. In IEEE International Conference on Networking, Architecture and Storage (NAS)","author":"Wang Wenjuan","year":"2016","unstructured":"Wenjuan Wang , Taoying Liu , Dixin Tang , Hong Liu , Wei Li , and Rubao Lee . 2016 . SparkArray: An Array-Based Scientific Data Management System Built on Apache Spark. In IEEE International Conference on Networking, Architecture and Storage (NAS) , Long Beach, CA, USA , August 8-10, 2016. IEEE Computer Society, 1\u201310. https:\/\/doi.org\/10.1109\/NAS.2016.7549422 Wenjuan Wang, Taoying Liu, Dixin Tang, Hong Liu, Wei Li, and Rubao Lee. 2016. SparkArray: An Array-Based Scientific Data Management System Built on Apache Spark. In IEEE International Conference on Networking, Architecture and Storage (NAS), Long Beach, CA, USA, August 8-10, 2016. IEEE Computer Society, 1\u201310. https:\/\/doi.org\/10.1109\/NAS.2016.7549422"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/CCGrid.2013.9"},{"key":"e_1_3_2_1_47_1","volume-title":"Shark: SQL and Rich Analytics at Scale. CoRR abs\/1211.6176(2012). arxiv:1211.6176http:\/\/arxiv.org\/abs\/1211.6176","author":"Xin Reynold","year":"2012","unstructured":"Reynold Xin , Josh Rosen , Matei Zaharia , Michael\u00a0 J. Franklin , Scott Shenker , and Ion Stoica . 2012 . Shark: SQL and Rich Analytics at Scale. CoRR abs\/1211.6176(2012). arxiv:1211.6176http:\/\/arxiv.org\/abs\/1211.6176 Reynold Xin, Josh Rosen, Matei Zaharia, Michael\u00a0J. Franklin, Scott Shenker, and Ion Stoica. 2012. Shark: SQL and Rich Analytics at Scale. CoRR abs\/1211.6176(2012). arxiv:1211.6176http:\/\/arxiv.org\/abs\/1211.6176"},{"volume-title":"Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12). 15\u201328.","author":"Matei Zaharia","key":"e_1_3_2_1_49_1","unstructured":"Matei Zaharia 2012. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing . In Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12). 15\u201328. Matei Zaharia 2012. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In Presented as part of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12). 15\u201328."},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463684"}],"event":{"name":"SSDBM 2022: 34th International Conference on Scientific and Statistical Database Management","acronym":"SSDBM 2022","location":"Copenhagen Denmark"},"container-title":["34th International Conference on Scientific and Statistical Database Management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3538712.3538715","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3538712.3538715","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:09:39Z","timestamp":1750183779000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3538712.3538715"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,6]]},"references-count":49,"alternative-id":["10.1145\/3538712.3538715","10.1145\/3538712"],"URL":"https:\/\/doi.org\/10.1145\/3538712.3538715","relation":{},"subject":[],"published":{"date-parts":[[2022,7,6]]}}}