{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,2]],"date-time":"2026-08-02T23:47:35Z","timestamp":1785714455728,"version":"3.56.0"},"reference-count":79,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2023,6,13]],"date-time":"2023-06-13T00:00:00Z","timestamp":1686614400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2023,6,13]]},"abstract":"<jats:p>Data Scientists deal with a wide variety of file data formats and data representations. Probably the most difficult to handle are custom data formats that liberally define their own particular flat or nested structure with multiple custom delimiters, multi-line records, or undocumented semantics of attribute sequences, co-appearances, and repetitions. As a prerequisite for exploratory ML model training, data scientists need to map these data representations into regular frames or matrices. Unfortunately, existing tools and frameworks provide only limited support for aiding this process, which causes redundant manual efforts and unnecessary data quality issues. In this paper, we initiate work on automatic matrix and frame reader generation by example. A user provides a sample of raw text data and its mapped matrix or frame representation. Our GIO framework then first identifies the mapping rules from raw to structured data, and subsequently generates source code of an efficient, multi-threaded reader for reading full raw datasets of this format. In order to facilitate manual improvements, both the mapping rules, and generated reader can be modified as needed. Our experiments show that GIO is able to correctly identify the mapping rules for basic text formats like CSV, LibSVM, MatrixMarket; custom text formats from publishing, automotive, and health care; as well as various nested formats such as JSON and XML. Additionally, the automatically generated readers yield competitive performance compared to hand-coded readers and tuned libraries like RapidJSON.<\/jats:p>","DOI":"10.1145\/3589265","type":"journal-article","created":{"date-parts":[[2023,6,20]],"date-time":"2023-06-20T20:26:45Z","timestamp":1687292805000},"page":"1-26","source":"Crossref","is-referenced-by-count":4,"title":["GIO: Generating Efficient Matrix and Frame Readers for Custom Data Formats by Example"],"prefix":"10.1145","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3723-6191","authenticated-orcid":false,"given":"Saeed","family":"Fathollahzadeh","sequence":"first","affiliation":[{"name":"Graz University of Technology, Graz, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1344-3663","authenticated-orcid":false,"given":"Matthias","family":"Boehm","sequence":"additional","affiliation":[{"name":"Technische Universit\u00e4t Berlin, Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,6,20]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"2000. Auto-lead Data Format \/ ADF: An Industry Standard Data Format for the Export and Import of Automotive Customer Leads using XML. https:\/\/adfxml.info\/adf_spec.pdf"},{"key":"e_1_2_2_3_1","unstructured":"2022. Gson. https:\/\/github.com\/google\/gson\/"},{"key":"e_1_2_2_4_1","unstructured":"2022. HAPI object-oriented HL7 2.x parser for Java. https:\/\/hapifhir.github.io\/hapi-hl7v2\/"},{"key":"e_1_2_2_5_1","unstructured":"2022. Jackson. https:\/\/github.com\/FasterXML\/jackson\/"},{"key":"e_1_2_2_6_1","unstructured":"2022. RapidJSON. http:\/\/rapidjson.org\/"},{"key":"e_1_2_2_7_1","unstructured":"2022. Schema Guru. https:\/\/github.com\/snowplow\/schema-guru"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3054772"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","unstructured":"Ioannis Alagiannis Renata Borovica Miguel Branco Stratos Idreos and Anastasia Ailamaki. 2012. NoDB: Efficient Query Execution on Raw Data Files. In SIGMOD. 241--252. https:\/\/doi.org\/10.1145\/2213836.2213864","DOI":"10.1145\/2213836.2213864"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.14778\/2367502.2367543"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043652.2043656"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.14778\/3402755.3402784"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2008.4497409"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989338"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","unstructured":"Michael Armbrust Reynold S. Xin Cheng Lian Yin Huai Davies Liu Joseph K. Bradley Xiangrui Meng Tomer Kaftan Michael J. Franklin Ali Ghodsi and Matei Zaharia. 2015. Spark SQL: Relational Data Processing in Spark. In SIGMOD. 1383--1394. https:\/\/doi.org\/10.1145\/2723372.2742797","DOI":"10.1145\/2723372.2742797"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.5441\/002"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.14778\/3565838.3565852"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1066157.1066283"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.14778\/3157794.3157801"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","unstructured":"Mohamed-Amine Baazizi Dario Colazzo Giorgio Ghelli and Carlo Sartiani. 2017. Counting types for massive JSON datasets. In DBPL@VLDB Workshop. 1--12. https:\/\/doi.org\/10.1145\/3122831.3122837","DOI":"10.1145\/3122831.3122837"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-018-0532--7"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","unstructured":"Mohamed Amine Baazizi Dario Colazzo Giorgio Ghelli and Carlo Sartiani. 2019. Schemas and Types for JSON Data: From Theory to Practice. In SIGMOD. 2060--2063. https:\/\/doi.org\/10.1145\/3299869.3314032","DOI":"10.1145\/3299869.3314032"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.5441\/002"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457549"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/SPIRE.2000.878178"},{"key":"e_1_2_2_26_1","first-page":"695","article-title":"Generic Schema Matching","volume":"4","author":"Bernstein Philip A.","year":"2011","unstructured":"Philip A. Bernstein, Jayant Madhavan, and Erhard Rahm. 2011. Generic Schema Matching, Ten Years Later. PVLDB 4, 11 (2011), 695--701. http:\/\/www.vldb.org\/pvldb\/vol4\/p695-bernstein_madhavan_rahm.pdf","journal-title":"Ten Years Later. PVLDB"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","unstructured":"George Beskales Ihab F. Ilyas Lukasz Golab and Artur Galiullin. 2013. On the relative trust between inconsistent data and inaccurate constraints. In ICDE. 541--552. https:\/\/doi.org\/10.1109\/ICDE.2013.6544854","DOI":"10.1109\/ICDE.2013.6544854"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.14778\/3402755.3402761"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3196896"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","unstructured":"Spyros Blanas Kesheng Wu Surendra Byna Bin Dong and Arie Shoshani. 2014. Parallel data analysis directly on scientific file formats. In SIGMOD. 385--396. https:\/\/doi.org\/10.1145\/2588555.2612185","DOI":"10.1145\/2588555.2612185"},{"key":"e_1_2_2_31_1","volume-title":"CIDR","author":"Boehm Matthias","unstructured":"Matthias Boehm, Iulian Antonov, Sebastian Baunsgaard, Mark Dokter, Robert Ginth\u00f6r, Kevin Innerebner, Florijan Klezin, Stefanie N. Lindstaedt, Arnab Phani, Benjamin Rath, Berthold Reinwald, Shafaq Siddiqui, and Sebastian Benjamin Wrede. 2020. SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle. In CIDR. http:\/\/cidrdb.org\/cidr2020\/papers\/p22-boehm-cidr20.pdf"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.14778\/3229863.3229865"},{"key":"e_1_2_2_33_1","volume-title":"Ulrike Fischer, Dirk Habich, and Wolfgang Lehner.","author":"B\u00f6hm Matthias","year":"2011","unstructured":"Matthias B\u00f6hm, Benjamin Schlegel, Peter Benjamin Volk, Ulrike Fischer, Dirk Habich, and Wolfgang Lehner. 2011. Efficient In-Memory Indexing with Generalized Prefix Trees. In BTW. 227--246. https:\/\/dl.gi.de\/20.500.12116\/19581"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1961189.1961199"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","unstructured":"Yu Cheng and Florin Rusu. 2014. Parallel in-situ data processing with speculative loading. In SIGMOD. 1287--1298. https:\/\/doi.org\/10.1145\/2588555.2593673","DOI":"10.1145\/2588555.2593673"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687553.1687576"},{"key":"e_1_2_2_37_1","volume-title":"Ziawasch Abedjan, Sibo Wang, Michael Stonebraker, Ahmed K. Elmagarmid, Ihab F. Ilyas, Samuel Madden, Mourad Ouzzani, and Nan Tang.","author":"Deng Dong","year":"2017","unstructured":"Dong Deng, Raul Castro Fernandez, Ziawasch Abedjan, Sibo Wang, Michael Stonebraker, Ahmed K. Elmagarmid, Ihab F. Ilyas, Samuel Madden, Mourad Ouzzani, and Nan Tang. 2017. The Data Civilizer System. In CIDR. http:\/\/cidrdb.org\/cidr2017\/papers\/p44-deng-cidr17.pdf"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","unstructured":"Hong Hai Do and Erhard Rahm. 2002. COMA - A System for Flexible Combination of Schema Matching Approaches. In VLDB. 610--621. https:\/\/doi.org\/10.1016\/B978--155860869--6\/50060--3","DOI":"10.1016\/B978--155860869--6"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","unstructured":"Dominik Durner Viktor Leis and Thomas Neumann. 2021. JSON Tiles: Fast Analytics on Semi-Structured Data. In SIGMOD. 445--458. https:\/\/doi.org\/10.1145\/3448016.3452809","DOI":"10.1145\/3448016.3452809"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tcs.2004.10.033"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","unstructured":"Chang Ge Yinan Li Eric Eilebrecht Badrish Chandramouli and Donald Kossmann. 2019. Speculative distributed CSV data parsing for big data analytics. In SIGMOD. 883--899. https:\/\/doi.org\/10.1145\/3299869.3319898","DOI":"10.1145\/3299869.3319898"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3319898"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389739"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","unstructured":"Laura M. Haas Mauricio A. Hern\u00e1ndez Howard Ho Lucian Popa and Mary Roth. 2005. Clio grows up: from research prototype to industrial tool. In SIGMOD. 805--810. https:\/\/doi.org\/10.1145\/1066157.1066252","DOI":"10.1145\/1066157.1066252"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/375663.375767"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330993"},{"key":"e_1_2_2_47_1","unstructured":"Stratos Idreos Ioannis Alagiannis Ryan Johnson and Anastasia Ailamaki. 2011. Here are my Data Files. Here are my Queries. Where are my Results?. In CIDR. 57--68. http:\/\/cidrdb.org\/cidr2011\/Papers\/CIDR11_Paper7.pdf"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484838.2484876"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.14778\/3436905.3436926"},{"key":"e_1_2_2_50_1","unstructured":"Peter Kairouz Brendan McMahan and Virginia Smith. 2020. Federated Learning Tutorial. In NeurIPS. https:\/\/slideslive.com\/38935813\/federated-learning-tutorial"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.14778\/2994509.2994516"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732977.2732986"},{"key":"e_1_2_2_53_1","unstructured":"Meike Klettke Uta St\u00f6rl and Stefanie Scherzinger. 2015. Schema Extraction and Structural Outlier Detection for JSON-based NoSQL Data Stores. In BTW. 425--444. https:\/\/dl.gi.de\/20.500.12116\/2420"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","unstructured":"Phokion G Kolaitis. 2005. Schema mappings data exchange and metadata management. In PODS. 61--75. https:\/\/doi.org\/10.1145\/1065167.1065176","DOI":"10.1145\/1065167.1065176"},{"key":"e_1_2_2_55_1","volume-title":"Impala: A Modern, Open-Source SQL Engine for Hadoop. In CIDR","author":"Marcel Kornacker","year":"2015","unstructured":"Marcel Kornacker et al . 2015. Impala: A Modern, Open-Source SQL Engine for Hadoop. In CIDR. http:\/\/cidrdb.org\/cidr2015\/Papers\/CIDR15_Paper28.pdf"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-019-00578--5"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","unstructured":"Viktor Leis Alfons Kemper and Thomas Neumann. 2013. The adaptive radix tree: ARTful indexing for main-memory databases. In ICDE. 38--49. https:\/\/doi.org\/10.1109\/ICDE.2013.6544812","DOI":"10.1109\/ICDE.2013.6544812"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.14778\/3115404.3115416"},{"key":"e_1_2_2_59_1","doi-asserted-by":"crossref","unstructured":"Ericsson M. Garcia-Martin G. Camarillo. 2008. Extensible Markup Language (XML) Format Extension for Representing Copy Control Attributes in Resource Lists. RFC 5364. RFC Editor. https:\/\/datatracker.ietf.org\/doc\/html\/rfc5364","DOI":"10.17487\/rfc5364"},{"key":"e_1_2_2_60_1","unstructured":"Jayant Madhavan Philip A. Bernstein and Erhard Rahm. 2001. Generic Schema Matching with Cupid. In VLDB. 49--58. http:\/\/www.vldb.org\/conf\/2001\/P049.pdf"},{"key":"e_1_2_2_61_1","volume-title":"Hern\u00e1ndez","author":"Miller Ren\u00e9e J.","year":"2000","unstructured":"Ren\u00e9e J. Miller, Laura M. Haas, and Mauricio A. Hern\u00e1ndez. 2000. Schema Mapping as Query Discovery. In VLDB. 77--88. http:\/\/www.vldb.org\/conf\/2000\/P077.pdf"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/321479.321481"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.14778\/3436905.3436910"},{"key":"e_1_2_2_64_1","volume-title":"Janet Wiener, and Sudarashan Chawathe","author":"Nestorov Svetlozar","year":"1997","unstructured":"Svetlozar Nestorov, Jeffrey Ullman, Janet Wiener, and Sudarashan Chawathe. 1997. Representative objects: Concise representations of semistructured, hierarchical data. In ICDE. 79--90."},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.14778\/3236187.3236207"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.5441\/002"},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","unstructured":"Li Qian Michael J Cafarella and HV Jagadish. 2012. Sample-driven schema mapping. In SIGMOD. 73--84. https:\/\/doi.org\/10.1145\/2213836.2213846","DOI":"10.1145\/2213836.2213846"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1007\/s007780100057"},{"key":"e_1_2_2_69_1","doi-asserted-by":"crossref","unstructured":"Y. Shafranovich. 2005. Common Format and MIME Type for Comma-Separated Values (CSV) Files. RFC 4180. RFC Editor. https:\/\/www.rfc-editor.org\/rfc\/rfc4180","DOI":"10.17487\/rfc4180"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","unstructured":"Vraj Shah Jonathan Lacanlale Premanand Kumar Kevin Yang and Arun Kumar. 2021. Towards Benchmarking Feature Type Inference for AutoML Platforms. In SIGMOD. 1584--1596. https:\/\/doi.org\/10.1145\/3448016.3457274","DOI":"10.1145\/3448016.3457274"},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.14778\/3377369.3377372"},{"key":"e_1_2_2_72_1","doi-asserted-by":"crossref","unstructured":"Ed. T. Bray. 2017. The JavaScript Object Notation (JSON) Data Interchange Format. RFC 8259. RFC Editor. https:\/\/datatracker.ietf.org\/doc\/html\/rfc8259","DOI":"10.17487\/RFC8259"},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","unstructured":"Jie Tang Jing Zhang Limin Yao Juanzi Li Li Zhang and Zhong Su. 2008. Arnetminer: extraction and mining of academic social networks. In SIGKDD. 990--998. https:\/\/doi.org\/10.1145\/1401890.1402008","DOI":"10.1145\/1401890.1402008"},{"key":"e_1_2_2_74_1","volume-title":"Janino: A super-small, super-fast Java Compiler. https:\/\/janino-compiler.github.io\/janino\/2014-02--18_SWM-JAK.pdf","author":"Unkrieg Arno","year":"2014","unstructured":"Arno Unkrieg. 2014. Janino: A super-small, super-fast Java Compiler. https:\/\/janino-compiler.github.io\/janino\/2014-02--18_SWM-JAK.pdf"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1007\/3--540--46439--5_21"},{"key":"e_1_2_2_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/3187009.3177735"},{"key":"e_1_2_2_77_1","doi-asserted-by":"publisher","unstructured":"Ling-Ling Yan Ren\u00e9e J. Miller Laura M. Haas and Ronald Fagin. 2001. Data-Driven Understanding and Refinement of Schema Mappings. In SIGMOD. 485--496. https:\/\/doi.org\/10.1145\/375663.375729","DOI":"10.1145\/375663.375729"},{"key":"e_1_2_2_78_1","unstructured":"Matei Zaharia Mosharaf Chowdhury Tathagata Das Ankur Dave Justin Ma Murphy McCauly Michael J. Franklin Scott Shenker and Ion Stoica. 2012. Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. In NSDI. 15--28. https:\/\/www.usenix.org\/conference\/nsdi12\/technical-sessions\/presentation\/zaharia"},{"key":"e_1_2_2_79_1","volume-title":"Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics. In CIDR","author":"Zaharia Matei","year":"2021","unstructured":"Matei Zaharia, Ali Ghodsi, Reynold Xin, and Michael Armbrust. 2021. Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics. In CIDR. http:\/\/cidrdb.org\/cidr2021\/papers\/cidr2021_paper17.pdf"},{"key":"e_1_2_2_80_1","doi-asserted-by":"publisher","DOI":"10.14778\/3407790.3407793"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3589265","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3589265","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:48:54Z","timestamp":1750182534000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3589265"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,13]]},"references-count":79,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,6,13]]}},"alternative-id":["10.1145\/3589265"],"URL":"https:\/\/doi.org\/10.1145\/3589265","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,13]]}}}