{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T05:06:33Z","timestamp":1750309593303,"version":"3.41.0"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2025,4,28]],"date-time":"2025-04-28T00:00:00Z","timestamp":1745798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGMOD Rec."],"published-print":{"date-parts":[[2025,4,28]]},"abstract":"<jats:p>Data files serve as a vital resource for all data-driven applications. Among these, comma-separated value (CSV) files are particularly popular with users and businesses due to their flexible standard. However, also due to this loose standard, the data in these files are often indeed ''raw'', fraught with many types of structural inconsistencies that hinder seamless ingestion into a data system. We say that rows in CSV files with such structural inconsistencies are ill-formed.<\/jats:p>\n          <jats:p>Traditionally, data practitioners write custom code to repair the structure and format of ill-formed rows, even before they can leverage data cleaning tools and libraries, which typically assume that data are already properly loaded. Writing such code and configuring loading scripts is tedious, time-consuming, and requires both expertise and frequent human intervention.<\/jats:p>\n          <jats:p>To address these challenges, we present Tasheeh - a system that automatically detects ill-formed rows containing data and then standardizes their structure into a uniform format based on the structure of well-formed rows. By automating these essential steps, our system frees up valuable time and resources, enabling practitioners to focus on downstream stages of the data processing pipeline.<\/jats:p>","DOI":"10.1145\/3733620.3733639","type":"journal-article","created":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T16:20:07Z","timestamp":1745943607000},"page":"90-99","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Repairing Raw Data Files with TASHEEH"],"prefix":"10.1145","volume":"54","author":[{"given":"Mazhar","family":"Hameed","sequence":"first","affiliation":[{"name":"Gisma University of Applied Sciences Potsdam, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gerardo","family":"Vitagliano","sequence":"additional","affiliation":[{"name":"MIT CSAIL Cambridge, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fabian","family":"Panse","sequence":"additional","affiliation":[{"name":"University of Augsburg Augsburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Felix","family":"Naumann","sequence":"additional","affiliation":[{"name":"Hasso Plattner Institute University of Potsdam, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,4,29]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274856.3274879"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536336.2536343"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-67731-2_39"},{"issue":"12","key":"e_1_2_1_4_1","first-page":"2075","article-title":"pattern-based table discovery in csv files","volume":"13","author":"Christodoulakis Christina","year":"2020","unstructured":"Christina Christodoulakis, Eric B Munson, Moshe Gabel, Angela Demke Brown, and Ren\u00b4ee J Miller. Pytheas: pattern-based table discovery in csv files. PVLDB, 13(12):2075--2089, 2020.","journal-title":"PVLDB"},{"key":"e_1_2_1_5_1","first-page":"1713","volume-title":"Proceedings of the International Conference on Management of Data (SIGMOD)","author":"Chu Xu","year":"2015","unstructured":"Xu Chu, Yeye He, Kaushik Chakrabarti, and Kris Ganjam. Tegra: Table extraction by global record alignment. In Proceedings of the International Conference on Management of Data (SIGMOD), pages 1713--1728, 2015."},{"key":"e_1_2_1_6_1","first-page":"1247","volume-title":"Proceedings of the International Conference on Management of Data (SIGMOD)","author":"Chu Xu","year":"2015","unstructured":"Xu Chu, John Morcos, Ihab F Ilyas, Mourad Ouzzani, Paolo Papotti, Nan Tang, and Yin Ye. Katara: A data cleaning system powered by knowledge bases and crowdsourcing. In Proceedings of the International Conference on Management of Data (SIGMOD), pages 1247--1261, 2015."},{"key":"e_1_2_1_7_1","first-page":"541","volume-title":"Proceedings of the International Conference on Management of Data (SIGMOD)","author":"Dallachiesa Michele","year":"2013","unstructured":"Michele Dallachiesa, Amr Ebaid, Ahmed Eldawy, Ahmed Elmagarmid, Ihab F Ilyas, Mourad Ouzzani, and Nan Tang. NADEEF: a commodity data cleaning system. In Proceedings of the International Conference on Management of Data (SIGMOD), pages 541--552, 2013."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01386390"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3085504.3085520"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.330169"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687627.1687749"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3319898"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1373\/clinchem.2005.053850"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3444831.3444835"},{"key":"e_1_2_1_15_1","first-page":"143","volume-title":"Proceedings of the International Conference on Extending Database Technology (EDBT)","author":"Hameed Mazhar","year":"2022","unstructured":"Mazhar Hameed, Gerardo Vitagliano, Lan Jiang, and Felix Naumann. Suragh: Syntactic pattern matching to identify ill-formed records. In Proceedings of the International Conference on Extending Database Technology (EDBT), pages 143--154, 2022."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3583780.3614747"},{"key":"e_1_2_1_17_1","first-page":"426","volume-title":"Proceedings of the International Conference on Extending Database Technology (EDBT)","author":"Hameed Mazhar","year":"2024","unstructured":"Mazhar Hameed, Gerardo Vitagliano, Fabian Panse, and Felix Naumann. TASHEEH: Repairing row-structure in raw CSV files. In Proceedings of the International Conference on Extending Database Technology (EDBT), pages 426--439, 2024."},{"issue":"10","key":"e_1_2_1_18_1","first-page":"1165","article-title":"an extensible search engine for data transformations","volume":"11","author":"He Yeye","year":"2018","unstructured":"Yeye He, Xu Chu, Kris Ganjam, Yudian Zheng, Vivek Narasayya, and Surajit Chaudhuri. Transform-data-by-example (TDE) an extensible search engine for data transformations. PVLDB, 11(10):1165--1177, 2018.","journal-title":"PVLDB"},{"issue":"2","key":"e_1_2_1_19_1","first-page":"23","article-title":"Self-service data preparation: Research to practice","volume":"41","author":"Hellerstein Joseph M","year":"2018","unstructured":"Joseph M Hellerstein, Jeffrey Heer, and Sean Kandel. Self-service data preparation: Research to practice. IEEE Data Engineering Bulletin, 41(2):23--34, 2018.","journal-title":"IEEE Data Engineering Bulletin"},{"key":"e_1_2_1_20_1","volume-title":"https:\/\/tools.ietf.org\/html\/rfc4180","author":"RFC","year":"2005","unstructured":"RFC 4180. https:\/\/tools.ietf.org\/html\/rfc4180, 2005. (last accessed January 19th, 2025)."},{"key":"e_1_2_1_21_1","volume-title":"Airbyte: The open-source data integration platform. https:\/\/airbyte.com. (last accessed","author":"Airbyte Inc.","year":"2025","unstructured":"Airbyte Inc. Airbyte: The open-source data integration platform. https:\/\/airbyte.com. (last accessed January 19, 2025)."},{"key":"e_1_2_1_22_1","volume-title":"www.openrefine.org. (last accessed January 19th","author":"Google Inc. Openrefine.","year":"2025","unstructured":"Google Inc. Openrefine. www.openrefine.org. (last accessed January 19th, 2025)."},{"key":"e_1_2_1_23_1","volume-title":"Trifacta data engineering cloud (now alteryx designer cloud after acquisition). www.trifacta.com. (last accessed January 19th","author":"Trifacta Inc.","year":"2025","unstructured":"Trifacta Inc. Trifacta data engineering cloud (now alteryx designer cloud after acquisition). www.trifacta.com. (last accessed January 19th, 2025)."},{"key":"e_1_2_1_24_1","first-page":"193","volume-title":"Proceedings of the International Conference on Extending Database Technology (EDBT)","author":"Jiang Lan","year":"2021","unstructured":"Lan Jiang, Gerardo Vitagliano, and Felix Naumann. Structure detection in verbose csv files. In Proceedings of the International Conference on Extending Database Technology (EDBT), pages 193--204, 2021."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/DAS.2018.48"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-99701-8_4"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDAR.2019.00206"},{"issue":"12","key":"e_1_2_1_28_1","first-page":"948","article-title":"Interactive data cleaning for statistical modeling","volume":"9","author":"Krishnan Sanjay","year":"2016","unstructured":"Sanjay Krishnan, Jiannan Wang, Eugene Wu, Michael J Franklin, and Ken Goldberg. ActiveClean: Interactive data cleaning for statistical modeling. PVLDB, 9(12):948--959, 2016.","journal-title":"PVLDB"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/OBD.2016.18"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-61033-7_1"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415572"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/860435.860479"},{"key":"e_1_2_1_33_1","volume-title":"Forbes","author":"Press Gil","year":"2016","unstructured":"Gil Press. Cleaning data: Most time-consuming, least enjoyable data science task. Forbes, March 2016."},{"key":"e_1_2_1_34_1","first-page":"2100","volume-title":"Proceedings of the International Conference on Knowledge Discovery and Data Mining (SIGKDD)","author":"Qahtan Abdulhakim A","year":"2018","unstructured":"Abdulhakim A Qahtan, Ahmed Elmagarmid, Raul Castro Fernandez, Mourad Ouzzani, and Nan Tang. FAHES: A robust disguised missing values detector. In Proceedings of the International Conference on Knowledge Discovery and Data Mining (SIGKDD), pages 2100--2109, 2018."},{"key":"e_1_2_1_35_1","first-page":"1981","volume-title":"Proceedings of the International Conference on Management of Data (SIGMOD)","author":"Raasveldt Mark","year":"2019","unstructured":"Mark Raasveldt and Hannes M\u00a8uhleisen. DuckDB: an embeddable analytical database. In Proceedings of the International Conference on Management of Data (SIGMOD), pages 1981--1984, 2019."},{"key":"e_1_2_1_36_1","unstructured":"Jeff Reback jbrockmendel Wes McKinney Joris Van den Bossche Matthew Roeschke Tom Augspurger Simon Hawkins Phillip Cloud gfyoung Sinhrks Patrick Hoefler Adam Klein Terji Petersen Jeff Tratner Chang She William Ayd Shahar Naveh JHM Darbyshire Richard Shadrach Marc Garcia Jeremy Schendel Andy Hayden Daniel Saxton Marco Edward Gorelli Fangchen Li Torsten W\u00a8ortwein Matthew Zeitlin Vytautas Jancauskas Ali McMaster and Thomas Li. pandas-dev\/pandas: Pandas 1.4.3 June 2022."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137628.3137631"},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the Workshop on Linked Data on the Web (LDOW)","author":"Sekhavat Yoones A","year":"2014","unstructured":"Yoones A Sekhavat, Francesco Di Paolo, Denilson Barbosa, and Paolo Merialdo. Knowledge base augmentation using tabular data. In Proceedings of the Workshop on Linked Data on the Web (LDOW), 2014."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3329486.3329499"},{"issue":"5","key":"e_1_2_1_40_1","first-page":"616","article-title":"Massively parallel parsing of delimiter-separated raw data","volume":"13","author":"Stehle Elias","year":"2020","unstructured":"Elias Stehle and Hans-Arno Jacobsen. ParPaRaw: Massively parallel parsing of delimiter-separated raw data. PVLDB, 13(5):616--628, 2020.","journal-title":"PVLDB"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE48307.2020.00068"},{"key":"e_1_2_1_42_1","volume-title":"www.tableau.com. (last accessed January 19th","author":"Tableau Software LLC","year":"2025","unstructured":"LLC Tableau Software. Tableau. www.tableau.com. (last accessed January 19th, 2025)."},{"issue":"8","key":"e_1_2_1_43_1","first-page":"1254","article-title":"Relational pre-trained transformer is almost all you need towards democratizing data preparation","volume":"14","author":"Tang Nan","year":"2021","unstructured":"Nan Tang, Ju Fan, Fangyi Li, Jianhong Tu, Xiaoyong Du, Guoliang Li, Sam Madden, and Mourad Ouzzani. RPT: Relational pre-trained transformer is almost all you need towards democratizing data preparation. PVLDB, 14(8):1254--1261, 2021.","journal-title":"PVLDB"},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the Conference on Innovative Data Systems Research (CIDR)","author":"Terrizzano Ignacio G","year":"2015","unstructured":"Ignacio G Terrizzano, Peter M Schwarz, Mary Roth, and John E Colino. Data wrangling: The challenging journey from the wild to the lake. In Proceedings of the Conference on Innovative Data Systems Research (CIDR), 2015."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-019-00646-y"},{"issue":"8","key":"e_1_2_1_46_1","first-page":"1870","article-title":"A data loading benchmark","volume":"16","author":"Vitagliano Gerardo","year":"2023","unstructured":"Gerardo Vitagliano, Mazhar Hameed, Lan Jiang, Lucas Reisener, Eugene Wu, and Felix Naumann. Pollock: A data loading benchmark. PVLDB, 16(8):1870--1882, 2023.","journal-title":"PVLDB"},{"key":"e_1_2_1_47_1","volume-title":"The string-to-string correction problem. Journal of the ACM (JACM), 21(1):168--173","author":"Wagner Robert A","year":"1974","unstructured":"Robert A Wagner and Michael J Fischer. The string-to-string correction problem. Journal of the ACM (JACM), 21(1):168--173, 1974."}],"container-title":["ACM SIGMOD Record"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3733620.3733639","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3733620.3733639","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:56:55Z","timestamp":1750298215000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3733620.3733639"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,28]]},"references-count":47,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,4,28]]}},"alternative-id":["10.1145\/3733620.3733639"],"URL":"https:\/\/doi.org\/10.1145\/3733620.3733639","relation":{},"ISSN":["0163-5808"],"issn-type":[{"type":"print","value":"0163-5808"}],"subject":[],"published":{"date-parts":[[2025,4,28]]},"assertion":[{"value":"2025-04-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}