{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T16:04:08Z","timestamp":1791648248242,"version":"4.3.4"},"reference-count":84,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2023,6,2]],"date-time":"2023-06-02T00:00:00Z","timestamp":1685664000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,6,2]],"date-time":"2023-06-02T00:00:00Z","timestamp":1685664000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/M025268\/1"],"award-info":[{"award-number":["EP\/M025268\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["780787"],"award-info":[{"award-number":["780787"]}],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["SN COMPUT. SCI."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Data analysis often uses data sets that were collected for different purposes. Indeed, new insights are often obtained by combining data sets that were produced independently of each other, for example by combining data from outside an organization with internal data resources. As a result, there is a need to discover, clean, integrate and restructure data into a form that is suitable for an intended analysis. Data preparation, also known as data wrangling, is the process by which data are transformed from its existing representation into a form that is suitable for analysis. In this paper, we review the state-of-the-art in data preparation, by: (i) describing functionalities that are central to data preparation pipelines, specifically profiling, matching, mapping, format transformation and data repair; and (ii) presenting how these capabilities surface in different approaches to data preparation, that involve programming, writing workflows, interacting with individual data sets as tables, and automating aspects of the process. These functionalities and approaches are illustrated with reference to a running example that combines open government data with web extracted real estate data.<\/jats:p>","DOI":"10.1007\/s42979-023-01828-8","type":"journal-article","created":{"date-parts":[[2023,6,2]],"date-time":"2023-06-02T13:05:11Z","timestamp":1685711111000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["Data Preparation: A Technological Perspective and Review"],"prefix":"10.1007","volume":"4","author":[{"given":"Alvaro A. A.","family":"Fernandes","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Koehler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nikolaos","family":"Konstantinou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pavel","family":"Pankin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2008-6617","authenticated-orcid":false,"given":"Norman W.","family":"Paton","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rizos","family":"Sakellariou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,6,2]]},"reference":[{"issue":"4","key":"1828_CR1","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1007\/s00778-015-0389-y","volume":"24","author":"Z Abedjan","year":"2015","unstructured":"Abedjan Z, Golab L, Naumann F. Profiling relational data: a survey. VLDB J. 2015;24(4):557\u201381.","journal-title":"VLDB J"},{"key":"1828_CR2","doi-asserted-by":"publisher","unstructured":"Abedjan Z, Morcos J, Ilyas IF, et\u00a0al. Dataxformer: a robust transformation discovery system. In: 32nd IEEE International Conference on Data Engineering, ICDE, 2016; pp. 1134\u20131145, https:\/\/doi.org\/10.1109\/ICDE.2016.7498319","DOI":"10.1109\/ICDE.2016.7498319"},{"issue":"6","key":"1828_CR3","doi-asserted-by":"publisher","first-page":"777","DOI":"10.1007\/s00778-017-0477-2","volume":"26","author":"SMF Ali","year":"2017","unstructured":"Ali SMF, Wrembel R. From conceptual design to performance optimization of ETL workflows: current state of research and open problems. VLDB J. 2017;26(6):777\u2013801. https:\/\/doi.org\/10.1007\/s00778-017-0477-2.","journal-title":"VLDB J"},{"key":"1828_CR4","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781139060158","volume-title":"Foundations of data exchange","author":"M Arenas","year":"2014","unstructured":"Arenas M, Barcel\u00f3 P, Libkin L, et al. Foundations of data exchange. Cambridge: Cambridge University Press; 2014."},{"key":"1828_CR5","doi-asserted-by":"publisher","unstructured":"Aumueller D, Do HH, Massmann S, et\u00a0al. Schema and Ontology Matching with COMA++. In: Proceedins of 2005 ACM SIGMOD International Conference on Management of Data. ACM, 2005; 906\u20138. https:\/\/doi.org\/10.1145\/1066157.1066283","DOI":"10.1145\/1066157.1066283"},{"key":"1828_CR6","unstructured":"Azarmi B. Talend for Big Data. Packt Publishing 2014."},{"key":"1828_CR7","doi-asserted-by":"publisher","unstructured":"Bahri M, Salutari F, Putina A, et\u00a0al. AutoML: state of the art with a focus on anomaly detection, challenges, and research directions. Int J Data Sci Anal. 2022. https:\/\/doi.org\/10.1007\/s41060-022-00309-0","DOI":"10.1007\/s41060-022-00309-0"},{"key":"1828_CR8","doi-asserted-by":"publisher","unstructured":"Bellahsene Z, Bonifati A, Rahm E. Schema Matching and Mapping. 2011. https:\/\/doi.org\/10.1007\/978-3-642-16518-4.","DOI":"10.1007\/978-3-642-16518-4"},{"key":"1828_CR9","doi-asserted-by":"publisher","unstructured":"Bertossi LE, Geerts F. Data quality and explainable AI. ACM J Data Inf Qual 2020;12(2):11:1\u201311:9. https:\/\/doi.org\/10.1145\/3386687","DOI":"10.1145\/3386687"},{"key":"1828_CR10","doi-asserted-by":"crossref","unstructured":"Beskales G, Ilyas IF, Golab L, et\u00a0al. On the relative trust between inconsistent data and inaccurate constraints. In: 29th IEEE International Conference on Data Engineering, ICDE, 2013; pp. 541\u2013552.","DOI":"10.1109\/ICDE.2013.6544854"},{"key":"1828_CR11","doi-asserted-by":"publisher","unstructured":"Bogatu A, Fernandes AAA, Paton NW, et\u00a0al. Synthedit: Format transformations by example using edit operations. In: 22nd International Conference on Extending Database Technology. OpenProceedings.org, 2019a:714\u2013717. https:\/\/doi.org\/10.5441\/002\/edbt.2019.94","DOI":"10.5441\/002\/edbt.2019.94"},{"issue":"7","key":"1828_CR12","doi-asserted-by":"publisher","first-page":"1044","DOI":"10.1093\/comjnl\/bxy118","volume":"62","author":"A Bogatu","year":"2019","unstructured":"Bogatu A, Paton NW, Fernandes AAA, et al. Towards automatic data format transformations: data wrangling at scale. Comput J. 2019;62(7):1044\u201360. https:\/\/doi.org\/10.1093\/comjnl\/bxy118.","journal-title":"Comput J"},{"key":"1828_CR13","doi-asserted-by":"publisher","unstructured":"Bogatu A, Fernandes AAA, Paton NW, et\u00a0al. Dataset discovery in data lakes. In: 36th IEEE International Conference on Data Engineering, ICDE. IEEE, 2020:709\u2013720. https:\/\/doi.org\/10.1109\/ICDE48307.2020.00067","DOI":"10.1109\/ICDE48307.2020.00067"},{"key":"1828_CR14","unstructured":"Bogorny V, Engel PM, Alvares LO. A reuse-based spatial data preparation framework for data mining. In: Proceedings of the 17th International Conference on Software Engineering and Knowledge Engineering (SEKE\u20192005), Taipei, Taiwan, Republic of China, July 14\u201316, 2005;649\u2013652."},{"key":"1828_CR15","doi-asserted-by":"publisher","unstructured":"Bonfitto S, Casiraghi E, Mesiti M. Table understanding approaches for extracting knowledge from heterogeneous tables. WIREs Data Mining Knowl Discov 2021;11(4) https:\/\/doi.org\/10.1002\/widm.1407.","DOI":"10.1002\/widm.1407"},{"key":"1828_CR16","unstructured":"Bouman R, van Dongen J. Pentaho Solutions: Business Intelligence and Data Warehousing with Pentaho and MySQL. Wiley Publishing. 2009."},{"key":"1828_CR17","doi-asserted-by":"publisher","unstructured":"Cappuzzo R, Papotti P, Thirumuruganathan S. Creating embeddings of heterogeneous relational datasets for data integration tasks. In: Proc. 2020 International Conference on Management of Data, SIGMOD. ACM, 2020:1335\u201349. https:\/\/doi.org\/10.1145\/3318464.3389742.","DOI":"10.1145\/3318464.3389742"},{"key":"1828_CR18","doi-asserted-by":"publisher","unstructured":"Chapman A, Simperl E, Koesten L, et al. Dataset search: a survey. VLDB J. 2020;29(1):251\u201372. https:\/\/doi.org\/10.1007\/s00778-019-00564-x.","DOI":"10.1007\/s00778-019-00564-x"},{"key":"1828_CR19","doi-asserted-by":"crossref","unstructured":"Chiang F, Miller RJ. A unified model for data and constraint repair. In: Proceedings of the 27th International Conference on Data Engineering, ICDE, 2011;446\u2013457.","DOI":"10.1109\/ICDE.2011.5767833"},{"key":"1828_CR20","doi-asserted-by":"publisher","unstructured":"Chu X, Morcos J, Ilyas IF, et\u00a0al. KATARA: A data cleaning system powered by knowledge bases and crowdsourcing. In: Proc. 2015 ACM SIGMOD International Conference on Management of Data. ACM, 2015;1247\u201361. https:\/\/doi.org\/10.1145\/2723372.2749431.","DOI":"10.1145\/2723372.2749431"},{"key":"1828_CR21","unstructured":"Deng D, Fernandez RC, Abedjan Z, et\u00a0al. The data civilizer system. In: CIDR 2017, 8th Biennial Conference on Innovative Data Systems Research 2017."},{"key":"1828_CR22","doi-asserted-by":"publisher","unstructured":"Diaz O, Kushibar K, Osuala R, et\u00a0al. Data preparation for artificial intelligence in medical imaging: A comprehensive guide to open-access platforms and tools. Physica Med. 2021;83:25\u201337. https:\/\/doi.org\/10.1016\/j.ejmp.2021.02.007. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S1120179721000958","DOI":"10.1016\/j.ejmp.2021.02.007"},{"key":"1828_CR23","doi-asserted-by":"publisher","unstructured":"Doan A, Domingos PM, Halevy AY. Reconciling schemas of disparate data sources: A machine-learning approach. In: Proc. ACM SIGMOD international conference on Management of data, 2001:509\u2013520. https:\/\/doi.org\/10.1145\/375663.375731","DOI":"10.1145\/375663.375731"},{"key":"1828_CR24","doi-asserted-by":"crossref","unstructured":"Doan A, Halevy AY, Ives ZG. Principles of Data Integration. Morgan Kaufmann, 2012. http:\/\/research.cs.wisc.edu\/dibook\/","DOI":"10.1016\/B978-0-12-416044-6.00015-6"},{"key":"1828_CR25","doi-asserted-by":"publisher","unstructured":"Drosos I, Barik T, Guo PJ, et\u00a0al. Wrex: A unified programming-by-example interaction for synthesizing readable code for data scientists. In: CHI \u201920: CHI Conference on Human Factors in Computing Systems. ACM, 2020:1\u201312. https:\/\/doi.org\/10.1145\/3313831.3376442.","DOI":"10.1145\/3313831.3376442"},{"issue":"1","key":"1828_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TKDE.2007.250581","volume":"19","author":"AK Elmagarmid","year":"2007","unstructured":"Elmagarmid AK, Ipeirotis PG, Verykios VS. Duplicate record detection: a survey. IEEE Trans Knowl Data Eng. 2007;19(1):1\u201316. https:\/\/doi.org\/10.1109\/TKDE.2007.250581.","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"1828_CR27","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1016\/j.cosrev.2015.05.002","volume":"17","author":"CK Emani","year":"2015","unstructured":"Emani CK, Cullot N, Nicolle C. Understandable big data: a survey. Comput Sci Rev. 2015;17:70\u201381. https:\/\/doi.org\/10.1016\/j.cosrev.2015.05.002.","journal-title":"Comput Sci Rev"},{"issue":"1","key":"1828_CR28","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1016\/j.tcs.2004.10.033","volume":"336","author":"R Fagin","year":"2005","unstructured":"Fagin R, Kolaitis PG, Miller RJ, et al. Data exchange: semantics and query answering. TCS. 2005;336(1):89\u2013124.","journal-title":"TCS"},{"key":"1828_CR29","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1007\/978-3-642-02463-4_12","volume-title":"Conceptual Modeling: Foundations and Applications, LNCS","author":"R Fagin","year":"2009","unstructured":"Fagin R, Haas LM, Hern\u00e1ndez M, et al. Clio: Schema mapping creation and data exchange. In: Conceptual Modeling: Foundations and Applications, LNCS, vol. 5600. Berlin: Springer; 2009. p. 198\u2013236."},{"key":"1828_CR30","doi-asserted-by":"crossref","unstructured":"Fan W, Geerts F. Foundations of Data Quality Management. Morgan & Claypool 2012.","DOI":"10.1007\/978-3-031-01892-3"},{"key":"1828_CR31","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1016\/j.knosys.2014.07.007","volume":"70","author":"E Ferrara","year":"2014","unstructured":"Ferrara E, Meo PD, Fiumara G, et al. Web data extraction, applications and techniques: a survey. Knowl Based Syst. 2014;70:301\u201323. https:\/\/doi.org\/10.1016\/j.knosys.2014.07.007.","journal-title":"Knowl Based Syst"},{"key":"1828_CR32","doi-asserted-by":"publisher","unstructured":"Fink M, Meilicke C, Stuckenschmidt H. Explaining differences between unaligned table snapshots. In: Proc. 23rd International Conference on Extending Database Technology, EDBT. OpenProceedings.org, 2020:133\u2013144. https:\/\/doi.org\/10.5441\/002\/edbt.2020.13","DOI":"10.5441\/002\/edbt.2020.13"},{"key":"1828_CR33","doi-asserted-by":"publisher","unstructured":"Furche T, Gottlob G, Libkin L, et\u00a0al. Data wrangling for big data: Challenges and opportunities. In: EDBT, 2016:473\u2013478. https:\/\/doi.org\/10.5441\/002\/edbt.2016.44","DOI":"10.5441\/002\/edbt.2016.44"},{"key":"1828_CR34","doi-asserted-by":"crossref","unstructured":"Gal A. Uncertain Schema Matching. Morgan & Claypool 2011.","DOI":"10.1007\/978-3-031-01845-9"},{"issue":"4","key":"1828_CR35","doi-asserted-by":"publisher","first-page":"867","DOI":"10.1007\/s00778-019-00586-5","volume":"29","author":"F Geerts","year":"2020","unstructured":"Geerts F, Mecca G, Papotti P, et al. Cleaning data with llunatic. VLDB J. 2020;29(4):867\u201392. https:\/\/doi.org\/10.1007\/s00778-019-00586-5.","journal-title":"VLDB J"},{"issue":"2","key":"1828_CR36","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1007\/s41060-018-0129-7","volume":"6","author":"Y van Gennip","year":"2018","unstructured":"van Gennip Y, Hunter B, Ma A, et al. Unsupervised record matching with noisy and incomplete data. Int J Data Sci Anal. 2018;6(2):109\u201329. https:\/\/doi.org\/10.1007\/s41060-018-0129-7.","journal-title":"Int J Data Sci Anal"},{"key":"1828_CR37","doi-asserted-by":"crossref","unstructured":"Gulwani S. Automating string processing in spreadsheets using input-output examples. In: Proc. 38th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL, 2011:317\u2013330","DOI":"10.1145\/1926385.1926423"},{"issue":"8","key":"1828_CR38","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1145\/2240236.2240260","volume":"55","author":"S Gulwani","year":"2012","unstructured":"Gulwani S, Harris WR, Singh R. Spreadsheet data manipulation using examples. Commun ACM. 2012;55(8):97\u2013105.","journal-title":"Commun ACM"},{"key":"1828_CR39","doi-asserted-by":"publisher","unstructured":"Guo C, Hedeler C, Paton NW, et\u00a0al. Matchbench: Benchmarking schema matching algorithms for schematic correspondences. In: 29th British National Conference on Databases, BNCOD, 2013:92\u2013106. https:\/\/doi.org\/10.1007\/978-3-642-39467-6_11","DOI":"10.1007\/978-3-642-39467-6_11"},{"issue":"4","key":"1828_CR40","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1007\/s007780100054","volume":"10","author":"AY Halevy","year":"2001","unstructured":"Halevy AY. Answering queries using views: a survey. VLDBJ. 2001;10(4):270\u201394. https:\/\/doi.org\/10.1007\/s007780100054.","journal-title":"VLDBJ"},{"issue":"3","key":"1828_CR41","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1145\/3444831.3444835","volume":"49","author":"M Hameed","year":"2020","unstructured":"Hameed M, Naumann F. Data preparation: a survey of commercial tools. SIGMOD Rec. 2020;49(3):18\u201329.","journal-title":"SIGMOD Rec"},{"key":"1828_CR42","doi-asserted-by":"publisher","unstructured":"He J, Veltri E, Santoro D, et\u00a0al. Interactive and deterministic data cleaning: A tossed stone raises a thousand ripples. Proc ACM SIGMOD International Conference on Management of Data 26-June-20 2016:893\u2013907. https:\/\/doi.org\/10.1145\/2882903.2915242.","DOI":"10.1145\/2882903.2915242"},{"key":"1828_CR43","unstructured":"He X, Zhao K, Chu X. Automl: a survey of the state-of-the-art. CoRR abs\/1908.00709. 2019 arXiv:1908.00709"},{"key":"1828_CR44","doi-asserted-by":"crossref","unstructured":"He Y, Jin Z, Chaudhuri S. Auto-transform: learning-to-transform by patterns. Proc VLDB Endow. 2020;13(11):2368\u20132381. http:\/\/www.vldb.org\/pvldb\/vol13\/p2368-he.pdf","DOI":"10.14778\/3407790.3407831"},{"key":"1828_CR45","unstructured":"Hellerstein JM, Heer J, Kandel S. Self-service data preparation: Research to practice. IEEE Data Eng Bull 2018a;41(2):23\u201334. http:\/\/sites.computer.org\/debull\/A18june\/p23.pdf"},{"key":"1828_CR46","unstructured":"Hellerstein JM, Heer J, Kandel S. Self-Service Data Preparation: Research to Practice. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering 2018b:23\u201334"},{"issue":"4","key":"1828_CR47","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1561\/1900000045","volume":"5","author":"IF Ilyas","year":"2015","unstructured":"Ilyas IF, Chu X. Trends in cleaning relational data: consistency and deduplication. Found Trends Datab. 2015;5(4):281\u2013393. https:\/\/doi.org\/10.1561\/1900000045.","journal-title":"Found Trends Datab."},{"key":"1828_CR48","doi-asserted-by":"crossref","unstructured":"Ioannidis YE. The history of histograms (abridged). In: VLDB. Morgan Kaufmann, 2003:19\u201330","DOI":"10.1016\/B978-012722442-8\/50011-2"},{"key":"1828_CR49","doi-asserted-by":"publisher","unstructured":"Jin Z, Anderson MR, Cafarella MJ, et\u00a0al. Foofah: Transforming data by example. In: Proc. of the 2017 ACM International Conference on Management of Data, SIGMOD. ACM, 2017:683\u2013698, https:\/\/doi.org\/10.1145\/3035918.3064034","DOI":"10.1145\/3035918.3064034"},{"issue":"4","key":"1828_CR50","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1177\/1473871611415994","volume":"10","author":"S Kandel","year":"2011","unstructured":"Kandel S, Heer J, Plaisant C, et al. Research directions in data wrangling: Visualizations and transformations for usable and credible data. Inf Vis. 2011;10(4):271\u201388.","journal-title":"Inf Vis"},{"key":"1828_CR51","doi-asserted-by":"crossref","unstructured":"Kandel S, Paepcke A, Hellerstein J, et\u00a0al. Wrangler: Interactive visual specification of data transformation scripts. In: CHI, 2011b:3363\u20133372","DOI":"10.1145\/1978942.1979444"},{"key":"1828_CR52","unstructured":"Kazil J, Jarmul K. Data Wrangling with Python: Tips and Tools to Make Your Life Easier, 1st edn. O\u2019Reilly Media, Inc. 2016."},{"issue":"3","key":"1828_CR53","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1007\/BF01263333","volume":"1","author":"W Kim","year":"1993","unstructured":"Kim W, Choi I, Gala SK, et al. On resolving schematic heterogeneity in multidatabase systems. Distributed and Parallel Databases. 1993;1(3):251\u201379. https:\/\/doi.org\/10.1007\/BF01263333.","journal-title":"Distributed and Parallel Databases"},{"key":"1828_CR54","doi-asserted-by":"publisher","unstructured":"Kluyver T, et\u00a0al. Jupyter notebooks - a publishing format for reproducible computational workflows. In: Loizides F, Schmidt B (eds) 20th International Conference on Electronic Publishing. IOS Press, 2016:87\u201390, https:\/\/doi.org\/10.3233\/978-1-61499-649-1-87","DOI":"10.3233\/978-1-61499-649-1-87"},{"issue":"1","key":"1828_CR55","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1109\/TBDATA.2019.2907588","volume":"7","author":"M Koehler","year":"2021","unstructured":"Koehler M, Abel E, Bogatu A, et al. Incorporating data context to cost-effectively automate end-to-end data wrangling. IEEE Trans Big Data. 2021;7(1):169\u201386. https:\/\/doi.org\/10.1109\/TBDATA.2019.2907588.","journal-title":"IEEE Trans Big Data"},{"key":"1828_CR56","doi-asserted-by":"crossref","unstructured":"Konstantinou N, Koehler M, Abel E, et al. The VADA architecture for cost-effective data wrangling. In: Proc. ACM international conference on management of data, SIGMOD; 2017. p. 1599\u2013602.","DOI":"10.1145\/3035918.3058730"},{"key":"1828_CR57","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1186\/s40537-019-0237-9","volume":"6","author":"N Konstantinou","year":"2019","unstructured":"Konstantinou N, Abel E, Bellomarini L, et al. VADA: an architecture for end user informed data preparation. J Big Data. 2019;6:74. https:\/\/doi.org\/10.1186\/s40537-019-0237-9.","journal-title":"J Big Data"},{"key":"1828_CR58","doi-asserted-by":"publisher","unstructured":"Kruse S, Papotti P, Naumann F. Estimating data integration and cleaning effort. In: Proceedings of the 18th International Conference on Extending Database Technology, EDBT 2015, Brussels, Belgium, March 23-27, 2015:61\u201372, https:\/\/doi.org\/10.5441\/002\/edbt.2015.07, https:\/\/doi.org\/10.5441\/002\/edbt.2015.07","DOI":"10.5441\/002\/edbt.2015.07"},{"issue":"12","key":"1828_CR59","first-page":"1438","volume":"4","author":"B Marnette","year":"2011","unstructured":"Marnette B, Mecca G, Papotti P, et al. ++spicy: an opensource tool for second-generation schema mapping and data exchange. PVLDB. 2011;4(12):1438\u201341.","journal-title":"PVLDB"},{"issue":"1","key":"1828_CR60","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1080\/2058802X.2019.1594871","volume":"2019","author":"L Maynard-Atem","year":"2019","unstructured":"Maynard-Atem L. The data series - data democratisation. Impact. 2019;2019(1):10\u20131. https:\/\/doi.org\/10.1080\/2058802X.2019.1594871.","journal-title":"Impact"},{"issue":"101","key":"1828_CR61","doi-asserted-by":"publisher","first-page":"904","DOI":"10.1016\/j.is.2021.101904","volume":"104","author":"L Mazilu","year":"2022","unstructured":"Mazilu L, Paton NW, Fernandes AAA, et al. Schema mapping generation in the wild. Inf Syst. 2022;104(101):904. https:\/\/doi.org\/10.1016\/j.is.2021.101904.","journal-title":"Inf Syst"},{"key":"1828_CR62","unstructured":"McKinney W. Python for Data Analysis, 2nd edn. O\u2019Reilly Media, Inc. 2018."},{"key":"1828_CR63","unstructured":"Mecca G, Papotti P, Santoro D. A short history of schema mapping systems. In: Twentieth Italian Symposium on Advanced Database Systems, SEBD 2012, 2012:99\u2013106, http:\/\/sebd2012.dei.unipd.it\/documents\/188475\/efd4de94-b0b6-4979-8f60-3628f30d6f03"},{"key":"1828_CR64","doi-asserted-by":"publisher","unstructured":"Nargesian F, Zhu E, Miller RJ, et\u00a0al. Data lake management: Challenges and opportunities. Proc VLDB Endow 2019;12(12):1986\u20131989. https:\/\/doi.org\/10.14778\/3352063.3352116","DOI":"10.14778\/3352063.3352116"},{"key":"1828_CR65","doi-asserted-by":"publisher","unstructured":"Nargesian F, Pu KQ, Zhu E, et\u00a0al. Organizing data lakes for navigation. In: Proceedings of the 2020 International Conference on Management of Data, SIGMOD. ACM, 2020; 1939\u20131950, https:\/\/doi.org\/10.1145\/3318464.3380605","DOI":"10.1145\/3318464.3380605"},{"key":"1828_CR66","doi-asserted-by":"publisher","unstructured":"Papenbrock T, Bergmann T, Finke M, et\u00a0al. Data profiling with metanome. Proc VLDB Endow 2015;8(12):1860\u20131863. https:\/\/doi.org\/10.14778\/2824032.2824086","DOI":"10.14778\/2824032.2824086"},{"key":"1828_CR67","doi-asserted-by":"publisher","unstructured":"Qian K, Popa L, Sen P. Active learning for large-scale entity resolution. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM. ACM, 2017:1379\u20131388, https:\/\/doi.org\/10.1145\/3132847.3132949","DOI":"10.1145\/3132847.3132949"},{"issue":"4","key":"1828_CR68","doi-asserted-by":"publisher","first-page":"334","DOI":"10.1007\/s007780100057","volume":"10","author":"E Rahm","year":"2001","unstructured":"Rahm E, Bernstein PA. A survey of approaches to automatic schema matching. VLDBJ. 2001;10(4):334\u201350. https:\/\/doi.org\/10.1007\/s007780100057.","journal-title":"VLDBJ"},{"key":"1828_CR69","unstructured":"Raman V, Hellerstein JM. Potter\u2019s wheel: An interactive data cleaning system. VLDB 2001 - Proceedings of 27th International Conference on Very Large Data Bases 2001:381\u2013390"},{"key":"1828_CR70","doi-asserted-by":"publisher","unstructured":"Rekatsinas T, Chu X, Ilyas IF, et\u00a0al. Holoclean: Holistic data repairs with probabilistic inference. Proc VLDB Endow 2017;10(11):1190\u20131201. https:\/\/doi.org\/10.14778\/3137628.3137631","DOI":"10.14778\/3137628.3137631"},{"key":"1828_CR71","unstructured":"Rostin A, Albrecht O, Bauckmann J, et\u00a0al. A machine learning approach to foreign key discovery. In: 12th International Workshop on the Web and Databases, WebDB 2009."},{"key":"1828_CR72","doi-asserted-by":"publisher","unstructured":"Santu SKK, Hassan MM, Smith MJ, et\u00a0al. Automl to date and beyond: Challenges and opportunities. ACM Comput Surv 2022;54(8):175:1\u2013175:36. https:\/\/doi.org\/10.1145\/3470918,","DOI":"10.1145\/3470918"},{"issue":"10","key":"1828_CR73","first-page":"816","volume":"9","author":"R Singh","year":"2016","unstructured":"Singh R. Blinkfill: Semi-supervised programming by example for syntactic string transformations. PVLDB. 2016;9(10):816\u201327.","journal-title":"PVLDB"},{"key":"1828_CR74","unstructured":"Stodder D. Improving Data Preparation for Business Analytics. Tech. rep., 2016. https:\/\/info.talend.com\/rs\/talend\/images\/WP_EN_DP_Improving_DataPrep_BusinessAnalytics.pdf"},{"key":"1828_CR75","unstructured":"Stonebraker M, Bruckner D, Ilyas IF, et\u00a0al. Data curation at scale: The data tamer system. In: CIDR 2013, Sixth Biennial Conference on Innovative Data Systems Research 2013."},{"key":"1828_CR76","doi-asserted-by":"publisher","unstructured":"Sukhobok D, Nikolov N, Roman D. Tabular Data Anomaly Patterns. Proceedings - 2017 International Conference on Big Data Innovations and Applications, Innovate-Data 2017 2018-January:25\u201334. 2018. https:\/\/doi.org\/10.1109\/Innovate-Data.2017.10","DOI":"10.1109\/Innovate-Data.2017.10"},{"key":"1828_CR77","unstructured":"Terrizzano I, Schwarz PM, Roth M, et\u00a0al. Data wrangling: The challenging journey from the wild to the lake. In: CIDR 2015."},{"key":"1828_CR78","doi-asserted-by":"publisher","unstructured":"Thirumuruganathan S, Tang N, Ouzzani M, et\u00a0al. Data curation with deep learning. In: Proceedings of the 23rd International Conference on Extending Database Technology, EDBT 2020. OpenProceedings.org, 2020:277\u2013286, https:\/\/doi.org\/10.5441\/002\/edbt.2020.25","DOI":"10.5441\/002\/edbt.2020.25"},{"issue":"3","key":"1828_CR79","first-page":"1","volume":"5","author":"P Vassiliadis","year":"2011","unstructured":"Vassiliadis P. A survey of extract-transform-load technology IJDWM. 2011;5(3):1\u201327.","journal-title":"A survey of extract-transform-load technology. IJDWM"},{"key":"1828_CR80","unstructured":"Verborgh R, Wilde MD. Using OpenRefine, 1st edn. Packt Publishing 2013."},{"key":"1828_CR81","volume-title":"Cloveretl designer: User\u2019s guide","author":"T Waller","year":"2018","unstructured":"Waller T, Korbel J, Stys M. Cloveretl designer: User\u2019s guide. Javlin: Tech. rep; 2018."},{"key":"1828_CR82","unstructured":"Wu B, Knoblock CA. An iterative approach to synthesize data transformation programs. In: Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25-31, 2015:1726\u20131732, http:\/\/ijcai.org\/Abstract\/15\/246"},{"key":"1828_CR83","doi-asserted-by":"crossref","unstructured":"Yang J, He Y, Chaudhuri S. Auto-pipeline: Synthesize data pipelines by-target using reinforcement learning and search. Proc VLDB Endow 2021;14(11):2563\u20132575. http:\/\/www.vldb.org\/pvldb\/vol14\/p2563-he.pdf","DOI":"10.14778\/3476249.3476303"},{"key":"1828_CR84","doi-asserted-by":"publisher","unstructured":"Zhu E, He Y, Chaudhuri S. Auto-join: Joining tables by leveraging transformations. Proc VLDB Endow 2017;10(10):1034\u20131045. https:\/\/doi.org\/10.14778\/3115404.3115409","DOI":"10.14778\/3115404.3115409"}],"container-title":["SN Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-023-01828-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s42979-023-01828-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-023-01828-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,17]],"date-time":"2023-06-17T15:25:45Z","timestamp":1687015545000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s42979-023-01828-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,2]]},"references-count":84,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,7]]}},"alternative-id":["1828"],"URL":"https:\/\/doi.org\/10.1007\/s42979-023-01828-8","relation":{},"ISSN":["2661-8907"],"issn-type":[{"value":"2661-8907","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,2]]},"assertion":[{"value":"16 May 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 April 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 June 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all authors, the corresponding author states that there is no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"425"}}