{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2023,1,13]],"date-time":"2023-01-13T23:54:50Z","timestamp":1673654090897},"reference-count":8,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2015,8]]},"abstract":"<jats:p>Integration systems are typically evaluated using a few real-world scenarios (e.g., bibliographical or biological datasets) or using synthetic scenarios (e.g., based on star-schemas or other patterns for schemas and constraints). Reusing such evaluations is a cumbersome task because their focus is usually limited to showcasing a specific feature of an approach. This makes it difficult to compare integration solutions, understand their generality, and understand their performance for different application scenarios. Based on this observation, we demonstrate some of the requirements for developing integration benchmarks. We argue that the major abstractions used for integration problems have converged in the last decade which enables the application of robust empirical methods to integration problems (from schema evolution, to data exchange, to answering queries using views and many more). Specifically, we demonstrate that schema mappings are the main abstraction that now drives most integration solutions and show how a metadata generator can be used to create more credible evaluations of the performance and scalability of data integration systems. We will use the demonstration to evangelize for more robust, shared empirical evaluations of data integration systems.<\/jats:p>","DOI":"10.14778\/2824032.2824111","type":"journal-article","created":{"date-parts":[[2015,9,16]],"date-time":"2015-09-16T12:18:17Z","timestamp":1442405897000},"page":"1960-1963","source":"Crossref","is-referenced-by-count":3,"title":["Gain control over your integration evaluations"],"prefix":"10.14778","volume":"8","author":[{"given":"Patricia C.","family":"Arocena","sequence":"first","affiliation":[{"name":"University of Toronto"}]},{"given":"Radu","family":"Ciucanu","sequence":"additional","affiliation":[{"name":"University of Oxford"}]},{"given":"Boris","family":"Glavic","sequence":"additional","affiliation":[{"name":"Illinois Inst. of Technology"}]},{"given":"Ren\u00e9e J.","family":"Miller","sequence":"additional","affiliation":[{"name":"University of Toronto"}]}],"member":"320","published-online":{"date-parts":[[2015,8]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-012-0264-z"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.14778\/1453856.1453886"},{"key":"e_1_2_1_3_1","first-page":"157","volume-title":"SIGMOD","author":"Arocena P. C.","year":"2013","unstructured":"P. C. Arocena , B. Glavic , and R. J. Miller . Value Invention in Data Exchange . In SIGMOD , pages 157 -- 168 , 2013 . 10.1145\/2463676.2465311 P. C. Arocena, B. Glavic, and R. J. Miller. Value Invention in Data Exchange. In SIGMOD, pages 157--168, 2013. 10.1145\/2463676.2465311"},{"key":"e_1_2_1_4_1","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1007\/978-3-642-16518-4_9","volume-title":"Schema Matching and Mapping","author":"Bellahsene Z.","year":"2011","unstructured":"Z. Bellahsene , A. Bonifati , F. Duchateau , and Y. Velegrakis . On Evaluating Schema Matching and Mapping . In Schema Matching and Mapping , pages 253 -- 291 . Springer , 2011 . Z. Bellahsene, A. Bonifati, F. Duchateau, and Y. Velegrakis. On Evaluating Schema Matching and Mapping. In Schema Matching and Mapping, pages 253--291. Springer, 2011."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-007-0059-9"},{"key":"e_1_2_1_6_1","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1007\/978-3-642-02463-4_12","volume-title":"Conceptual Modeling: Foundations and Applications","author":"Fagin R.","year":"2009","unstructured":"R. Fagin , L. M. Haas , M. A. Hern\u00e1ndez , R. J. Miller , L. Popa , and Y. Velegrakis . Clio: Schema Mapping Creation and Data Exchange . In Conceptual Modeling: Foundations and Applications , pages 198 -- 236 , 2009 . 10.1007\/978-3-642-02463-4_12 R. Fagin, L. M. Haas, M. A. Hern\u00e1ndez, R. J. Miller, L. Popa, and Y. Velegrakis. Clio: Schema Mapping Creation and Data Exchange. In Conceptual Modeling: Foundations and Applications, pages 198--236, 2009. 10.1007\/978-3-642-02463-4_12"},{"issue":"12","key":"e_1_2_1_7_1","first-page":"1438","article-title":"++Spicy: an OpenSource Tool for Second-Generation Schema Mapping and Data Exchange","volume":"4","author":"Marnette B.","year":"2011","unstructured":"B. Marnette , G. Mecca , P. Papotti , S. Raunich , and D. Santoro . ++Spicy: an OpenSource Tool for Second-Generation Schema Mapping and Data Exchange . PVLDB , 4 ( 12 ): 1438 -- 1441 , 2011 . B. Marnette, G. Mecca, P. Papotti, S. Raunich, and D. Santoro. ++Spicy: an OpenSource Tool for Second-Generation Schema Mapping and Data Exchange. PVLDB, 4(12):1438--1441, 2011.","journal-title":"PVLDB"},{"key":"e_1_2_1_8_1","first-page":"1006","volume-title":"VLDB","author":"Yu C.","year":"2005","unstructured":"C. Yu and L. Popa . Semantic Adaptation of Schema Mappings when Schemas Evolve . VLDB , pages 1006 -- 1017 , 2005 . C. Yu and L. Popa. Semantic Adaptation of Schema Mappings when Schemas Evolve. VLDB, pages 1006--1017, 2005."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/2824032.2824111","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:07:40Z","timestamp":1672222060000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/2824032.2824111"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,8]]},"references-count":8,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2015,8]]}},"alternative-id":["10.14778\/2824032.2824111"],"URL":"https:\/\/doi.org\/10.14778\/2824032.2824111","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2015,8]]}}}