{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T23:39:30Z","timestamp":1774654770427,"version":"3.50.1"},"reference-count":25,"publisher":"Association for Computing Machinery (ACM)","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2015,11]]},"abstract":"<jats:p>Given the maturity of the data integration field it is surprising that rigorous empirical evaluations of research ideas are so scarce. We identify a major roadblock for empirical work - the lack of comprehensive metadata generators that can be used to create benchmarks for different integration tasks. This makes it difficult to compare integration solutions, understand their generality, and understand their performance. We present iBench, the first metadata generator that can be used to evaluate a wide-range of integration tasks (data exchange, mapping creation, mapping composition, schema evolution, among many others). iBench permits control over the size and characteristics of the metadata it generates (schemas, constraints, and mappings). Our evaluation demonstrates that iBench can efficiently generate very large, complex, yet realistic scenarios with different characteristics. We also present an evaluation of three mapping creation systems using iBench and show that the intricate control that iBench provides over metadata scenarios can reveal new and important empirical insights. iBench is an open-source, extensible tool that we are providing to the community. We believe it will raise the bar for empirical evaluation and comparison of data integration systems.<\/jats:p>","DOI":"10.14778\/2850583.2850586","type":"journal-article","created":{"date-parts":[[2016,2,1]],"date-time":"2016-02-01T14:10:31Z","timestamp":1454335831000},"page":"108-119","source":"Crossref","is-referenced-by-count":37,"title":["The iBench integration metadata generator"],"prefix":"10.14778","volume":"9","author":[{"given":"Patricia C.","family":"Arocena","sequence":"first","affiliation":[{"name":"University of Toronto"}]},{"given":"Boris","family":"Glavic","sequence":"additional","affiliation":[{"name":"Illinois Institute of Technology"}]},{"given":"Radu","family":"Ciucanu","sequence":"additional","affiliation":[{"name":"University of Oxford"}]},{"given":"Ren\u00e9e J.","family":"Miller","sequence":"additional","affiliation":[{"name":"University of Toronto"}]}],"member":"320","published-online":{"date-parts":[[2015,11]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-012-0264-z"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.14778\/1453856.1453886"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989338"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcss.2013.01.002"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824111"},{"key":"e_1_2_1_6_1","volume-title":"U. of Toronto","author":"Arocena P. C.","year":"2015","unstructured":"P. C. Arocena , B. Glavic , R. Ciucanu , and R. J. Miller . The iBench Integration Metadata Generator: Extended Version. Technical report , U. of Toronto , 2015 . https:\/\/bitbucket.org\/ibencher\/ibench. P. C. Arocena, B. Glavic, R. Ciucanu, and R. J. Miller. The iBench Integration Metadata Generator: Extended Version. Technical report, U. of Toronto, 2015. https:\/\/bitbucket.org\/ibencher\/ibench."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2465311"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/564691.564769"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-007-0059-9"},{"key":"e_1_2_1_10_1","first-page":"171","volume-title":"Graph Data Exchange with Target Constraints. In EDBT\/ICDT Workshops--GraphQ","author":"Boneva I.","year":"2015","unstructured":"I. Boneva , A. Bonifati , and R. Ciucanu . Graph Data Exchange with Target Constraints. In EDBT\/ICDT Workshops--GraphQ , pages 171 -- 176 , 2015 . I. Boneva, A. Bonifati, and R. Ciucanu. Graph Data Exchange with Target Constraints. In EDBT\/ICDT Workshops--GraphQ, pages 171--176, 2015."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1292609.1292615"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02463-4_12"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1O16\/j.tcs.2004.10.033"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1114244.1114249"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2005.140"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/543613.543644"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920859"},{"issue":"12","key":"e_1_2_1_18_1","first-page":"1438","volume":"4","author":"Marnette B.","year":"2011","unstructured":"B. Marnette , G. Mecca , P. Papotti , S. Raunich , and D. Santoro . ++Spicy: an OpenSource Tool for Second-Generation Schema Mapping and Data Exchange. PVLDB , 4 ( 12 ): 1438 -- 1441 , 2011 . B. Marnette, G. Mecca, P. Papotti, S. Raunich, and D. Santoro. ++Spicy: an OpenSource Tool for Second-Generation Schema Mapping and Data Exchange. PVLDB, 4(12):1438--1441, 2011.","journal-title":"PVLDB"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1559845.1559914"},{"key":"e_1_2_1_20_1","volume-title":"The Amalgam Schema and Data Integration Test Suite. www.cs.toronto.edu\/~miller\/amalgam","author":"Miller R. J.","year":"2001","unstructured":"R. J. Miller , D. Fisla , M. Huang , D. Kymlicka , F. Ku , and V. Lee . The Amalgam Schema and Data Integration Test Suite. www.cs.toronto.edu\/~miller\/amalgam , 2001 . R. J. Miller, D. Fisla, M. Huang, D. Kymlicka, F. Ku, and V. Lee. The Amalgam Schema and Data Integration Test Suite. www.cs.toronto.edu\/~miller\/amalgam, 2001."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2209249.2209271"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/1287369.1287421"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/237661.237717"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1629175.1629201"},{"key":"e_1_2_1_25_1","first-page":"1006","volume-title":"VLDB","author":"Yu C.","year":"2005","unstructured":"C. Yu and L. Popa . Semantic Adaptation of Schema Mappings when Schemas Evolve . VLDB , pages 1006 -- 1017 , 2005 . C. Yu and L. Popa. Semantic Adaptation of Schema Mappings when Schemas Evolve. VLDB, pages 1006--1017, 2005."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/2850583.2850586","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:16:54Z","timestamp":1672222614000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/2850583.2850586"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,11]]},"references-count":25,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2015,11]]}},"alternative-id":["10.14778\/2850583.2850586"],"URL":"https:\/\/doi.org\/10.14778\/2850583.2850586","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2015,11]]}}}