{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T06:51:39Z","timestamp":1777013499489,"version":"3.51.4"},"reference-count":16,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,10,30]],"date-time":"2023-10-30T00:00:00Z","timestamp":1698624000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGMOD Rec."],"published-print":{"date-parts":[[2023,10,30]]},"abstract":"<jats:p>The large variety of specialized data processing platforms and the increased complexity of data analytics has led to the need for unifying data analytics within a single framework. Such a framework should free users from the burden of (i) choosing the right platform( s) and (ii) gluing code between the different parts of their pipelines. Apache Wayang (Incubating) is the only open-source framework that provides a systematic solution to unified data analytics by integrating multiple heterogeneous data processing platforms. It achieves that by decoupling applications from the underlying platforms and providing an optimizer so that users do not have to specify the platforms on which their pipeline should run. Wayang provides a unified view and processing model, effectively integrating the hodgepodge of heterogeneous platforms into a single framework with increased usability without sacrificing performance and total cost of ownership. In this paper, we present the architecture ofWayang, describe its main components, and give an outlook on future directions.<\/jats:p>","DOI":"10.1145\/3631504.3631510","type":"journal-article","created":{"date-parts":[[2023,11,2]],"date-time":"2023-11-02T14:48:51Z","timestamp":1698936531000},"page":"30-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Apache Wayang: A Unified Data Analytics Framework"],"prefix":"10.1145","volume":"52","author":[{"given":"Kaustubh","family":"Beedkar","sequence":"first","affiliation":[{"name":"Indian Institute of Technology Delhi"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bertty","family":"Contreras-Rojas","sequence":"additional","affiliation":[{"name":"Technische Universit\u00a8at Berlin"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haralampos","family":"Gavriilidis","sequence":"additional","affiliation":[{"name":"Technische Universit\u00a8at Berlin"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zoi","family":"Kaoudi","sequence":"additional","affiliation":[{"name":"IT University of Copenhagen"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Volker","family":"Markl","sequence":"additional","affiliation":[{"name":"Technische Universit\u00a8at Berlin"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rodrigo","family":"Pardo-Meza","sequence":"additional","affiliation":[{"name":"Technische Universit\u00a8at Berlin"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jorge-Arnulfo","family":"Quian\u00e9-Ruiz","sequence":"additional","affiliation":[{"name":"IT University of Copenhagen"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,11,2]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"https:\/\/beam.apache.org. Retrieved","year":"2023","unstructured":"The Unified Apache Beam Model. https:\/\/beam.apache.org. Retrieved May , 2023 . The Unified Apache Beam Model. https:\/\/beam.apache.org. Retrieved May, 2023."},{"key":"e_1_2_1_2_1","volume-title":"https:\/\/www.databloom.ai\/","year":"2022","unstructured":"Databloom ai : https:\/\/www.databloom.ai\/ , 2022 . Databloom ai: https:\/\/www.databloom.ai\/, 2022."},{"key":"e_1_2_1_3_1","volume-title":"Rheem: enabling cross-platform data processing -- may the big data be with you! In PVLDB","author":"Agrawal D.","year":"2018","unstructured":"D. Agrawal Rheem: enabling cross-platform data processing -- may the big data be with you! In PVLDB , 2018 . D. Agrawal et al. Rheem: enabling cross-platform data processing -- may the big data be with you! In PVLDB, 2018."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3190662"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2465327"},{"key":"e_1_2_1_6_1","first-page":"133","volume-title":"BIRTE","author":"Doka K.","year":"2016","unstructured":"K. Doka , I. Mytilinis , N. Papailiou , V. Giannakouris , D. Tsoumakos , and N. Koziris . Multi-engine analytics with ires . In BIRTE , pages 133 -- 154 , 2016 . K. Doka, I. Mytilinis, N. Papailiou, V. Giannakouris, D. Tsoumakos, and N. Koziris. Multi-engine analytics with ires. In BIRTE, pages 133--154, 2016."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2814710.2814713"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE55515.2023.00214"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2741948.2741968"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.14778\/3554821.3554898"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3064042"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE48307.2020.00132"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-020-00612-x"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2019.00162"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007568.1007642"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM50108.2020.00055"}],"container-title":["ACM SIGMOD Record"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3631504.3631510","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3631504.3631510","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:52Z","timestamp":1750291432000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3631504.3631510"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,30]]},"references-count":16,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,10,30]]}},"alternative-id":["10.1145\/3631504.3631510"],"URL":"https:\/\/doi.org\/10.1145\/3631504.3631510","relation":{},"ISSN":["0163-5808"],"issn-type":[{"value":"0163-5808","type":"print"}],"subject":[],"published":{"date-parts":[[2023,10,30]]},"assertion":[{"value":"2023-11-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}