{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T01:09:23Z","timestamp":1789434563395,"version":"build-2803163510"},"reference-count":33,"publisher":"Association for Computing Machinery (ACM)","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2022,1]]},"abstract":"<jats:p>In recent years, complex data mining and machine learning algorithms have become more common in data analytics. Several specialized systems exist to evaluate these algorithms on ever-growing data sets, which are built to efficiently execute different types of complex analytics queries.<\/jats:p>\n          <jats:p>However, using these various systems comes at a price. Moving data out of traditional database systems is often slow as it requires exporting and importing data, which is typically performed using the relatively inefficient CSV format. Additionally, database systems usually offer strong ACID guarantees, which are lost when adding new, external systems. This disadvantage can be detrimental to the consistency of the results.<\/jats:p>\n          <jats:p>Most data scientists still prefer not to use classical database systems for data analytics. The main reason why RDBMS are not used is that SQL is difficult to work with due to its declarative and set-oriented nature, and is not easily extensible.<\/jats:p>\n          <jats:p>We present User-Defined Operators (UDOs) as a concept to include custom algorithms into modern query engines. Users can write idiomatic code in the programming language of their choice, which is then directly integrated into existing database systems. We show that our implementation can compete with specialized tools and existing query engines while retaining all beneficial properties of the database system.<\/jats:p>","DOI":"10.14778\/3510397.3510408","type":"journal-article","created":{"date-parts":[[2022,5,18]],"date-time":"2022-05-18T22:23:10Z","timestamp":1652912590000},"page":"1119-1131","source":"Crossref","is-referenced-by-count":24,"title":["User-defined operators"],"prefix":"10.14778","volume":"15","author":[{"given":"Moritz","family":"Sichert","sequence":"first","affiliation":[{"name":"Technische Universit\u00e4t M\u00fcnchen"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Neumann","sequence":"additional","affiliation":[{"name":"Technische Universit\u00e4t M\u00fcnchen"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,5,18]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2022. PL\/pgSQL - SQL Procedural Language. Retrieved January 14 2022 from https:\/\/www.postgresql.org\/docs\/14\/plpgsql.html  2022. PL\/pgSQL - SQL Procedural Language. Retrieved January 14 2022 from https:\/\/www.postgresql.org\/docs\/14\/plpgsql.html"},{"key":"e_1_2_1_2_1","volume-title":"Retrieved","year":"2022","unstructured":"2022. Rust Programming Language . Retrieved January 14, 2022 from https:\/\/www.rust-lang.org\/ 2022. Rust Programming Language. Retrieved January 14, 2022 from https:\/\/www.rust-lang.org\/"},{"key":"e_1_2_1_3_1","unstructured":"2022. Transact-SQL Reference. Retrieved January 14 2022 from https:\/\/docs.microsoft.com\/en-us\/sql\/t-sql\/language-reference  2022. Transact-SQL Reference. Retrieved January 14 2022 from https:\/\/docs.microsoft.com\/en-us\/sql\/t-sql\/language-reference"},{"key":"e_1_2_1_4_1","volume-title":"Chris Olah","author":"Abadi Mart\u00edn","unstructured":"Mart\u00edn Abadi , Ashish Agarwal , Paul Barham , Eugene Brevdo , Zhifeng Chen , Craig Citro , Gregory S. Corrado , Andy Davis , Jeffrey Dean , Matthieu Devin , Sanjay Ghemawat , Ian J. Goodfellow , Andrew Harp , Geoffrey Irving , Michael Isard , Yangqing Jia , Rafal J\u00f3zefowicz , Lukasz Kaiser , Manjunath Kudlur , Josh Levenberg , Dan Man\u00e9 , Rajat Monga , Sherry Moore , Derek Gordon Murray , Chris Olah , Mike Schuster , Jonathon Shlens , Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul A. Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda B. Vi\u00e9gas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. CoRR abs\/1603.04467 (2016). Mart\u00edn Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Gregory S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian J. Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal J\u00f3zefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Man\u00e9, Rajat Monga, Sherry Moore, Derek Gordon Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul A. Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda B. Vi\u00e9gas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. CoRR abs\/1603.04467 (2016)."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687553.1687592"},{"key":"e_1_2_1_6_1","unstructured":"Peter A. Boncz Marcin Zukowski and Niels Nes. 2005. MonetDB\/X100: Hyper-Pipelining Query Execution. In CIDR. www.cidrdb.org 225--237.  Peter A. Boncz Marcin Zukowski and Niels Nes. 2005. MonetDB\/X100: Hyper-Pipelining Query Execution. In CIDR. www.cidrdb.org 225--237."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824045"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1327452.1327492"},{"key":"e_1_2_1_9_1","unstructured":"Christian Duta Denis Hirn and Torsten Grust. 2020. Compiling PL\/SQL Away. In CIDR. www.cidrdb.org.  Christian Duta Denis Hirn and Torsten Grust. 2020. Compiling PL\/SQL Away. In CIDR. www.cidrdb.org."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687553.1687567"},{"key":"e_1_2_1_11_1","volume-title":"McKenna","author":"Graefe Goetz","year":"1993","unstructured":"Goetz Graefe and William J . McKenna . 1993 . The Volcano Optimizer Generator: Extensibility and Efficient Search. In ICDE. IEEE Computer Society , 209--218. Goetz Graefe and William J. McKenna. 1993. The Volcano Optimizer Generator: Extensibility and Efficient Search. In ICDE. IEEE Computer Society, 209--218."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.14778\/3457390.3457402"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.14778\/2367502.2367510"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457272"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-020-00643-4"},{"key":"e_1_2_1_16_1","volume-title":"Adaptive Execution of Compiled Queries","author":"Kohn Andr\u00e9","unstructured":"Andr\u00e9 Kohn , Viktor Leis , and Thomas Neumann . 2018. Adaptive Execution of Compiled Queries . In ICDE. IEEE Computer Society , 197--208. Andr\u00e9 Kohn, Viktor Leis, and Thomas Neumann. 2018. Adaptive Execution of Compiled Queries. In ICDE. IEEE Computer Society, 197--208."},{"key":"e_1_2_1_17_1","volume-title":"Adve","author":"Lattner Chris","year":"2004","unstructured":"Chris Lattner and Vikram S . Adve . 2004 . LLVM : A Compilation Framework for Lifelong Program Analysis & Transformation. In CGO. IEEE Computer Society , 75--88. Chris Lattner and Vikram S. Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In CGO. IEEE Computer Society, 75--88."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2610507"},{"key":"e_1_2_1_19_1","volume-title":"XRM - An Extended (N-ary) Relational Memory. Research Report \/ G \/ IBM \/ Cambridge Scientific Center G320-2096","author":"Lorie Raymond A.","year":"1974","unstructured":"Raymond A. Lorie . 1974. XRM - An Extended (N-ary) Relational Memory. Research Report \/ G \/ IBM \/ Cambridge Scientific Center G320-2096 ( 1974 ). Raymond A. Lorie. 1974. XRM - An Extended (N-ary) Relational Memory. Research Report \/ G \/ IBM \/ Cambridge Scientific Center G320-2096 (1974)."},{"key":"e_1_2_1_20_1","unstructured":"Derek Gordon Murray Frank McSherry Rebecca Isaacs Michael Isard Paul Barham and Mart\u00edn Abadi. 2013. Naiad: a timely dataflow system. In SOSP. ACM 439--455.  Derek Gordon Murray Frank McSherry Rebecca Isaacs Michael Isard Paul Barham and Mart\u00edn Abadi. 2013. Naiad: a timely dataflow system. In SOSP. ACM 439--455."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983551"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.14778\/2002938.2002940"},{"key":"e_1_2_1_23_1","volume-title":"Freitag","author":"Neumann Thomas","year":"2020","unstructured":"Thomas Neumann and Michael J . Freitag . 2020 . Umbra : A Disk-Based System with In-Memory Performance. In CIDR. www.cidrdb.org. Thomas Neumann and Michael J. Freitag. 2020. Umbra: A Disk-Based System with In-Memory Performance. In CIDR. www.cidrdb.org."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.14778\/3213880.3213890"},{"key":"e_1_2_1_25_1","unstructured":"Linnea Passing Manuel Then Nina Hubig Harald Lang Michael Schreier Stephan G\u00fcnnemann Alfons Kemper and Thomas Neumann. 2017. SQL- and Operator-centric Data Analytics in Relational Main-Memory Databases. In EDBT. OpenProceedings.org 84--95.  Linnea Passing Manuel Then Nina Hubig Harald Lang Michael Schreier Stephan G\u00fcnnemann Alfons Kemper and Thomas Neumann. 2017. SQL- and Operator-centric Data Analytics in Relational Main-Memory Databases. In EDBT. OpenProceedings.org 84--95."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3320212"},{"key":"e_1_2_1_27_1","first-page":"1","article-title":"Freedom for the SQL-Lambda","volume":"6","author":"Sch\u00fcle Maximilian E.","year":"2020","unstructured":"Maximilian E. Sch\u00fcle , Jakob Huber , Alfons Kemper , and Thomas Neumann . 2020 . Freedom for the SQL-Lambda : Just-in-Time-Compiling User-Injected Functions in PostgreSQL. In SSDBM. ACM , 6 : 1 -- 6 :12. Maximilian E. Sch\u00fcle, Jakob Huber, Alfons Kemper, and Thomas Neumann. 2020. Freedom for the SQL-Lambda: Just-in-Time-Compiling User-Injected Functions in PostgreSQL. In SSDBM. ACM, 6:1--6:12.","journal-title":"Just-in-Time-Compiling User-Injected Functions in PostgreSQL. In SSDBM. ACM"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.14778\/3447689.3447692"},{"key":"e_1_2_1_29_1","volume-title":"Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing","author":"Zaharia Matei","unstructured":"Matei Zaharia , Mosharaf Chowdhury , Tathagata Das , Ankur Dave , Justin Ma , Murphy McCauly , Michael J. Franklin , Scott Shenker , and Ion Stoica . 2012. Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing . In NSDI. USENIX Association , 15--28. Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauly, Michael J. Franklin, Scott Shenker, and Ion Stoica. 2012. Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing. In NSDI. USENIX Association, 15--28."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.14778\/3447689.3447697"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.14778\/3467861.3467867"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.14778\/3457390.3457392"},{"key":"e_1_2_1_33_1","volume-title":"Boncz","author":"Zukowski Marcin","year":"2012","unstructured":"Marcin Zukowski , Mark van de Wiel , and Peter A . Boncz . 2012 . Vectorwise : A Vectorized Analytical DBMS. In ICDE. IEEE Computer Society , 1349--1350. Marcin Zukowski, Mark van de Wiel, and Peter A. Boncz. 2012. Vectorwise: A Vectorized Analytical DBMS. In ICDE. IEEE Computer Society, 1349--1350."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3510397.3510408","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T09:25:37Z","timestamp":1672219537000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3510397.3510408"}},"subtitle":["efficiently integrating custom algorithms into modern databases"],"short-title":[],"issued":{"date-parts":[[2022,1]]},"references-count":33,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2022,1]]}},"alternative-id":["10.14778\/3510397.3510408"],"URL":"https:\/\/doi.org\/10.14778\/3510397.3510408","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2022,1]]}}}