{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T04:58:50Z","timestamp":1755838730659},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"11","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2018,7]]},"abstract":"<jats:p>With the tremendous growth in data science and machine learning, it has become increasingly clear that traditional relational database management systems (RDBMS) are lacking appropriate support for the programming paradigms required by such applications, whose developers prefer tools that perform the computation outside the database system. While the database community has attempted to integrate some of these tools in the RDBMS, this has not swayed the trend as existing solutions are often not convenient for the incremental, iterative development approach used in these fields. In this paper, we propose AIDA - an abstraction for advanced in-database analytics. AIDA emulates the syntax and semantics of popular data science packages but transparently executes the required transformations and computations inside the RDBMS. In particular, AIDA works with a regular Python interpreter as a client to connect to the database. Furthermore, it supports the seamless use of both relational and linear algebra operations using a unified abstraction. AIDA relies on the RDBMS engine to efficiently execute relational operations and on an embedded Python interpreter and NumPy to perform linear algebra operations. Data reformatting is done transparently and avoids data copy whenever possible. AIDA does not require changes to statistical packages or the RDBMS facilitating portability.<\/jats:p>","DOI":"10.14778\/3236187.3236194","type":"journal-article","created":{"date-parts":[[2018,9,10]],"date-time":"2018-09-10T12:12:28Z","timestamp":1536581548000},"page":"1400-1413","source":"Crossref","is-referenced-by-count":22,"title":["AIDA"],"prefix":"10.14778","volume":"11","author":[{"given":"Joseph Vinish","family":"D'silva","sequence":"first","affiliation":[{"name":"McGill University, Montr\u00e9al, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Florestan","family":"De Moor","sequence":"additional","affiliation":[{"name":"McGill University, Montr\u00e9al, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bettina","family":"Kemme","sequence":"additional","affiliation":[{"name":"McGill University, Montr\u00e9al, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,7]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137765.3137791"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2018.00048"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742797"},{"key":"e_1_2_1_4_1","volume-title":"Oracle","author":"Ashdown L.","year":"2017","unstructured":"L. Ashdown , T. Kyte , Oracle Database Concepts, 12c Release 2 (12.2) . Oracle , 2017 . L. Ashdown, T. Kyte, et al. Oracle Database Concepts, 12c Release 2 (12.2). Oracle, 2017."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/276304.276386"},{"key":"e_1_2_1_6_1","volume-title":"Next-Generation Web Frameworks in Python. O'Reilly Short Cut","author":"Daly L.","year":"2007","unstructured":"L. Daly . Next-Generation Web Frameworks in Python. O'Reilly Short Cut . O'Reilly Media , 2007 . L. Daly. Next-Generation Web Frameworks in Python. O'Reilly Short Cut. O'Reilly Media, 2007."},{"key":"e_1_2_1_7_1","volume-title":"The Machine Learning Database. White paper","year":"2016","unstructured":"Datacratic. The Machine Learning Database. White paper , 2016 . Datacratic. The Machine Learning Database. White paper, 2016."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1516360.1516362"},{"key":"e_1_2_1_9_1","volume-title":"Extending Apache Spark SQL Data Source APIs with Join Push Down","author":"Li Ioana","year":"2017","unstructured":"Delaney, Ioana and Li , Jia. Extending Apache Spark SQL Data Source APIs with Join Push Down , 2017 . {https:\/\/databricks.com\/session\/extending-apache-spark-sql-data-source-apis-with-join-push-down, accessed 20-April-2018}. Delaney, Ioana and Li, Jia. Extending Apache Spark SQL Data Source APIs with Join Push Down, 2017. {https:\/\/databricks.com\/session\/extending-apache-spark-sql-data-source-apis-with-join-push-down, accessed 20-April-2018}."},{"key":"e_1_2_1_10_1","author":"Diakopoulos N.","year":"2014","unstructured":"N. Diakopoulos , S. Cass , and J. Romero . Data-Driven Rankings: the Design and Development of the IEEE Top Programming Languages News App. In Symposium on Computation+ Journalism , 2014 . N. Diakopoulos, S. Cass, and J. Romero. Data-Driven Rankings: the Design and Development of the IEEE Top Programming Languages News App. In Symposium on Computation+ Journalism, 2014.","journal-title":"In Symposium on Computation+ Journalism"},{"key":"e_1_2_1_11_1","doi-asserted-by":"crossref","unstructured":"T. W. Dinsmore. In-Memory Analytics pages 97--116. Apress Berkeley CA 2016. T. W. Dinsmore. In-Memory Analytics pages 97--116. Apress Berkeley CA 2016.","DOI":"10.1007\/978-1-4842-1311-7_5"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2347736.2347755"},{"key":"e_1_2_1_13_1","volume-title":"GNU Octave","author":"Eaton J. W.","year":"2007","unstructured":"J. W. Eaton , D. Bateman , and S. Hauberg . GNU Octave . Free Software Foundation , 2007 . J. W. Eaton, D. Bateman, and S. Hauberg. GNU Octave. Free Software Foundation, 2007."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2213836.2213874"},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4842-1191-5","volume-title":"Database Systems: A Pragmatic Approach","author":"Foster E. C.","year":"2016","unstructured":"E. C. Foster and S. Godbole . Database Systems: A Pragmatic Approach . Apress , 2016 . E. C. Foster and S. Godbole. Database Systems: A Pragmatic Approach. Apress, 2016."},{"key":"e_1_2_1_16_1","unstructured":"M. Fowler. Domain-Specific Languages. Pearson Education 2010. M. Fowler. Domain-Specific Languages . Pearson Education 2010."},{"issue":"12","key":"e_1_2_1_17_1","first-page":"1307","volume":"4","author":"Gro\u00dfe P.","year":"2011","unstructured":"P. Gro\u00dfe , W. Lehner , T. Weichert , F. F\u00e4rber , and W.-S. Li. Bridging Two Worlds with RICE Integrating R into the SAP In-Memory Computing Engine. PVLDB , 4 ( 12 ): 1307 -- 1317 , 2011 . P. Gro\u00dfe, W. Lehner, T. Weichert, F. F\u00e4rber, and W.-S. Li. Bridging Two Worlds with RICE Integrating R into the SAP In-Memory Computing Engine. PVLDB, 4(12):1307--1317, 2011.","journal-title":"SAP In-Memory Computing Engine. PVLDB"},{"key":"e_1_2_1_18_1","author":"Guyon I.","year":"2003","unstructured":"I. Guyon and A. Elisseeff . An Introduction to Variable and Feature Selection. Journal of Machine Learning Research, 3(Mar):1157--1182 , 2003 . I. Guyon and A. Elisseeff. An Introduction to Variable and Feature Selection. Journal of Machine Learning Research, 3(Mar):1157--1182, 2003.","journal-title":"Journal of Machine Learning Research, 3(Mar):1157--1182"},{"key":"e_1_2_1_19_1","volume-title":"International Conference on Simp\u00f3sio Brasileiro de Banco de Dados (SSBD)","author":"Holanda P.","year":"2017","unstructured":"P. Holanda , M. Raasveldt , and M. Kersten . Don't Hold My UDFs Hostage-Exporting UDFs For Debugging Purposes . In International Conference on Simp\u00f3sio Brasileiro de Banco de Dados (SSBD) , 2017 . P. Holanda, M. Raasveldt, and M. Kersten. Don't Hold My UDFs Hostage-Exporting UDFs For Debugging Purposes. In International Conference on Simp\u00f3sio Brasileiro de Banco de Dados (SSBD), 2017."},{"key":"e_1_2_1_20_1","volume-title":"June","author":"IBM.","year":"2008","unstructured":"IBM. IBM InfoSphere DataStage Balanced Optimization. White paper , June 2008 . IBM. IBM InfoSphere DataStage Balanced Optimization. White paper, June 2008."},{"issue":"3","key":"e_1_2_1_21_1","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1080\/10618600.1996.10474713","volume":"5","author":"Ihaka R.","year":"1996","unstructured":"R. Ihaka and R. Gentleman . R : A Language for Data Analysis and Graphics. Journal of Computational and Graphical Statistics , 5 ( 3 ): 299 -- 314 , 1996 . R. Ihaka and R. Gentleman. R: A Language for Data Analysis and Graphics. Journal of Computational and Graphical Statistics, 5(3):299--314, 1996.","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"e_1_2_1_22_1","volume-title":"Cost-effective Scalability and Performance through Pushdown Processing. White paper","author":"Informatica Corporation","year":"2007","unstructured":"Informatica Corporation . How to Achieve Flexible , Cost-effective Scalability and Performance through Pushdown Processing. White paper , November 2007 . Informatica Corporation. How to Achieve Flexible, Cost-effective Scalability and Performance through Pushdown Processing. White paper, November 2007."},{"key":"e_1_2_1_23_1","volume-title":"Linux Journal","author":"Kuchling A. M.","year":"1998","unstructured":"A. M. Kuchling . The Python DB-API . Linux Journal , 1998(49es):8, 1998 . A. M. Kuchling. The Python DB-API. Linux Journal, 1998(49es):8, 1998."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3054775"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2618243.2618265"},{"key":"e_1_2_1_26_1","first-page":"294","volume-title":"VLDB","author":"Linnemann V.","year":"1988","unstructured":"V. Linnemann , K. K\u00fcspert , P. Dadam , P. Pistor , R. Erbe , A. Kemper , N. S\u00fcdkamp , G. Walch , and M. Wallrath . Design and Implementation of an Extensible Database Management System Supporting User Defined Data Types and Functions . In VLDB , pages 294 -- 305 , 1988 . V. Linnemann, K. K\u00fcspert, P. Dadam, P. Pistor, R. Erbe, A. Kemper, N. S\u00fcdkamp, G. Walch, and M. Wallrath. Design and Implementation of an Extensible Database Management System Supporting User Defined Data Types and Functions. In VLDB, pages 294--305, 1988."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2017.108"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/304181.304211"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2769493.2769526"},{"key":"e_1_2_1_30_1","volume-title":"Feb.","author":"McKerns M. M.","year":"2012","unstructured":"M. M. McKerns , L. Strand , T. Sullivan , A. Fang , and M. A. Aivazis . Building a Framework for Predictive Science. CoRR, abs\/1202.1056 , Feb. 2012 . M. M. McKerns, L. Strand, T. Sullivan, A. Fang, and M. A. Aivazis. Building a Framework for Predictive Science. CoRR, abs\/1202.1056, Feb. 2012."},{"key":"e_1_2_1_31_1","first-page":"1","volume-title":"Python for High Performance and Scientific Computing","author":"McKinney W.","year":"2011","unstructured":"W. McKinney . pandas : a Foundational Python Library for Data Analysis and Statistics . Python for High Performance and Scientific Computing , pages 1 -- 9 , 2011 . W. McKinney. pandas: a Foundational Python Library for Data Analysis and Statistics. Python for High Performance and Scientific Computing, pages 1--9, 2011."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137628.3137634"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1412331.1412334"},{"key":"e_1_2_1_34_1","volume-title":"Morgan Kaufmann Series in Data. Morgan Kaufmann","author":"Melton J.","year":"2002","unstructured":"J. Melton and A. R. Simon . SQL:1999: Understanding Relational Language Components . Morgan Kaufmann Series in Data. Morgan Kaufmann , 2002 . J. Melton and A. R. Simon. SQL:1999: Understanding Relational Language Components. Morgan Kaufmann Series in Data. Morgan Kaufmann, 2002."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.5555\/2946645.2946679"},{"key":"e_1_2_1_36_1","volume-title":"MicroStrategy","year":"2012","unstructured":"MicroStrategy. Architecture for Enterprise Business Intelligence. White paper , MicroStrategy , Incorporated , 2012 . MicroStrategy. Architecture for Enterprise Business Intelligence. White paper, MicroStrategy, Incorporated, 2012."},{"key":"e_1_2_1_37_1","volume-title":"Teradata","author":"Miller T.","year":"2016","unstructured":"T. Miller . Using R and Python in the Teradata Database. White paper , Teradata , 2016 . T. Miller. Using R and Python in the Teradata Database. White paper, Teradata, 2016."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484838.2484869"},{"key":"e_1_2_1_39_1","volume-title":"et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12(Oct):2825--2830","author":"Pedregosa F.","year":"2011","unstructured":"F. Pedregosa , G. Varoquaux , A. Gramfort , V. Michel , B. Thirion , O. Grisel , M. Blondel , P. Prettenhofer , R. Weiss , V. Dubourg , et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12(Oct):2825--2830 , 2011 . F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12(Oct):2825--2830, 2011."},{"key":"e_1_2_1_40_1","volume-title":"Data Science","author":"Piatetsky G.","year":"2017","unstructured":"G. Piatetsky . New Leader , Trends, and Surprises in Analytics , Data Science , Machine Learning Software Poll , 2017 . {https:\/\/www.kdnuggets.com\/2017\/05\/poll-analytics-data-science-machine-learning-software-leaders.html, accessed 07-December-2017}. G. Piatetsky. New Leader, Trends, and Surprises in Analytics, Data Science, Machine Learning Software Poll, 2017. {https:\/\/www.kdnuggets.com\/2017\/05\/poll-analytics-data-science-machine-learning-software-leaders.html, accessed 07-December-2017}."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/s003780050004"},{"key":"e_1_2_1_42_1","volume-title":"R: A Language and Environment for Statistical Computing","author":"Team R Core","year":"2014","unstructured":"R Core Team . R: A Language and Environment for Statistical Computing . R Foundation for Statistical Computing, Vienna , Austria , 2014 . R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2014."},{"key":"e_1_2_1_43_1","volume-title":"Voter Classification Using MonetDB\/Python","author":"Raasveldt M.","year":"2016","unstructured":"M. Raasveldt . Voter Classification Using MonetDB\/Python , 2016 . {https:\/\/www.monetdb.org\/blog\/voter-classification-using-monetdbpython, accessed 07-December-2017}. M. Raasveldt. Voter Classification Using MonetDB\/Python, 2016. {https:\/\/www.monetdb.org\/blog\/voter-classification-using-monetdbpython, accessed 07-December-2017}."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2949689.2949703"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.14778\/3115404.3115408"},{"key":"e_1_2_1_46_1","volume-title":"Evaluation of NoSQL and Array Databases for Scientific Applications. In DataCloud Workshop","author":"Ramakrishnan L.","year":"2013","unstructured":"L. Ramakrishnan , P. K. Mantha , Y. Yao , and R. S. Canon . Evaluation of NoSQL and Array Databases for Scientific Applications. In DataCloud Workshop , 2013 . L. Ramakrishnan, P. K. Mantha, Y. Yao, and R. S. Canon. Evaluation of NoSQL and Array Databases for Scientific Applications. In DataCloud Workshop, 2013."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742911"},{"key":"e_1_2_1_48_1","volume-title":"SAP SE","author":"SAP.","year":"2017","unstructured":"SAP. SAP Business Objects Web Intelligence User's Guide . SAP SE , 2017 . SAP. SAP Business Objects Web Intelligence User's Guide. SAP SE, 2017."},{"key":"e_1_2_1_49_1","volume-title":"Pearson Education","author":"Schneider R. D.","year":"2005","unstructured":"R. D. Schneider . MySQL Database Design and Tuning . Pearson Education , 2005 . R. D. Schneider. MySQL Database Design and Tuning. Pearson Education, 2005."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2013.19"},{"key":"e_1_2_1_51_1","volume-title":"PostgreSQL 10.0 Documentation","author":"The PostgreSQL Global Development Group","year":"2017","unstructured":"The PostgreSQL Global Development Group . Procedural Languages . In PostgreSQL 10.0 Documentation , 2017 . The PostgreSQL Global Development Group. Procedural Languages. In PostgreSQL 10.0 Documentation, 2017."},{"key":"e_1_2_1_52_1","volume-title":"May","author":"Team Theano Development","year":"2016","unstructured":"Theano Development Team . Theano: A Python Framework for Fast Computation of Mathematical Expressions. arXiv e-prints, abs\/1605.02688 , May 2016 . Theano Development Team. Theano: A Python Framework for Fast Computation of Mathematical Expressions. arXiv e-prints, abs\/1605.02688, May 2016."},{"key":"e_1_2_1_53_1","unstructured":"Transaction Processing Performance Council. TPC Benchmark H 2017. Transaction Processing Performance Council. TPC Benchmark H 2017."},{"key":"e_1_2_1_54_1","volume-title":"Direct and Inverse Solutions of Geodesics on the Ellipsoid with Application of Nested Equations. Survey Review, 23(176):88--93","author":"Vincenty T.","year":"1975","unstructured":"T. Vincenty . Direct and Inverse Solutions of Geodesics on the Ellipsoid with Application of Nested Equations. Survey Review, 23(176):88--93 , 1975 . T. Vincenty. Direct and Inverse Solutions of Geodesics on the Ellipsoid with Application of Nested Equations. Survey Review, 23(176):88--93, 1975."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2011.37"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3003665.3003669"},{"key":"e_1_2_1_57_1","volume-title":"Data Science with Microsoft SQL Server","author":"Woody B.","year":"2016","unstructured":"B. Woody , D. Dea , D. GuhaThakurta , G. Bansal , M. Conners , and T. Wee-Hyong . Data Science with Microsoft SQL Server 2016 . Microsoft Press , 2016. B. Woody, D. Dea, D. GuhaThakurta, G. Bansal, M. Conners, and T. Wee-Hyong. Data Science with Microsoft SQL Server 2016. Microsoft Press, 2016."},{"key":"e_1_2_1_58_1","volume-title":"Spark: Cluster Computing With Working Sets. HotCloud, 10(10--10):95","author":"Zaharia M.","year":"2010","unstructured":"M. Zaharia , M. Chowdhury , M. J. Franklin , S. Shenker , and I. Stoica . Spark: Cluster Computing With Working Sets. HotCloud, 10(10--10):95 , 2010 . M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. Spark: Cluster Computing With Working Sets. HotCloud, 10(10--10):95, 2010."},{"key":"e_1_2_1_59_1","volume-title":"CIDR","author":"Zhang Y.","year":"2009","unstructured":"Y. Zhang , H. Herodotou , and J. Yang . RIOT: I\/O-Efficient Numerical Computing without SQL . In CIDR , 2009 . Y. Zhang, H. Herodotou, and J. Yang. RIOT: I\/O-Efficient Numerical Computing without SQL. In CIDR, 2009."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463684"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2010.5447819"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3236187.3236194","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,10]],"date-time":"2024-07-10T04:28:13Z","timestamp":1720585693000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3236187.3236194"}},"subtitle":["abstraction for advanced in-database analytics"],"short-title":[],"issued":{"date-parts":[[2018,7]]},"references-count":61,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2018,7]]}},"alternative-id":["10.14778\/3236187.3236194"],"URL":"https:\/\/doi.org\/10.14778\/3236187.3236194","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2018,7]]}}}