{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,10]],"date-time":"2026-05-10T18:21:03Z","timestamp":1778437263765,"version":"3.51.4"},"reference-count":72,"publisher":"Association for Computing Machinery (ACM)","issue":"5","funder":[{"DOI":"10.13039\/501100001824","name":"Czech Science Foundation","doi-asserted-by":"crossref","award":["24-11664S"],"award-info":[{"award-number":["24-11664S"]}],"id":[{"id":"10.13039\/501100001824","id-type":"DOI","asserted-by":"crossref"}]},{"name":"European Union\u2019s Sponsor Horizon Europe Research and Innovation Program","award":["#101070149"],"award-info":[{"award-number":["#101070149"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2025,10,31]]},"abstract":"<jats:p>\n            Transformer models have continuously expanded into all machine learning domains convertible to the underlying sequence-to-sequence representation, including tabular data. However, while ubiquitous, this representation restricts its extension to the more general case of\n            <jats:italic toggle=\"yes\">relational databases<\/jats:italic>\n            . In this article, we introduce a modular neural message-passing scheme that closely adheres to the formal relational model, enabling direct end-to-end learning of tabular transformers from database storage systems. We address the associated challenges of appropriate learning data representation and loading, which are critical in the database setting, and compare our approach against a number of representative models from various related fields across a significantly wide range of datasets. Our results then demonstrate superior performance of this newly proposed class of neural architectures.\n          <\/jats:p>","DOI":"10.1145\/3749991","type":"journal-article","created":{"date-parts":[[2025,7,22]],"date-time":"2025-07-22T22:20:28Z","timestamp":1753222828000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Tabular Transformers Meet Relational Databases"],"prefix":"10.1145","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-8561-8106","authenticated-orcid":false,"given":"Jakub","family":"Pele\u0161ka","sequence":"first","affiliation":[{"name":"Czech Technical University, Prague, Czech Republic"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6964-4232","authenticated-orcid":false,"given":"Gustav","family":"\u0160\u00edr","sequence":"additional","affiliation":[{"name":"Czech Technical University, Prague, Czech Republic"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,9,18]]},"reference":[{"key":"e_1_3_3_2_2","unstructured":"David Aha. 1987. UCI Machine Learning Repository. Retrieved from https:\/\/archive.ics.uci.edu"},{"key":"e_1_3_3_3_2","doi-asserted-by":"crossref","unstructured":"Takuya Akiba Shotaro Sano Toshihiko Yanase Takeru Ohta and Masanori Koyama. 2019. Optuna: A next-generation hyperparameter optimization framework. arXiv:1907.10902. Retrieved from https:\/\/arxiv.org\/abs\/1907.10902","DOI":"10.1145\/3292500.3330701"},{"key":"e_1_3_3_4_2","unstructured":"Sercan O. Arik and Tomas Pfister. 2020. TabNet: Attentive interpretable tabular learning. arXiv:1908.07442. Retrieved from https:\/\/arxiv.org\/abs\/1908.07442"},{"key":"e_1_3_3_5_2","unstructured":"Jimmy Lei Ba Jamie Ryan Kiros and Geoffrey E. Hinton. 2016. Layer normalization. arXiv:1607.06450. Retrieved from https:\/\/arxiv.org\/abs\/1607.06450"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00544"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3449980"},{"key":"e_1_3_3_8_2","unstructured":"Jonathan Berant Daniel Deutch Amir Globerson Tova Milo and Tomer Wolfson. 2018. Explaining queries over web tables to non-experts. arXiv:1808.04614. Retrieved from https:\/\/arxiv.org\/abs\/1808.04614"},{"key":"e_1_3_3_9_2","first-page":"249","volume-title":"Proceedings of the 1974 ACM SIGFIDET (Now SIGMOD) Workshop on Data Description, Access and Control (SIGFIDET \u201974)","author":"Chamberlin Donald D.","year":"1974","unstructured":"Donald D. Chamberlin and Raymond F. Boyce. 1974. SEQUEL: A structured English query language. In Proceedings of the 1974 ACM SIGFIDET (Now SIGMOD) Workshop on Data Description, Access and Control (SIGFIDET \u201974). ACM, New York, NY, 249\u2013264."},{"key":"e_1_3_3_10_2","unstructured":"Jintai Chen Jiahuan Yan Danny Ziyi Chen and Jian Wu. 2023. ExcelFormer: A neural network surpassing GBDTs on tabular data. arXiv:2301.02819. Retrieved from https:\/\/arxiv.org\/abs\/2301.02819"},{"key":"e_1_3_3_11_2","unstructured":"Kuan-Yu Chen Ping-Han Chiang Hsin-Rung Chou Ting-Wei Chen and Tien-Hao Chang. 2023. Trompt: Towards a better deep neural network for tabular data. arXiv:2305.18446. Retrieved from https:\/\/arxiv.org\/abs\/2305.18446"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.5555\/77708"},{"key":"e_1_3_3_14_2","unstructured":"Code17 GmbH. 2024. getML. Retrieved from https:\/\/getml.com"},{"key":"e_1_3_3_15_2","doi-asserted-by":"crossref","unstructured":"Andrew Cropper Sebastijan Duman\u010di\u0107 and Stephen H. Muggleton. 2020. Turning 30: New ideas in inductive logic programming. arXiv:2002.11002. Retrieved from https:\/\/arxiv.org\/abs\/2002.11002","DOI":"10.24963\/ijcai.2020\/673"},{"key":"e_1_3_3_16_2","unstructured":"Milan Cvitkovic. 2020. Supervised learning on relational databases with graph neural networks. arXiv:2002.02046. Retrieved from https:\/\/arxiv.org\/abs\/2002.02046"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-68856-3"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3542700.3542709"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33013558"},{"key":"e_1_3_3_20_2","volume-title":"Proceedings of the 41st International Conference on Machine Learning","author":"Fey Matthias","year":"2024","unstructured":"Matthias Fey, Weihua Hu, Kexin Huang, Jan Eric Lenssen, Rishabh Ranjan, Joshua Robinson, Rex Ying, Jiaxuan You, and Jure Leskovec. 2024. Position: Relational deep Learning\u2014Graph representation learning on relational databases. In Proceedings of the 41st International Conference on Machine Learning."},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.5555\/2558577.2558594"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1013203451"},{"key":"e_1_3_3_23_2","volume-title":"Logic for Computer Science: Foundations of Automatic Theorem Proving","author":"Gallier Jean H.","year":"2015","unstructured":"Jean H. Gallier. 2015. Logic for Computer Science: Foundations of Automatic Theorem Proving. Courier Dover Publications."},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3570991.3571041"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.5555\/1296231"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10710-017-9314-z"},{"key":"e_1_3_3_27_2","volume-title":"Information Modeling and Relational Databases","author":"Halpin Terry","year":"2010","unstructured":"Terry Halpin and Tony Morgan. 2010. Information Modeling and Relational Databases. Morgan Kaufmann."},{"key":"e_1_3_3_28_2","first-page":"1024","article-title":"Inductive representation learning on large graphs","author":"Hamilton Will","year":"2017","unstructured":"Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, 1024\u20131034.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.5555\/1564770"},{"key":"e_1_3_3_30_2","unstructured":"Benjamin Hilprecht Kristian Kersting and Carsten Binnig. 2023. SPARE: A single-pass neural model for relational databases. arXiv:2310.13581. Retrieved from https:\/\/arxiv.org\/abs\/2310.13581"},{"key":"e_1_3_3_31_2","unstructured":"Weihua Hu Matthias Fey Marinka Zitnik Yuxiao Dong Hongyu Ren Bowen Liu Michele Catasta and Jure Leskovec. 2021. Open graph benchmark: Datasets for machine learning on graphs. arXiv:2005.00687. Retrieved from https:\/\/arxiv.org\/abs\/2005.00687"},{"key":"e_1_3_3_32_2","unstructured":"Weihua Hu Yiwen Yuan Zecheng Zhang Akihiro Nitta Kaidi Cao Vid Kocijan Jure Leskovec and Matthias Fey. 2024. PyTorch frame: A modular framework for multi-modal tabular learning. arXiv:2404.00776. Retrieved from https:\/\/arxiv.org\/abs\/2404.00776"},{"key":"e_1_3_3_33_2","unstructured":"Xuanwei Hu Weijing Tang Cheng-Kang Hsieh and Shuaiwen Shi. 2020. TabTransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678. Retrieved from https:\/\/arxiv.org\/abs\/2012.06678"},{"key":"e_1_3_3_34_2","unstructured":"Ziniu Hu Yuxiao Dong Kuansan Wang and Yizhou Sun. 2020. Heterogeneous graph transformer. arXiv:2003.01332. Retrieved from https:\/\/arxiv.org\/abs\/2003.01332"},{"key":"e_1_3_3_35_2","unstructured":"Xin Huang Ashish Khetan Milan Cvitkovic and Zohar S. Karnin. 2020. TabTransformer: Tabular data modeling using contextual embeddings. arXiv:2012.06678. Retrieved from https:\/\/arxiv.org\/abs\/2012.06678"},{"key":"e_1_3_3_36_2","unstructured":"Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167. Retrieved from https:\/\/arxiv.org\/abs\/1502.03167"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-7138-7"},{"key":"e_1_3_3_38_2","unstructured":"A. Johnson L. Bulgarelli T. Pollard S. Horng L. A. Celi and R. Mark. 2020. MIMIC-IV (Version 1.0). PhysioNet. Retrieved from https:\/\/doi.org\/10.13026\/s6n6-xd98"},{"key":"e_1_3_3_39_2","first-page":"23928","article-title":"Well-tuned simple nets excel on tabular datasets","volume":"34","author":"Kadra Arlind","year":"2021","unstructured":"Arlind Kadra, Marius Lindauer, Frank Hutter, and Josif Grabocka. 2021. Well-tuned simple nets excel on tabular datasets. In Advances in Neural Information Processing Systems, Vol. 34, 23928\u201323941.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-014-5443-2"},{"key":"e_1_3_3_41_2","unstructured":"Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980. Retrieved from https:\/\/arxiv.org\/abs\/1412.6980"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-04599-2_11"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-39917-9_14"},{"key":"e_1_3_3_44_2","unstructured":"Jure Leskovec and Andrej Krevl. 2014. SNAP Datasets: Stanford Large Network Dataset Collection. Retrieved from http:\/\/snap.stanford.edu\/data"},{"key":"e_1_3_3_45_2","unstructured":"David Lindner J\u00e1nos Kram\u00e1r Matthew Rahtz Thomas McGrath and Vladimir Mikulik. 2023. Tracr: Compiled transformers as a laboratory for interpretability. arXiv:2301.05062. Retrieved from https:\/\/arxiv.org\/abs\/2301.05062"},{"key":"e_1_3_3_46_2","unstructured":"Shengchao Liu David Vazquez Jian Tang and Pierre-Andre Noel. 2022. Flaky performances when pre-training on relational databases with a plan for future characterization efforts. In Proceedings of the 1st Workshop on Pre-Training: Perspectives Pitfalls and Paths Forward at ICML 2022."},{"key":"e_1_3_3_47_2","unstructured":"Philipp Moritz Robert Nishihara Stephanie Wang Alexey Tumanov Richard Liaw Eric Liang Melih Elibol Zongheng Yang William Paul Michael I. Jordan et al. 2018. Ray: A distributed framework for emerging AI applications. arXiv:1712.05889. Retrieved from https:\/\/arxiv.org\/abs\/1712.05889"},{"key":"e_1_3_3_48_2","unstructured":"Jan Motl and Oliver Schulte. 2015. The CTU Prague relational learning repository. arXiv:1511.03086. Retrieved from https:\/\/arxiv.org\/abs\/1511.03086"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/0743-1066(94)90035-3"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-011-5244-9"},{"key":"e_1_3_3_51_2","first-page":"14501","article-title":"Recipe for a general, powerful, scalable graph transformer","volume":"35","author":"Ramp\u00e1\u0161ek Ladislav","year":"2022","unstructured":"Ladislav Ramp\u00e1\u0161ek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. 2022. Recipe for a general, powerful, scalable graph transformer. In Advances in Neural Information Processing Systems, Vol. 35, 14501\u201314515.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_52_2","doi-asserted-by":"crossref","unstructured":"Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. arXiv:1908.10084. Retrieved from https:\/\/arxiv.org\/abs\/1908.10084","DOI":"10.18653\/v1\/D19-1410"},{"key":"e_1_3_3_53_2","volume-title":"Proceedings of the 38th Conference on Neural Information Processing Systems Datasets and Benchmarks Track","author":"Robinson Joshua","year":"2024","unstructured":"Joshua Robinson, Rishabh Ranjan, Weihua Hu, Kexin Huang, Jiaqi Han, Alejandro Dobles, Matthias Fey, Jan Eric Lenssen, Yiwen Yuan, Zecheng Zhang, et al. 2024. RelBench: A benchmark for deep learning on relational databases. In Proceedings of the 38th Conference on Neural Information Processing Systems Datasets and Benchmarks Track."},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-1309"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2008.2005605"},{"key":"e_1_3_3_56_2","doi-asserted-by":"crossref","unstructured":"Michael Schlichtkrull Thomas N. Kipf Peter Bloem Rianne Van Den Berg Ivan Titov and Max Welling. 2018. Modeling relational data with graph convolutional networks. In Proceedings of the European Semantic Web Conference. Springer 593\u2013607.","DOI":"10.1007\/978-3-319-93417-4_38"},{"key":"e_1_3_3_57_2","doi-asserted-by":"crossref","unstructured":"Luciano Serafini and Artur S. d\u2019Avila Garcez. 2016. Learning and reasoning with logic tensor networks. In AI*IA 2016 Advances in Artificial Intelligence. Giovanni Adorni Stefano Cagnoni Marco Gori and Marco Maratea (Eds.) Lecture Notes in Computer Science Vol. 10037 334\u2013348.","DOI":"10.1007\/978-3-319-49130-1_25"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.11.011"},{"key":"e_1_3_3_59_2","unstructured":"Gowthami Somepalli Micah Goldblum Avi Schwarzschild C. Bayan Bruss and Tom Goldstein. 2021. SAINT: Improved neural networks for tabular data via row attention and contrastive pre-training. arXiv:2106.01342. Retrieved from https:\/\/arxiv.org\/abs\/2106.01342"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.11203"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.14778\/3402707.3402736"},{"key":"e_1_3_3_62_2","unstructured":"The Alteryx. 2023. Featuretools. Retrieved from https:\/\/www.featuretools.com"},{"key":"e_1_3_3_63_2","article-title":"Attention is all you need","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_64_2","unstructured":"Liane Vogel Jan-Micha Bodensohn and Carsten Binnig. 2024. WikiDBs: A large-scale corpus of relational databases from Wikidata. In Proceedings of the 38th Conference on Neural Information Processing Systems Datasets and Benchmarks Track."},{"key":"e_1_3_3_65_2","unstructured":"Liane Vogel Benjamin Hilprecht and Carsten Binnig. 2023. Towards foundation models for relational databases [Vision Paper]. arXiv:2305.15321. Retrieved from https:\/\/arxiv.org\/abs\/2305.15321"},{"key":"e_1_3_3_66_2","first-page":"27236","article-title":"4DBInfer: A 4D benchmarking toolbox for graph-centric predictive modeling on RDBs","volume":"37","author":"Wang Minjie","year":"2024","unstructured":"Minjie Wang, Quan Gan, David Wipf, Zhenkun Cai, Ning Li, Jianheng Tang, Yanlin Zhang, Zizhao Zhang, Zunyao Mao, and Yakun Song. 2024. 4DBInfer: A 4D benchmarking toolbox for graph-centric predictive modeling on RDBs. In Advances in Neural Information Processing Systems 37 (Dec. 2024), 27236\u201327273","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2017.2754499"},{"key":"e_1_3_3_68_2","first-page":"4","article-title":"A comprehensive survey on graph neural networks","author":"Wu Zonghan","year":"2020","unstructured":"Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S. Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems 32, 1 (2020), 4\u201324.","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_3_3_69_2","volume-title":"Advances in Neural Information Processing Systems","author":"Yun Seongjun","year":"2019","unstructured":"Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J. Kim. 2019. Graph transformer networks. In Advances in Neural Information Processing Systems, Vol. 32, Curran Associates, Inc."},{"issue":"4","key":"e_1_3_3_70_2","first-page":"39","article-title":"Accelerating the machine learning lifecycle with MLflow","volume":"41","author":"Zaharia Matei","year":"2018","unstructured":"Matei Zaharia, Andrew Chen, Aaron Davidson, Ali Ghodsi, Sue Ann Hong, Andy Konwinski, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, and Mani Parkhe. 2018. Accelerating the machine learning lifecycle with MLflow. IEEE Data Engineering Bulletin 41, 4 (2018), 39\u201345.","journal-title":"IEEE Data Engineering Bulletin"},{"key":"e_1_3_3_71_2","unstructured":"Luk\u00e1\u0161 Zahradn\u00edk Jan Neumann and Gustav \u0160\u00edr. 2023. A deep learning blueprint for relational databases. In Proceedings of the NeurIPS 2023 2nd Table Representation Learning Workshop."},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330961"},{"key":"e_1_3_3_73_2","unstructured":"Han Zhang Quan Gan David Wipf and Weinan Zhang. 2023. Gfs: Graph-based feature synthesis for prediction over relational databases. arXiv:2312.02037. Retrieved from https:\/\/arxiv.org\/abs\/2312.02037"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3749991","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,18]],"date-time":"2025-09-18T17:03:55Z","timestamp":1758215035000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3749991"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,18]]},"references-count":72,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,10,31]]}},"alternative-id":["10.1145\/3749991"],"URL":"https:\/\/doi.org\/10.1145\/3749991","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,18]]},"assertion":[{"value":"2025-02-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-06-27","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}