{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T22:12:47Z","timestamp":1766182367930,"version":"build-2065373602"},"reference-count":26,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2024,7,11]],"date-time":"2024-07-11T00:00:00Z","timestamp":1720656000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004281","name":"National Centre of Science (Poland)","doi-asserted-by":"publisher","award":["2021\/43\/B\/ST6\/02853","PLG\/2023\/016636"],"award-info":[{"award-number":["2021\/43\/B\/ST6\/02853","PLG\/2023\/016636"]}],"id":[{"id":"10.13039\/501100004281","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011089","name":"Polish high-performance computing infrastructure PLGrid (HPC Center: ACK Cyfronet AGH) for providing computer facilities and support within computational","doi-asserted-by":"publisher","award":["2021\/43\/B\/ST6\/02853","PLG\/2023\/016636"],"award-info":[{"award-number":["2021\/43\/B\/ST6\/02853","PLG\/2023\/016636"]}],"id":[{"id":"10.13039\/501100011089","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We introduce NodeFlow, a flexible framework for probabilistic regression on tabular data that combines Neural Oblivious Decision Ensembles (NODEs) and Conditional Continuous Normalizing Flows (CNFs). It offers improved modeling capabilities for arbitrary probabilistic distributions, addressing the limitations of traditional parametric approaches. In NodeFlow, the NODE captures complex relationships in tabular data through a tree-like structure, while the conditional CNF utilizes the NODE\u2019s output space as a conditioning factor. The training process of NodeFlow employs standard gradient-based learning, facilitating the end-to-end optimization of the NODEs and CNF-based density estimation. This approach ensures outstanding performance, ease of implementation, and scalability, making NodeFlow an appealing choice for practitioners and researchers. Comprehensive assessments on benchmark datasets underscore NodeFlow\u2019s efficacy, revealing its achievement of state-of-the-art outcomes in multivariate probabilistic regression setup and its strong performance in univariate regression tasks. Furthermore, ablation studies are conducted to justify the design choices of NodeFlow. In conclusion, NodeFlow\u2019s end-to-end training process and strong performance make it a compelling solution for practitioners and researchers. Additionally, it opens new avenues for research and application in the field of probabilistic regression on tabular data.<\/jats:p>","DOI":"10.3390\/e26070593","type":"journal-article","created":{"date-parts":[[2024,7,11]],"date-time":"2024-07-11T15:59:48Z","timestamp":1720713588000},"page":"593","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["NodeFlow: Towards End-to-End Flexible Probabilistic Regression on Tabular Data"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2579-8293","authenticated-orcid":false,"given":"Patryk","family":"Wielopolski","sequence":"first","affiliation":[{"name":"Department of Artificial Intelligence, Wroc\u0142aw University of Science and Technology, 50-370 Wroc\u0142aw, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Oleksii","family":"Furman","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence, Wroc\u0142aw University of Science and Technology, 50-370 Wroc\u0142aw, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4217-7712","authenticated-orcid":false,"given":"Maciej","family":"Zi\u0119ba","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence, Wroc\u0142aw University of Science and Technology, 50-370 Wroc\u0142aw, Poland"},{"name":"Tooploox Ltd., 53-601 Wroc\u0142aw, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,11]]},"reference":[{"unstructured":"Borisov, V., Leemann, T., Se\u00dfler, K., Haug, J., Pawelczyk, M., and Kasneci, G. (2021). Deep Neural Networks and Tabular Data: A Survey. arXiv.","key":"ref_1"},{"unstructured":"Grinsztajn, L., Oyallon, E., and Varoquaux, G. (December, January 28). Why do tree-based models still outperform deep learning on typical tabular data?. Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA.","key":"ref_2"},{"doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","key":"ref_3","DOI":"10.1145\/2939672.2939785"},{"unstructured":"Prokhorenkova, L.O., Gusev, G., Vorobev, A., Dorogush, A.V., and Gulin, A. (2018, January 3\u20138). CatBoost: Unbiased boosting with categorical features. Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, Montr\u00e9al, QC, Canada.","key":"ref_4"},{"unstructured":"Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T. (2017, January 4\u20139). LightGBM: A Highly Efficient Gradient Boosting Decision Tree. Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, CA, USA.","key":"ref_5"},{"unstructured":"Popov, S., Morozov, S., and Babenko, A. (2020, January 26\u201330). Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data. Proceedings of the 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia.","key":"ref_6"},{"unstructured":"Abutbul, A., Elidan, G., Katzir, L., and El-Yaniv, R. (2020). DNF-Net: A Neural Architecture for Tabular Data. arXiv.","key":"ref_7"},{"unstructured":"Ranzato, M., Beygelzimer, A., Dauphin, Y.N., Liang, P., and Vaughan, J.W. (2021, January 6\u201314). Revisiting Deep Learning Models for Tabular Data. Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, Virtual.","key":"ref_8"},{"unstructured":"Duan, T., Anand, A., Ding, D.Y., Thai, K.K., Basu, S., Ng, A.Y., and Schuler, A. (2020, January 13\u201318). NGBoost: Natural Gradient Boosting for Probabilistic Prediction. Proceedings of the 37th International Conference on Machine Learning, ICML 2020, PMLR, Virtual Event.","key":"ref_9"},{"doi-asserted-by":"crossref","unstructured":"Sprangers, O., Schelter, S., and de Rijke, M. (2021, January 14\u201318). Probabilistic Gradient Boosting Machines for Large-Scale Probabilistic Regression. Proceedings of the KDD \u201921: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore.","key":"ref_10","DOI":"10.1145\/3447548.3467278"},{"unstructured":"Malinin, A., Prokhorenkova, L., and Ustimenko, A. (2021, January 3\u20137). Uncertainty in Gradient Boosting via Ensembles. Proceedings of the 9th International Conference on Learning Representations, ICLR 2021, Virtual Event.","key":"ref_11"},{"key":"ref_12","first-page":"2631","article-title":"TreeFlow: Going Beyond Tree-Based Parametric Probabilistic Regression","volume":"Volume 372","author":"Wielopolski","year":"2023","journal-title":"ECAI 2023"},{"unstructured":"Ren, L., Sun, G., and Wu, J. (2019). RoNGBa: A Robustly Optimized Natural Gradient Boosting Training Approach with Leaf Number Clipping. arXiv.","key":"ref_13"},{"unstructured":"Arik, S.\u00d6., and Pfister, T. (2021, January 2\u20139). TabNet: Attentive Interpretable Tabular Learning. Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event.","key":"ref_14"},{"unstructured":"Somepalli, G., Goldblum, M., Schwarzschild, A., Bruss, C.B., and Goldstein, T. (2021). SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training. arXiv.","key":"ref_15"},{"unstructured":"Lakshminarayanan, B., Pritzel, A., and Blundell, C. (2017, January 4\u20139). Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, CA, USA.","key":"ref_16"},{"key":"ref_17","first-page":"1050","article-title":"Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning","volume":"Volume 48","author":"Balcan","year":"2016","journal-title":"JMLR Workshop and Conference Proceedings, Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York, NY, USA, 19\u201324 June 2016"},{"key":"ref_18","first-page":"1861","article-title":"Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks","volume":"Volume 37","author":"Bach","year":"2016","journal-title":"JMLR Workshop and Conference Proceedings, Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6\u201311 July 2015"},{"unstructured":"Korhonen, A., Traum, D.R., and M\u00e0rquez, L. (August, January 28). Sparse Sequence-to-Sequence Models. Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy. Volume 1: Long Papers;.","key":"ref_19"},{"doi-asserted-by":"crossref","unstructured":"Yang, G., Huang, X., Hao, Z., Liu, M., Belongie, S.J., and Hariharan, B. (November, January 27). PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Republic of Korea.","key":"ref_20","DOI":"10.1109\/ICCV.2019.00464"},{"unstructured":"Sendera, M., Tabor, J., Nowak, A., Bedychaj, A., Patacchiola, M., Trzcinski, T., Spurek, P., and Zieba, M. (2021, January 6\u201314). Non-Gaussian Gaussian Processes for Few-Shot Regression. Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, Virtual.","key":"ref_21"},{"unstructured":"Grathwohl, W., Chen, R.T.Q., Bettencourt, J., Sutskever, I., and Duvenaud, D. (2019, January 6\u20139). FFJORD: Free-Form Continuous Dynamics for Scalable Reversible Generative Models. Proceedings of the 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA.","key":"ref_22"},{"unstructured":"Carlisle, M. (2023, October 07). Racist Data Destruction?. Available online: https:\/\/medium.com\/@docintangible\/racist-data-destruction-113e3eff54a8.","key":"ref_23"},{"doi-asserted-by":"crossref","unstructured":"McInnes, L., and Healy, J. (2018). UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv.","key":"ref_24","DOI":"10.21105\/joss.00861"},{"key":"ref_25","first-page":"185:1","article-title":"Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization","volume":"18","author":"Li","year":"2017","journal-title":"J. Mach. Learn. Res."},{"unstructured":"Hutter, F., Hoos, H.H., and Leyton-Brown, K. (2014, January 21\u201326). An Efficient Approach for Assessing Hyperparameter Importance. Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China. JMLR Workshop and Conference Proceedings.","key":"ref_26"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/7\/593\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:13:20Z","timestamp":1760109200000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/7\/593"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,11]]},"references-count":26,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["e26070593"],"URL":"https:\/\/doi.org\/10.3390\/e26070593","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2024,7,11]]}}}