{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,6]],"date-time":"2026-04-06T10:17:55Z","timestamp":1775470675180,"version":"3.50.1"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T00:00:00Z","timestamp":1747699200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T00:00:00Z","timestamp":1747699200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int. J. Mach. Learn. &amp; Cyber."],"published-print":{"date-parts":[[2025,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>This paper explores the use of XGBoost for composite quantile regression. XGBoost is a highly popular model renowned for its flexibility, efficiency, and capability to deal with missing data. The optimization uses a second order approximation of the loss function, complicating the use of loss functions with a zero or vanishing second derivative. Quantile regression\u2014a popular approach to obtain conditional quantiles when point estimates alone are insufficient\u2014unfortunately uses such a loss function, the pinball loss. Existing workarounds are typically inefficient and can result in severe quantile crossings. In this paper, we present a smooth approximation of the pinball loss, the arctan pinball loss, that is tailored to the needs of XGBoost. Specifically, contrary to other smooth approximations, the arctan pinball loss has a relatively large second derivative, which makes it more suitable to use in the second order approximation. Using this loss function enables the simultaneous prediction of multiple quantiles, which is more efficient and results in far fewer quantile crossings.<\/jats:p>","DOI":"10.1007\/s13042-025-02671-4","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T11:21:55Z","timestamp":1747740115000},"page":"7575-7589","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Composite quantile regression with XGBoost using the novel arctan pinball loss"],"prefix":"10.1007","volume":"16","author":[{"given":"Laurens","family":"Sluijterman","sequence":"first","affiliation":[]},{"given":"Frank","family":"Kreuwel","sequence":"additional","affiliation":[]},{"given":"Eric","family":"Cator","sequence":"additional","affiliation":[]},{"given":"Tom","family":"Heskes","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,5,20]]},"reference":[{"issue":"1","key":"2671_CR1","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1214\/10-AOS827","volume":"39","author":"A Belloni","year":"2011","unstructured":"Belloni A, Chernozhukov V (2011) $$\\ell$$1-penalized quantile regression in high-dimensional sparse models. Ann Stat 39(1):82\u2013130","journal-title":"Ann Stat"},{"issue":"9","key":"2671_CR2","doi-asserted-by":"publisher","first-page":"1277","DOI":"10.1016\/j.cageo.2010.07.005","volume":"37","author":"AJ Cannon","year":"2011","unstructured":"Cannon AJ (2011) Quantile regression neural networks: implementation in R and application to precipitation downscaling. Comput Geosci 37(9):1277\u20131284","journal-title":"Comput Geosci"},{"key":"2671_CR3","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1214\/aos\/1176348119","volume":"19","author":"P Chaudhuri","year":"1991","unstructured":"Chaudhuri P (1991) Nonparametric estimates of regression quantiles and their local Bahadur representation. Ann Stat 19:2","journal-title":"Ann Stat"},{"key":"2671_CR4","doi-asserted-by":"crossref","unstructured":"Chen T, Guestrin C (2016) XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD \u201916, pages 785\u2013794, New York, NY, USA. Association for Computing Machinery","DOI":"10.1145\/2939672.2939785"},{"issue":"s1","key":"2671_CR5","doi-asserted-by":"publisher","first-page":"S50","DOI":"10.1111\/j.1368-423X.2008.00274.x","volume":"12","author":"X Chen","year":"2009","unstructured":"Chen X, Koenker R, Xiao Z (2009) Copula-based nonlinear quantile autoregression. Economet J 12(s1):S50\u2013S67","journal-title":"Economet J"},{"issue":"2","key":"2671_CR6","doi-asserted-by":"publisher","first-page":"491","DOI":"10.1016\/j.jeconom.2005.02.009","volume":"132","author":"V Chernozhukov","year":"2006","unstructured":"Chernozhukov V, Hansen C (2006) Instrumental quantile regression inference for structural and treatment effect models. J Econ 132(2):491\u2013525","journal-title":"J Econ"},{"issue":"1\/2","key":"2671_CR7","first-page":"12","volume":"32","author":"MH DeGroot","year":"1983","unstructured":"DeGroot MH, Fienberg SE (1983) The comparison and evaluation of forecasters. J R Stat Soc Ser D 32(1\/2):12\u201322","journal-title":"J R Stat Soc Ser D"},{"issue":"7","key":"2671_CR8","doi-asserted-by":"publisher","first-page":"149","DOI":"10.3390\/info9070149","volume":"9","author":"SS Dhaliwal","year":"2018","unstructured":"Dhaliwal SS, Nahid A-A, Abbas R (2018) Effective intrusion detection system Using XGBoost. Information 9(7):149","journal-title":"Information"},{"key":"2671_CR9","unstructured":"Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, pages 1050\u20131059"},{"issue":"477","key":"2671_CR10","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1198\/016214506000001437","volume":"102","author":"T Gneiting","year":"2007","unstructured":"Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J Am Stat Assoc 102(477):359\u2013378","journal-title":"J Am Stat Assoc"},{"key":"2671_CR11","doi-asserted-by":"crossref","unstructured":"Gumus M, Kiran MS (2017) Crude oil price forecasting using XGBoost. In 2017 International Conference on Computer Science and Engineering (UBMK), pages 1100\u20131103, Antalya. IEEE","DOI":"10.1109\/UBMK.2017.8093500"},{"key":"2671_CR12","unstructured":"Guo C, Pleiss G, Sun Y, Weinberger KQ (2017) On calibration of modern neural networks. In International Conference on Machine Learning, pages 1321\u20131330. PMLR"},{"key":"2671_CR13","unstructured":"Hatalis K, Lamadrid AJ, Scheinberg K, Kishore S (2019) A novel smoothed loss and penalty function for noncrossing composite quantile estimation via deep neural networks. arXiv preprint arXiv:1909.12122"},{"key":"2671_CR14","unstructured":"Hern\u00e1ndez-Lobato JM, Adams R (2015) Probabilistic backpropagation for scalable learning of Bayesian neural networks. In International Conference on Machine Learning, pages 1861\u20131869"},{"key":"2671_CR15","doi-asserted-by":"publisher","first-page":"58392","DOI":"10.1109\/ACCESS.2020.2982418","volume":"8","author":"H Jiang","year":"2020","unstructured":"Jiang H, He Z, Ye G, Zhang H (2020) Network intrusion detection based on PSO-Xgboost Model. IEEE Access 8:58392\u201358401","journal-title":"IEEE Access"},{"key":"2671_CR16","doi-asserted-by":"crossref","unstructured":"Koenker R (2017) Quantile regression 40 years on. Technical report, The IFS","DOI":"10.1920\/wp.cem.2017.3617"},{"key":"2671_CR17","doi-asserted-by":"publisher","first-page":"33","DOI":"10.2307\/1913643","volume":"8","author":"R Koenker","year":"1978","unstructured":"Koenker R, Bassett G Jr (1978) Regression quantiles. Econometrica 8:33\u201350","journal-title":"Econometrica"},{"issue":"475","key":"2671_CR18","doi-asserted-by":"publisher","first-page":"980","DOI":"10.1198\/016214506000000672","volume":"101","author":"R Koenker","year":"2006","unstructured":"Koenker R, Xiao Z (2006) Quantile autoregression. J Am Stat Assoc 101(475):980\u2013990","journal-title":"J Am Stat Assoc"},{"key":"2671_CR19","unstructured":"Kuleshov V, Fenner N, Ermon S (2018) Accurate uncertainties for deep learning using calibrated regression. In Proceedings of the 35th International Conference on Machine Learning, pages 2796\u20132804. PMLR"},{"key":"2671_CR20","first-page":"6402","volume":"8","author":"B Lakshminarayanan","year":"2017","unstructured":"Lakshminarayanan B, Pritzel A, Blundell C (2017) Simple and scalable predictive uncertainty estimation using deep ensembles. Adv Neural Inf Proces Syst 8:6402\u20136413","journal-title":"Adv Neural Inf Proces Syst"},{"issue":"1","key":"2671_CR21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1017\/S0266466603191013","volume":"19","author":"S Lee","year":"2003","unstructured":"Lee S (2003) Efficient semiparametric estimation of a partially linear quantile regression model. Economet Theor 19(1):1\u201331","journal-title":"Economet Theor"},{"key":"2671_CR22","doi-asserted-by":"publisher","first-page":"8","DOI":"10.3389\/fgene.2019.01077","volume":"10","author":"W Li","year":"2019","unstructured":"Li W, Yin Y, Quan X, Zhang H (2019) Gene expression value prediction based on XGBoost algorithm. Front Genet 10:8","journal-title":"Front Genet"},{"key":"2671_CR23","doi-asserted-by":"publisher","DOI":"10.1016\/j.jhydrol.2021.126382","volume":"598","author":"M Ma","year":"2021","unstructured":"Ma M, Zhao G, He B, Li Q, Dong H, Wang S, Wang Z (2021) XGBoost-based method for flash flood risk assessment. J Hydrol 598:126382","journal-title":"J Hydrol"},{"key":"2671_CR24","first-page":"983","volume":"7","author":"N Meinshausen","year":"2006","unstructured":"Meinshausen N (2006) Quantile regression forests. J Mach Learn Res 7:983\u2013999","journal-title":"J Mach Learn Res"},{"issue":"1","key":"2671_CR25","first-page":"41","volume":"26","author":"AH Murphy","year":"1977","unstructured":"Murphy AH, Winkler RL (1977) Reliability of subjective probability forecasts of precipitation and temperature. J Roy Stat Soc: Ser C (Appl Stat) 26(1):41\u201347","journal-title":"J Roy Stat Soc: Ser C (Appl Stat)"},{"key":"2671_CR26","doi-asserted-by":"crossref","unstructured":"Niculescu-Mizil A, Caruana R (2005) Predicting good probabilities with supervised learning. In Proceedings of the 22nd International Conference on Machine Learning, pages 625\u2013632","DOI":"10.1145\/1102351.1102430"},{"key":"2671_CR27","doi-asserted-by":"crossref","unstructured":"Nix DA, Weigend AS (1994) Estimating the mean and variance of the target probability distribution. In Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN\u201994), volume\u00a01, pages 55\u201360. IEEE","DOI":"10.1109\/ICNN.1994.374138"},{"key":"2671_CR28","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1016\/j.eswa.2019.01.083","volume":"125","author":"J Nobre","year":"2019","unstructured":"Nobre J, Neves RF (2019) Combining principal component analysis, discrete wavelet transform and XGBoost to trade in the financial markets. Expert Syst Appl 125:181\u2013194","journal-title":"Expert Syst Appl"},{"issue":"6","key":"2671_CR29","doi-asserted-by":"publisher","first-page":"2131","DOI":"10.1109\/TCBB.2019.2911071","volume":"17","author":"A Ogunleye","year":"2020","unstructured":"Ogunleye A, Wang Q-G (2020) XGBoost model for chronic kidney disease diagnosis. IEEE\/ACM Trans Comput Biol Bioinf 17(6):2131\u20132140","journal-title":"IEEE\/ACM Trans Comput Biol Bioinf"},{"issue":"1","key":"2671_CR30","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1016\/0304-4076(86)90016-3","volume":"32","author":"JL Powell","year":"1986","unstructured":"Powell JL (1986) Censored regression quantiles. J Econ 32(1):143\u2013155","journal-title":"J Econ"},{"key":"2671_CR31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2021\/2577375","volume":"2021","author":"S Ramaneswaran","year":"2021","unstructured":"Ramaneswaran S, Srinivasan K, Vincent PMDR, Chang C-Y (2021) Hybrid Inception v3 XGBoost model for acute lymphoblastic leukemia classification. Comput Math Methods Med 2021:1\u201310","journal-title":"Comput Math Methods Med"},{"key":"2671_CR32","unstructured":"Romano Y, Patterson E, Candes E (2019) Conformalized quantile regression. In Advances in Neural Information Processing Systems, volume\u00a032. Curran Associates, Inc"},{"issue":"2","key":"2671_CR33","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1023\/A:1022648800760","volume":"5","author":"RE Schapire","year":"1990","unstructured":"Schapire RE (1990) The strength of weak learnability. Mach Learn 5(2):197\u2013227","journal-title":"Mach Learn"},{"key":"2671_CR34","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2024.106203","volume":"173","author":"L Sluijterman","year":"2024","unstructured":"Sluijterman L, Cator E, Heskes T (2024) How to evaluate uncertainty estimates in machine learning for regression? Neural Netw 173:106203","journal-title":"Neural Netw"},{"key":"2671_CR35","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1016\/j.eswa.2017.01.054","volume":"76","author":"Q Xu","year":"2017","unstructured":"Xu Q, Deng K, Jiang C, Sun F, Huang X (2017) Composite quantile regression neural network with applications. Expert Syst Appl 76:129\u2013139","journal-title":"Expert Syst Appl"},{"issue":"2","key":"2671_CR36","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1080\/10618600.2017.1385469","volume":"27","author":"X Yang","year":"2018","unstructured":"Yang X, Narisetty NN, He X (2018) A new approach to censored quantile regression estimation. J Comput Graph Stat 27(2):417\u2013425","journal-title":"J Comput Graph Stat"},{"key":"2671_CR37","unstructured":"Yin X, Fallah-Shorshani M, McConnell R, Fruin S, Chiang Y-Y, Franklin M (2023) Quantile extreme gradient boosting for uncertainty quantification. arXiv preprint arXiv:2304.11732"},{"issue":"3","key":"2671_CR38","doi-asserted-by":"publisher","first-page":"191","DOI":"10.1007\/s13042-011-0031-2","volume":"2","author":"S Zheng","year":"2011","unstructured":"Zheng S (2011) Gradient descent algorithms for quantile regression with smooth approximation. Int J Mach Learn Cybern 2(3):191\u2013207","journal-title":"Int J Mach Learn Cybern"},{"issue":"3","key":"2671_CR39","doi-asserted-by":"publisher","first-page":"1108","DOI":"10.1214\/07-AOS507","volume":"36","author":"H Zou","year":"2008","unstructured":"Zou H, Yuan M (2008) Composite quantile regression and the oracle model selection theory. Ann Stat 36(3):1108\u20131126","journal-title":"Ann Stat"}],"container-title":["International Journal of Machine Learning and Cybernetics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13042-025-02671-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s13042-025-02671-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13042-025-02671-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T16:57:26Z","timestamp":1760547446000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s13042-025-02671-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,20]]},"references-count":39,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2025,10]]}},"alternative-id":["2671"],"URL":"https:\/\/doi.org\/10.1007\/s13042-025-02671-4","relation":{},"ISSN":["1868-8071","1868-808X"],"issn-type":[{"value":"1868-8071","type":"print"},{"value":"1868-808X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,20]]},"assertion":[{"value":"12 April 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 April 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 May 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no Conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}