{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,27]],"date-time":"2026-07-27T13:25:41Z","timestamp":1785158741877,"version":"3.55.0"},"reference-count":63,"publisher":"Springer Science and Business Media LLC","issue":"24","license":[{"start":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T00:00:00Z","timestamp":1683849600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T00:00:00Z","timestamp":1683849600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003170","name":"Stiftelsen f\u00f6r Kunskaps- och Kompetensutveckling","doi-asserted-by":"publisher","award":["20200067"],"award-info":[{"award-number":["20200067"]}],"id":[{"id":"10.13039\/501100003170","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100016777","name":"Karlstad University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100016777","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Most machine learning methods assume that the input data distribution is the same in the training and testing phases. However, in practice, this stationarity is usually not met and the distribution of inputs differs, leading to unexpected performance of the learned model in deployment. The issue in which the training and test data inputs follow different probability distributions while the input\u2013output relationship remains unchanged is referred to as covariate shift. In this paper, the performance of conventional machine learning models was experimentally evaluated in the presence of covariate shift. Furthermore, a region-based evaluation was performed by decomposing the domain of probability density function of the input data to assess the classifier\u2019s performance per domain region. Distributional changes were simulated in a two-dimensional classification problem. Subsequently, a higher four-dimensional experiments were conducted. Based on the experimental analysis, the Random Forests algorithm is the most robust classifier in the two-dimensional case, showing the lowest degradation rate for accuracy and F1-score metrics, with a range between 0.1% and 2.08%. Moreover, the results reveal that in higher-dimensional experiments, the performance of the models is predominantly influenced by the complexity of the classification function, leading to degradation rates exceeding 25% in most cases. It is also concluded that the models exhibit high bias toward the region with high density in the input space domain of the training samples.<\/jats:p>","DOI":"10.1007\/s00521-023-08622-w","type":"journal-article","created":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T20:02:11Z","timestamp":1683921731000},"page":"17555-17577","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["A domain-region based evaluation of ML performance robustness to covariate shift"],"prefix":"10.1007","volume":"35","author":[{"given":"Firas","family":"Bayram","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9051-7609","authenticated-orcid":false,"given":"Bestoun S.","family":"Ahmed","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,5,12]]},"reference":[{"key":"8622_CR1","doi-asserted-by":"crossref","unstructured":"L\u2019heureux A, Grolinger K, Elyamany HF, Capretz MA (2017) Machine learning with big data, challenges and approaches. IEEE Access 5:7776\u20137797","DOI":"10.1109\/ACCESS.2017.2696365"},{"key":"8622_CR2","doi-asserted-by":"crossref","unstructured":"Witten IH, Frank E, Hall MA, Pal CJ (2017) Probabilistic methods. In: Witten IH, Frank E, Hall MA, Pal CJ (eds.) Data mining (Fourth Edition), Fourth edition edn., pp 335\u2013416. Morgan Kaufmann, San Francisco, CA, USA","DOI":"10.1016\/B978-0-12-804291-5.00009-X"},{"key":"8622_CR3","unstructured":"Brownlee J (2019) Probability for machine learning: discover how to harness uncertainty with Python"},{"key":"8622_CR4","doi-asserted-by":"crossref","unstructured":"Gama Ja, \u017dliobaitundefined I, Bifet A, Pechenizkiy M, Bouchachia A (2014) A survey on concept drift adaptation. ACM Comput Surv 46(4)","DOI":"10.1145\/2523813"},{"issue":"12","key":"8622_CR5","first-page":"2346","volume":"31","author":"J Lu","year":"2019","unstructured":"Lu J, Liu A, Dong F, Gu F, Gama J, Zhang G (2019) Learning under concept drift: a review. IEEE Trans Knowl Data Eng 31(12):2346\u20132363","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"1","key":"8622_CR6","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1016\/j.patcog.2011.06.019","volume":"45","author":"JG Moreno-Torres","year":"2012","unstructured":"Moreno-Torres JG, Raeder T, Alaiz-Rodr\u00edguez R, Chawla NV, Herrera F (2012) A unifying view on dataset shift in classification. Pattern Recogn 45(1):521\u2013530","journal-title":"Pattern Recogn"},{"issue":"3","key":"8622_CR7","doi-asserted-by":"publisher","first-page":"659","DOI":"10.1016\/j.patcog.2014.07.028","volume":"48","author":"H Raza","year":"2015","unstructured":"Raza H, Prasad G, Li Y (2015) EWMA model based shift-detection methods for detecting covariate shifts in non-stationary environments. Pattern Recogn 48(3):659\u2013669","journal-title":"Pattern Recogn"},{"key":"8622_CR8","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2022.108632","volume":"245","author":"F Bayram","year":"2022","unstructured":"Bayram F, Ahmed BS, Kassler A (2022) From concept drift to model degradation: an overview on performance-aware drift detectors. Knowl-Based Syst 245:108632","journal-title":"Knowl-Based Syst"},{"issue":"2","key":"8622_CR9","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1016\/S0378-3758(00)00115-4","volume":"90","author":"H Shimodaira","year":"2000","unstructured":"Shimodaira H (2000) Improving predictive inference under covariate shift by weighting the log-likelihood function. J Stat Plann Inf 90(2):227\u2013244","journal-title":"J Stat Plann Inf"},{"key":"8622_CR10","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/9780262017091.001.0001","volume-title":"Machine learning in non-stationary environments: introduction to covariate shift adaptation","author":"M Sugiyama","year":"2012","unstructured":"Sugiyama M, Kawanabe M (2012) Machine learning in non-stationary environments: introduction to covariate shift adaptation. The MIT Press, Cambridge"},{"key":"8622_CR11","series-title":"Computer Science Department","volume-title":"The problem of concept drift: definitions and related work","author":"A Tsymbal","year":"2004","unstructured":"Tsymbal A (2004) The problem of concept drift: definitions and related work. Computer Science Department. Trinity College, Dublin"},{"issue":"1","key":"8622_CR12","first-page":"3338","volume":"18","author":"D Tasche","year":"2017","unstructured":"Tasche D (2017) Fisher consistency for prior probability shift. J Mach Learn Res 18(1):3338\u20133369","journal-title":"J Mach Learn Res"},{"key":"8622_CR13","unstructured":"Li F, Lam H, Prusty S (2020) Robust importance weighting for covariate shift. In: International conference on artificial intelligence and statistics, pp 352\u2013362. PMLR"},{"issue":"2","key":"8622_CR14","first-page":"345","volume":"21","author":"A Subbaswamy","year":"2020","unstructured":"Subbaswamy A, Saria S (2020) From development to deployment: dataset shift, causality, and shift-stable models in health ai. Biostatistics 21(2):345\u2013352","journal-title":"Biostatistics"},{"key":"8622_CR15","first-page":"11539","volume":"33","author":"S Schneider","year":"2020","unstructured":"Schneider S, Rusak E, Eck L, Bringmann O, Brendel W, Bethge M (2020) Improving robustness against common corruptions by covariate shift adaptation. Adv Neural Inf Process Syst 33:11539\u201311551","journal-title":"Adv Neural Inf Process Syst"},{"key":"8622_CR16","unstructured":"Duchi JC, Hashimoto T, Namkoong, H (2019) Distributionally robust losses against mixture covariate shifts. Under Rev 2"},{"key":"8622_CR17","doi-asserted-by":"crossref","unstructured":"Fei G, Liu B (2015) Social media text classification under negative covariate shift. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 2347\u20132356","DOI":"10.18653\/v1\/D15-1282"},{"key":"8622_CR18","doi-asserted-by":"crossref","unstructured":"He H, Zha S, Wang H (2019) Unlearn dataset bias in natural language inference by fitting the residual. arXiv preprint arXiv:1908.10763","DOI":"10.18653\/v1\/D19-6115"},{"key":"8622_CR19","doi-asserted-by":"publisher","DOI":"10.1016\/j.csda.2021.107382","volume":"168","author":"PF Wiemann","year":"2022","unstructured":"Wiemann PF, Klein N, Kneib T (2022) Correcting for sample selection bias in Bayesian distributional regression models. Comput Stat Data Anal 168:107382","journal-title":"Comput Stat Data Anal"},{"key":"8622_CR20","first-page":"138","volume":"17","author":"Y Tsuboi","year":"2009","unstructured":"Tsuboi Y, Kashima H, Hido S, Bickel S, Sugiyama M (2009) Direct density ratio estimation for large-scale covariate shift adaptation. J Inf Process 17:138\u2013155","journal-title":"J Inf Process"},{"key":"8622_CR21","doi-asserted-by":"crossref","unstructured":"Sugiyama M, Krauledat M, M\u00fcller K-R (2007) Covariate shift adaptation by importance weighted cross validation. J Mach Learn Res 8(5)","DOI":"10.1007\/11861898_36"},{"key":"8622_CR22","unstructured":"Subbaswamy A, Adams R, Saria S (2021) Evaluating model robustness and stability to dataset shift. In: International conference on artificial intelligence and statistics, pp 2611\u20132619. PMLR"},{"issue":"13","key":"8622_CR23","doi-asserted-by":"publisher","first-page":"3643","DOI":"10.3390\/s20133643","volume":"20","author":"H Liu","year":"2020","unstructured":"Liu H, Wu Y, Cao Y, Lv W, Han H, Li Z, Chang J (2020) Well logging based lithology identification model establishment under data drift: a transfer learning method. Sensors 20(13):3643","journal-title":"Sensors"},{"key":"8622_CR24","doi-asserted-by":"crossref","unstructured":"Sakai T, Shimizu N (2019) Covariate shift adaptation on learning from positive and unlabeled data. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 4838\u20134845","DOI":"10.1609\/aaai.v33i01.33014838"},{"key":"8622_CR25","doi-asserted-by":"crossref","unstructured":"Quionero-Candela J, Sugiyama M, Schwaighofer A, Lawrence ND (2009) Dataset shift in machine learning. The MIT Press, Cambridge","DOI":"10.7551\/mitpress\/9780262170055.001.0001"},{"key":"8622_CR26","doi-asserted-by":"crossref","unstructured":"Huang J, Gretton A, Borgwardt K, Sch\u00f6lkopf B, Smola A (2006) Correcting sample selection bias by unlabeled data. Adv Neural Inf Process Syst 19","DOI":"10.7551\/mitpress\/7503.003.0080"},{"key":"8622_CR27","unstructured":"Sugiyama M, Nakajima S, Kashima H, Buenau P, Kawanabe M (2007) Direct importance estimation with model selection and its application to covariate shift adaptation. Adv Neural Inf Process Syst 20"},{"key":"8622_CR28","unstructured":"Kanamori T, Hido S, Sugiyama M (2009) A least-squares approach to direct importance estimation. J Mach Learn Res 10:1391\u20131445"},{"key":"8622_CR29","doi-asserted-by":"crossref","unstructured":"Sugiyama M, Suzuki T, Nakajima S, Kashima H, von B\u00fcnau P, Kawanabe M (2008) Direct importance estimation for covariate shift adaptation. Ann Inst Stat Math 60(4):699\u2013746","DOI":"10.1007\/s10463-008-0197-x"},{"key":"8622_CR30","doi-asserted-by":"publisher","first-page":"73804","DOI":"10.1109\/ACCESS.2019.2920486","volume":"7","author":"SV Chapaneri","year":"2019","unstructured":"Chapaneri SV, Jayaswal DJ (2019) Covariate shift adaptation for structured regression with Frank\u2013Wolfe algorithms. IEEE Access 7:73804\u201373818","journal-title":"IEEE Access"},{"key":"8622_CR31","doi-asserted-by":"crossref","unstructured":"Alaiz-Rodr\u00edguez R, Japkowicz N (2008) Assessing the impact of changing environments on classifier performance. In: Conference of the Canadian society for computational studies of intelligence, pp 13\u201324. Springer","DOI":"10.1007\/978-3-540-68825-9_2"},{"key":"8622_CR32","doi-asserted-by":"crossref","unstructured":"Abbasian H, Drummond C, Japkowicz N, Matwin S (2010) Robustness of classifiers to changing environments. In: Canadian conference on artificial intelligence, pp 232\u2013243. Springer","DOI":"10.1007\/978-3-642-13059-5_23"},{"key":"8622_CR33","unstructured":"Chen M, Goel K, Sohoni NS, Poms F, Fatahalian K, R\u00e9 C (2021) Mandoline: Model evaluation under distribution shift. In: International conference on machine learning, pp 1617\u20131629. PMLR"},{"key":"8622_CR34","unstructured":"Sagawa S, Koh PW, Hashimoto TB, Liang, P (2019) Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731"},{"key":"8622_CR35","unstructured":"Garg S, Balakrishnan S, Lipton ZC, Neyshabur B, Sedghi H (2022) Leveraging unlabeled data to predict out-of-distribution performance. In: ICLR"},{"key":"8622_CR36","doi-asserted-by":"crossref","unstructured":"Guillory D, Shankar V, Ebrahimi S, Darrell T, Schmidt L (2021) Predicting with confidence on unseen distributions. In: Proceedings of the IEEE\/CVF international conference on computer vision, pp 1134\u20131144","DOI":"10.1109\/ICCV48922.2021.00117"},{"key":"8622_CR37","doi-asserted-by":"crossref","unstructured":"Deng W, Zheng L (2021) Are labels always necessary for classifier accuracy evaluation? In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 15069\u201315078","DOI":"10.1109\/CVPR46437.2021.01482"},{"key":"8622_CR38","unstructured":"Recht B, Roelofs R, Schmidt L, Shankar V (2019) Do imagenet classifiers generalize to imagenet? In: International conference on machine learning, pp 5389\u20135400. PMLR"},{"key":"8622_CR39","first-page":"18583","volume":"33","author":"R Taori","year":"2020","unstructured":"Taori R, Dave A, Shankar V, Carlini N, Recht B, Schmidt L (2020) Measuring robustness to natural distribution shifts in image classification. Adv Neural Inf Process Syst 33:18583\u201318599","journal-title":"Adv Neural Inf Process Syst"},{"key":"8622_CR40","unstructured":"Miller JP, Taori R, Raghunathan A, Sagawa S, Koh PW, Shankar V, Liang P, Carmon Y, Schmidt L (2021) Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization. In: International conference on machine learning, pp 7721\u20137735. PMLR"},{"key":"8622_CR41","unstructured":"Yadav C, Bottou L (2019) Cold case: The lost mnist digits. Adv Neural Inf Process Syst 32"},{"key":"8622_CR42","unstructured":"Miller J, Krauth K, Recht B, Schmidt L (2020) The effect of natural distribution shift on question answering models. In: International conference on machine learning, pp 6905\u20136916. PMLR"},{"key":"8622_CR43","doi-asserted-by":"crossref","unstructured":"Rajpurkar P, Zhang J, Lopyrev K, Liang P (2016) Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250","DOI":"10.18653\/v1\/D16-1264"},{"issue":"5","key":"8622_CR44","first-page":"1","volume":"8","author":"M Sugiyama","year":"2007","unstructured":"Sugiyama M, Krauledat M, M\u00fcller K-R (2007) Covariate shift adaptation by importance weighted cross validation. J Mach Learn Res 8(5):1\u201321","journal-title":"J Mach Learn Res"},{"key":"8622_CR45","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1016\/j.neucom.2011.09.016","volume":"80","author":"H Hachiya","year":"2012","unstructured":"Hachiya H, Sugiyama M, Ueda N (2012) Importance-weighted least-squares probabilistic classifier for covariate shift adaptation with application to human activity recognition. Neurocomputing 80:93\u2013101","journal-title":"Neurocomputing"},{"key":"8622_CR46","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1016\/j.eswa.2018.03.021","volume":"104","author":"PR Almeida","year":"2018","unstructured":"Almeida PR, Oliveira LS, Britto AS Jr, Sabourin R (2018) Adapting dynamic classifier selection for concept drift. Expert Syst Appl 104:67\u201385","journal-title":"Expert Syst Appl"},{"key":"8622_CR47","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s12530-016-9168-2","volume":"9","author":"I Khamassi","year":"2018","unstructured":"Khamassi I, Mouchaweh MS, Hammami M, Gh\u00e9dira K (2018) Discussion and review on evolving data streams and concept drift adapting. Evol Syst 9:1\u201323","journal-title":"Evol Syst"},{"issue":"3","key":"8622_CR48","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1007\/BF00994018","volume":"20","author":"C Cortes","year":"1995","unstructured":"Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273\u2013297","journal-title":"Mach Learn"},{"key":"8622_CR49","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-84858-7","volume-title":"The elements of statistical learning: data mining, inference, and prediction","author":"T Hastie","year":"2009","unstructured":"Hastie T, Tibshirani R, Friedman JH, Friedman JH (2009) The elements of statistical learning: data mining, inference, and prediction, vol 2. Springer, New York"},{"issue":"1","key":"8622_CR50","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L (2001) Random forests. Mach Learn 45(1):5\u201332","journal-title":"Mach Learn"},{"key":"8622_CR51","volume-title":"Pattern recognition and machine learning","author":"CM Bishop","year":"2006","unstructured":"Bishop CM, Nasrabadi NM (2006) Pattern recognition and machine learning, vol 4. Springer, New York"},{"issue":"7","key":"8622_CR52","doi-asserted-by":"publisher","first-page":"621","DOI":"10.2165\/00002018-200730070-00010","volume":"30","author":"DJ Hand","year":"2007","unstructured":"Hand DJ (2007) Principles of data mining. Drug Saf 30(7):621\u2013622","journal-title":"Drug Saf"},{"key":"8622_CR53","unstructured":"Ovadia Y, Fertig E, Ren J, Nado Z, Sculley D, Nowozin S, Dillon J, Lakshminarayanan B, Snoek J (2019) Can you trust your model\u2019s uncertainty? Evaluating predictive uncertainty under dataset shift. Adv Neural Inf Process Syst 32"},{"key":"8622_CR54","unstructured":"Amodei D, Olah C, Steinhardt J, Christiano P, Schulman J, Man\u00e9 D (2016) Concrete problems in AI safety. arXiv preprint arXiv:1606.06565"},{"key":"8622_CR55","unstructured":"Lesch S, Kleinbauer T, Alexandersson J (2005) A new metric for the evaluation of dialog act classification. In: Proceedings of the 9th workshop on the semantics and pragmatics of dialogue (SEMDIAL: DIALOR), Nancy, France, pp 143\u20136. Citeseer"},{"key":"8622_CR56","unstructured":"Rabanser S, G\u00fcnnemann S, Lipton Z (2019) Failing loudly: an empirical study of methods for detecting dataset shift. Adv Neural Inf Process Syst 32"},{"key":"8622_CR57","doi-asserted-by":"crossref","unstructured":"Clark C, Yatskar M, Zettlemoyer L (2019) Don\u2019t take the easy way out: ensemble based methods for avoiding known dataset biases. arXiv preprint arXiv:1909.03683","DOI":"10.18653\/v1\/D19-1418"},{"key":"8622_CR58","unstructured":"Shafieezadeh\u00a0Abadeh S, Mohajerin\u00a0Esfahani PM, Kuhn D (2015) Distributionally robust logistic regression. Adv Neural Inf Process Syst 28"},{"issue":"5","key":"8622_CR59","doi-asserted-by":"publisher","first-page":"1315","DOI":"10.1016\/j.patcog.2012.10.020","volume":"46","author":"E Atashpaz-Gargari","year":"2013","unstructured":"Atashpaz-Gargari E, Sima C, Braga-Neto UM, Dougherty ER (2013) Relationship between the accuracy of classifier error estimation and complexity of decision boundary. Pattern Recogn 46(5):1315\u20131322","journal-title":"Pattern Recogn"},{"key":"8622_CR60","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1016\/j.tcs.2013.09.027","volume":"519","author":"C Cortes","year":"2014","unstructured":"Cortes C, Mohri M (2014) Domain adaptation and sample bias correction theory and algorithm for regression. Theoret Comput Sci 519:103\u2013126","journal-title":"Theoret Comput Sci"},{"issue":"3","key":"8622_CR61","doi-asserted-by":"publisher","first-page":"459","DOI":"10.1007\/s10844-020-00634-5","volume":"56","author":"S Micevska","year":"2021","unstructured":"Micevska S, Awad A, Sakr S (2021) SDDM: an interpretable statistical concept drift detection method for data streams. J Intell Inf Syst 56(3):459\u2013484","journal-title":"J Intell Inf Syst"},{"key":"8622_CR62","doi-asserted-by":"crossref","unstructured":"Lima M, A\u00a0Fagundes R.A.d (2021) A comparative study on concept drift detectors for regression. In: Brazilian conference on intelligent systems, pp 390\u2013405. Springer","DOI":"10.1007\/978-3-030-91702-9_26"},{"key":"8622_CR63","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1007\/s10994-019-05840-z","volume":"109","author":"A Cano","year":"2020","unstructured":"Cano A, Krawczyk B (2020) Kappa updated ensemble for drifting data stream mining. Mach Learn 109:175\u2013218","journal-title":"Mach Learn"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-023-08622-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-023-08622-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-023-08622-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T17:50:32Z","timestamp":1690393832000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-023-08622-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,12]]},"references-count":63,"journal-issue":{"issue":"24","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["8622"],"URL":"https:\/\/doi.org\/10.1007\/s00521-023-08622-w","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,12]]},"assertion":[{"value":"21 October 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 April 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 May 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflicts of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}