{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T10:40:46Z","timestamp":1761129646824,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2019,7,17]],"date-time":"2019-07-17T00:00:00Z","timestamp":1563321600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Point estimation of class prevalences in the presence of dataset shift has been a popular research topic for more than two decades. Less attention has been paid to the construction of confidence and prediction intervals for estimates of class prevalences. One little considered question is whether or not it is necessary for practical purposes to distinguish confidence and prediction intervals. Another question so far not yet conclusively answered is whether or not the discriminatory power of the classifier or score at the basis of an estimation method matters for the accuracy of the estimates of the class prevalences. This paper presents a simulation study aimed at shedding some light on these and other related questions.<\/jats:p>","DOI":"10.3390\/make1030047","type":"journal-article","created":{"date-parts":[[2019,7,17]],"date-time":"2019-07-17T11:25:12Z","timestamp":1563362712000},"page":"805-831","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Confidence Intervals for Class Prevalences under Prior Probability Shift"],"prefix":"10.3390","volume":"1","author":[{"given":"Dirk","family":"Tasche","sequence":"first","affiliation":[{"name":"Independent Researcher, 8032 Z\u00fcrich, Switzerland"}]}],"member":"1968","published-online":{"date-parts":[[2019,7,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1016\/j.patcog.2012.07.022","article-title":"On the study of nearest neighbor algorithms for prevalence estimation in binary problems","volume":"46","author":"Barranquero","year":"2013","journal-title":"Pattern Recognit."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1007\/s10618-008-0097-y","article-title":"Quantifying counts and costs via classification","volume":"17","author":"Forman","year":"2008","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.ins.2012.05.028","article-title":"Class distribution estimation based on the Hellinger distance","volume":"218","author":"Alegre","year":"2013","journal-title":"Inf. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1007\/s10994-016-5604-6","article-title":"Class-prior estimation for learning from positive and unlabeled data","volume":"106","author":"Niu","year":"2017","journal-title":"Mach. Learn."},{"key":"ref_5","first-page":"74:1","article-title":"A Review on Quantification Learning","volume":"50","author":"Chawla","year":"2017","journal-title":"ACM Comput. Surv."},{"key":"ref_6","unstructured":"Riquelme, J., Troncoso, A., and Garc\u00eda, S. (2018). An\u00e1lisis de algoritmos de cuantificac\u00edon basados en ajuste de distribuciones. IX Simposio de Teor\u00eda y Aplicaciones de la Miner\u00eda de Datos (TAMIDA 2018), Asociaci\u00f3n Espa\u00f1ola para la Inteligencia Artificial."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Keith, K., and O\u2019Connor, B. (November, January 31). Uncertainty-aware generative models for inferring document class prevalence. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.","DOI":"10.18653\/v1\/D18-1487"},{"key":"ref_8","unstructured":"Maletzke, A., Dos Reis, D., Cherman, E., and Batista, G. (February, January 27). DyS: A Framework for Mixture Models in Quantification. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19), Honolulu, HI, USA."},{"key":"ref_9","first-page":"1","article-title":"Quantification Under Prior Probability Shift: The Ratio Estimator and its Extensions","volume":"20","author":"Vaz","year":"2019","journal-title":"J. Mach. Learn. Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1111\/j.1540-5907.2009.00428.x","article-title":"A Method of Automated Nonparametric Content Analysis for Social Science","volume":"54","author":"Hopkins","year":"2010","journal-title":"Am. J. Polit. Sci."},{"key":"ref_11","unstructured":"Daughton, A., and Paul, M. (February, January 27). Constructing Accurate Confidence Intervals when Aggregating Social Media Data for Public Health Monitoring. Proceedings of the AAAI International Workshop on Health Intelligence (W3PHIAI), Honolulu, HI, USA."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Meeker, W., Hahn, G., and Escobar, L. (2017). Statistical Intervals: A Guide for Practitioners and Researchers, John Wiley & Sons. [2nd ed.].","DOI":"10.1002\/9781118594841"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1016\/j.patcog.2014.07.032","article-title":"Quantification-oriented learning based on reliable classifiers","volume":"48","author":"Barranquero","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_14","unstructured":"Zhang, K., Sch\u00f6lkopf, B., Muandet, K., and Wang, Z. (2013, January 16\u201321). Domain Adaptation Under Target and Conditional Shift. Proceedings of the 30th International Conference on International Conference on Machine Learning, JMLR.org, ICML\u201913, Atlanta, GA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1016\/j.csda.2012.07.007","article-title":"Drift mining in data: A framework for addressing drift in classification","volume":"57","author":"Hofer","year":"2013","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_16","first-page":"3122","article-title":"Detecting and Correcting for Label Shift with Black Box Predictors","volume":"Volume 80","author":"Dy","year":"2018","journal-title":"Proceedings of the 35th International Conference on Machine Learning"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1016\/j.patcog.2011.06.019","article-title":"A unifying view on dataset shift in classification","volume":"45","author":"Raeder","year":"2012","journal-title":"Pattern Recognit."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1162\/089976602753284446","article-title":"Adjusting the Outputs of a Classifier to New a Priori Probabilities: A Simple Procedure","volume":"14","author":"Saerens","year":"2001","journal-title":"Neural Comput."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Bella, A., Ferri, C., Hernandez-Orallo, J., and Ram\u00edrez-Quintana, M. (2010, January 3\u201317). Quantification via probability estimators. Proceedings of the 2010 IEEE International Conference on Data Mining(ICDM), Sydney, NSW, Australia.","DOI":"10.1109\/ICDM.2010.75"},{"key":"ref_20","unstructured":"R Core Team (2014). R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1587\/transinf.2015EDP7212","article-title":"Computationally Efficient Class-Prior Estimation under Class Balance Change Using Energy Distance","volume":"99","author":"Kawakubo","year":"2016","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Cramer, J. (2003). Logit Models From Economics and Other Fields, Cambridge University Press.","DOI":"10.1017\/CBO9780511615412"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Davison, A., and Hinkley, D. (1997). Bootstrap Methods and their Application, Cambridge University Press.","DOI":"10.1017\/CBO9780511802843"},{"key":"ref_24","first-page":"3338","article-title":"Fisher Consistency for Prior Probability Shift","volume":"18","author":"Tasche","year":"2017","journal-title":"J. Mach. Learn. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1093\/oxfordjournals.aje.a120610","article-title":"Comparison of a screening test and a reference test in epidemiologic studies. II. A probabilistic model for the comparison of diagnostic tests","volume":"83","author":"Gart","year":"1966","journal-title":"Am. J. Epidemiol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1127","DOI":"10.1080\/03610927608827429","article-title":"The numerical evaluation of the maximum-likelihood estimate of mixture proportions","volume":"5","author":"Peters","year":"1976","journal-title":"Commun. Stat. Theory Methods"},{"key":"ref_27","unstructured":"Casella, G., and Berger, R. (2002). Statistical Inference, Thomson Learning. [2nd ed.]."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5052","DOI":"10.1214\/17-EJS1338SI","article-title":"Beyond sigmoids: How to obtain well-calibrated probabilities from binary classifiers with beta calibration","volume":"11","author":"Kull","year":"2017","journal-title":"Electron. J. Stat."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/j.ejor.2014.11.022","article-title":"Adapting a classification rule to local and global shift when only unlabelled data are available","volume":"243","author":"Hofer","year":"2015","journal-title":"Eur. J. Oper. Res."},{"key":"ref_30","unstructured":"Fr\u00fchwirth-Schnatter, S. (2006). Finite Mixture and Markov Switching Models: Modeling and Applications to Random Processes, Springer."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.neunet.2013.11.010","article-title":"Semi-supervised learning of class balance under class-prior change by distribution matching","volume":"50","author":"Sugiyama","year":"2014","journal-title":"Neural Netw."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1137\/1026034","article-title":"Mixture densities, maximum likelihood and the EM algorithm","volume":"26","author":"Redner","year":"1984","journal-title":"SIAM Rev."},{"key":"ref_33","unstructured":"Tasche, D. (2013). The Law of Total Odds. arXiv."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/1\/3\/47\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:06:42Z","timestamp":1760188002000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/1\/3\/47"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,17]]},"references-count":33,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2019,9]]}},"alternative-id":["make1030047"],"URL":"https:\/\/doi.org\/10.3390\/make1030047","relation":{},"ISSN":["2504-4990"],"issn-type":[{"type":"electronic","value":"2504-4990"}],"subject":[],"published":{"date-parts":[[2019,7,17]]}}}