{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T17:48:58Z","timestamp":1791395338564,"version":"4.3.3"},"reference-count":73,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T00:00:00Z","timestamp":1705449600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["J of Applied Econometrics"],"published-print":{"date-parts":[[2024,4]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>We study approaches for adjusting machine learning methods when the training sample differs from the prediction sample on unobserved dimensions. The machine learning literature predominately assumes selection only on observed dimensions. Common approaches are to weight or include variables that influence selection as solutions to selection on observables. Simulation results show that selection on unobservables increases mean squared prediction error using popular machine\u2010learning algorithms. Common machine learning practices such as weighting or including variables that influence selection into the training or prediction sample often worsen sample selection bias. We propose two control function approaches that remove the effects of selection bias before training and find that they reduce mean\u2010squared prediction error in simulations. We apply these approaches to predicting the vote share of the incumbent in gubernatorial elections using previously observed re\u2010election bids. We find that ignoring selection on unobservables leads to substantially higher predicted vote shares for the incumbent than when the control function approach is used.<\/jats:p>","DOI":"10.1002\/jae.3029","type":"journal-article","created":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T08:49:11Z","timestamp":1705481351000},"page":"383-400","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Addressing sample selection bias for machine learning methods"],"prefix":"10.1002","volume":"39","author":[{"given":"Dylan","family":"Brewer","sequence":"first","affiliation":[{"name":"School of Economics Georgia Institute of Technology  Atlanta Georgia USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alyssa","family":"Carlson","sequence":"additional","affiliation":[{"name":"Department of Economics University of Missouri  Columbia Missouri USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2024,1,17]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/0304-4076(93)90111-H"},{"key":"e_1_2_8_3_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0022381610000940"},{"key":"e_1_2_8_4_1","doi-asserted-by":"publisher","DOI":"10.1086\/717933"},{"key":"e_1_2_8_5_1","doi-asserted-by":"publisher","DOI":"10.2307\/1912776"},{"key":"e_1_2_8_6_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0022381608081024"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.aal4321"},{"key":"e_1_2_8_8_1","first-page":"507","volume-title":"The economics of artificial intelligence: An agenda","author":"Athey S.","year":"2018"},{"key":"e_1_2_8_9_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.p20151021"},{"key":"e_1_2_8_10_1","unstructured":"Ballotpedia. (1969).Encyclopedia of American politics. Retrieved June 2020 fromhttps:\/\/ballotpedia.org\/Ballotpedia:About"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.3982\/ECTA12723"},{"key":"e_1_2_8_12_1","doi-asserted-by":"publisher","DOI":"10.1093\/restud\/rdt044"},{"key":"e_1_2_8_13_1","doi-asserted-by":"publisher","DOI":"10.2307\/2946699"},{"key":"e_1_2_8_14_1","doi-asserted-by":"crossref","unstructured":"Bhattacharya J. &Vogt W. B.(2007).Do Instrumental Variables Belong in Propensity Scores?(0343): National Bureau of Economic Research Inc.https:\/\/www.nber.org\/papers\/t0343","DOI":"10.3386\/t0343"},{"key":"e_1_2_8_15_1","unstructured":"Bia M. Huber M. &Laffrs L.(2020).Double machine learning for sample selection models: arXiv.org\u00a0https:\/\/arxiv.org\/abs\/2012.00745"},{"key":"e_1_2_8_16_1","doi-asserted-by":"crossref","unstructured":"Bickel S. Br\u00fcckner M. &Scheffer T.(2007).Discriminative learning for differing training and test distributions. InProceedings of the 24th International Conference on Machine Learning pp.81\u201388.","DOI":"10.1145\/1273496.1273507"},{"key":"e_1_2_8_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/0304-4076(86)90038-2"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-937X.00236"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1017\/S026646661200028X"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1983.10478017"},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.1177\/1091142109344586"},{"key":"e_1_2_8_22_1","unstructured":"Fan W. Davidson I. Zadrozny B. &Yu P. S.(2005).An improved categorization of classifier's sensitivity on sample selection bias. InFifth ieee international conference on data mining IEEE."},{"key":"e_1_2_8_23_1","unstructured":"Farrell M. H. Liang T. &Misra S.(2021).Deep learning for individual heterogeneity: An automatic inference framework: arXiv.orghttps:\/\/arxiv.org\/abs\/2010.14694v2"},{"key":"e_1_2_8_24_1","unstructured":"Feng Y.et al. (2021).Causal inference in possibly nonlinear factor models: arXiv.org\u00a0https:\/\/arxiv.org\/abs\/2008.13651v3"},{"key":"e_1_2_8_25_1","volume-title":"The elements of statistical learning: data mining, inference, and prediction","author":"Friedman J.","year":"2009"},{"key":"e_1_2_8_26_1","unstructured":"Ghalebikesabi S. Cornish R. Holmes C. &Kelly L.(2021).Deep generative missingness pattern\u2010set mixture models. InThe 24th International Conference on Artificial Intelligence and Statistics pp.3727\u20133735."},{"key":"e_1_2_8_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-398750-1.50009-7"},{"key":"e_1_2_8_28_1","unstructured":"Gong Y. Hajimirsadeghi H. He J. Durand T. &Mori G.(2021).Variational selective autoencoder: Learning from partially\u2010observed heterogeneous data. InThe 24th International Conference on Artificial Intelligence and Statistics pp.2377\u20132385."},{"key":"e_1_2_8_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.red.2007.04.005"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1017\/psrm.2014.43"},{"key":"e_1_2_8_31_1","doi-asserted-by":"publisher","DOI":"10.2307\/1912352"},{"key":"e_1_2_8_32_1","doi-asserted-by":"publisher","DOI":"10.1111\/1468-0262.00442"},{"key":"e_1_2_8_33_1","doi-asserted-by":"crossref","unstructured":"Huang J. Gretton A. Borgwardt K. Sch\u00f6lkopf B. &Smola A.(2006).Correcting sample selection bias by unlabeled data. Advances in neural information processing systems 19.","DOI":"10.7551\/mitpress\/7503.003.0080"},{"key":"e_1_2_8_34_1","unstructured":"H\u00fcnermund P. Louw B. &Caspi I.(2021).Double machine learning and bad controls\u2014A cautionary tale: arXiv.org\u00a0https:\/\/arxiv.org\/abs\/2108.11294"},{"key":"e_1_2_8_35_1","unstructured":"Ipsen N. B. Mattei P.\u2010A. &Frellsen J.(2021).not\u2010MIWAE: Deep generative modelling with missing not at random data. InThe 2021 International Conference on Learning Representations."},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.aal2887"},{"key":"e_1_2_8_37_1","unstructured":"Klarner C.(2013).Governors dataset. Retrieved May 2020 fromhttps:\/\/doi.org\/10.7910\/DVN\/PQ0Y1N"},{"key":"e_1_2_8_38_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.p20151023"},{"key":"e_1_2_8_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/0304-4076(94)01720-4"},{"key":"e_1_2_8_40_1","doi-asserted-by":"publisher","DOI":"10.2307\/440290"},{"key":"e_1_2_8_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jeconom.2006.11.004"},{"key":"e_1_2_8_42_1","doi-asserted-by":"publisher","DOI":"10.1111\/ajps.12287"},{"key":"e_1_2_8_43_1","unstructured":"Ma C. &Zhang C.(2021).Identifiable generative models for missing not at random data imputation. InAdvances in neural information processing systems 34 pp.27645\u201327658."},{"key":"e_1_2_8_44_1","unstructured":"McDowell C.(1948).Gubernatorial elections Center on the American Governor Eagleton Institute of Politics Rutgers University. Retrieved June 2020 fromhttp:\/\/governors.rutgers.edu\/testing\/wp\u2010content\/uploads\/2014\/09\/Incumb_Chart_Word_2013.pdf"},{"key":"e_1_2_8_45_1","doi-asserted-by":"publisher","DOI":"10.1257\/jep.31.2.87"},{"key":"e_1_2_8_46_1","unstructured":"National Governors Association(1969).Former governors. Retrieved June 2020 fromhttps:\/\/www.nga.org\/former\u2010governors\/"},{"key":"e_1_2_8_47_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1368-423X.2008.00263.x"},{"key":"e_1_2_8_48_1","unstructured":"CQ Press. (1967).Voting and elections collection. Governor election returns county detail by year. Retrieved May 2020 fromhttp:\/\/library.cqpress.com\/elections\/download\u2010data.php"},{"key":"e_1_2_8_49_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-6419.00104"},{"key":"e_1_2_8_50_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1995.10476493"},{"key":"e_1_2_8_51_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1987.10478441"},{"key":"e_1_2_8_52_1","doi-asserted-by":"publisher","DOI":"10.1162\/003465302317411523"},{"key":"e_1_2_8_53_1","unstructured":"Schnabel T. Swaminathan A. Singh A. Chandak N. &Joachims T.(2016).Recommendations as treatments: Debiasing learning and evaluation. InThe 33rd International Conference on Machine Learning pp.1670\u20131679."},{"key":"e_1_2_8_54_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0378-3758(00)00115-4"},{"key":"e_1_2_8_55_1","doi-asserted-by":"crossref","unstructured":"Steck H.(2010).Training and testing of recommender systems on data missing not at random. InThe 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining pp.713\u2013722.","DOI":"10.1145\/1835804.1835895"},{"key":"e_1_2_8_56_1","doi-asserted-by":"publisher","DOI":"10.4135\/9781473957978.n25"},{"key":"e_1_2_8_57_1","unstructured":"Sugiyama M. Nakajima S. Kashima H. Buenau P. V. &Kawanabe M.(2007).Direct importance estimation with model selection and its application to covariate shift adaptation. Advances in neural information processing systems 20."},{"key":"e_1_2_8_58_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10463-008-0197-x"},{"key":"e_1_2_8_59_1","unstructured":"The Council of State Governments(1960).The book of the states. Retrieved June 2020 fromhttp:\/\/knowledgecenter.csg.org\/kc\/category\/content\u2010type\/content\u2010type\/book\u2010states"},{"key":"e_1_2_8_60_1","unstructured":"US BEA(1950).CPI for all urban consumers (CPI\u2010U). Retrieved July 2020 fromhttps:\/\/data.bls.gov\/timeseries\/CUUR0000SA0"},{"key":"e_1_2_8_61_1","unstructured":"US BEA(1969).Economic profile by county (CAINC30). Regional Economic Accounts. Retrieved March 2019 fromhttps:\/\/apps.bea.gov\/regional\/downloadzip.cfm"},{"key":"e_1_2_8_62_1","doi-asserted-by":"publisher","DOI":"10.1198\/073500102288618739"},{"key":"e_1_2_8_63_1","doi-asserted-by":"publisher","DOI":"10.1257\/jep.28.2.3"},{"key":"e_1_2_8_64_1","doi-asserted-by":"publisher","DOI":"10.2307\/146317"},{"key":"e_1_2_8_65_1","unstructured":"Wang X. Zhang R. Sun Y. &Qi J.(2019).Doubly robust joint learning for recommendation on data missing not at random. InThe 36th International Conference on Machine Learning pp.6638\u20136647."},{"key":"e_1_2_8_66_1","doi-asserted-by":"publisher","DOI":"10.1002\/smj.2995"},{"key":"e_1_2_8_67_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10258-002-0008-x"},{"key":"e_1_2_8_68_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jeconom.2007.02.002"},{"key":"e_1_2_8_69_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.rie.2016.01.001"},{"key":"e_1_2_8_70_1","doi-asserted-by":"crossref","unstructured":"Zadrozny B.(2004).Learning and evaluating classifiers under sample selection bias. InThe 21st International Conference on Machine Learning.","DOI":"10.1145\/1015330.1015425"},{"key":"e_1_2_8_71_1","doi-asserted-by":"crossref","unstructured":"Zadrozny B. &Elkan C.(2001).Learning and making decisions when costs and probabilities are both unknown. InThe Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining pp.204\u2013213.","DOI":"10.1145\/502512.502540"},{"key":"e_1_2_8_72_1","unstructured":"Zadrozny B. Langford J. &Abe N.(2003).Cost\u2010sensitive learning by cost\u2010proportionate example weighting. InThird IEEE International Conference on Data Mining pp.435\u2013442."},{"key":"e_1_2_8_73_1","doi-asserted-by":"crossref","unstructured":"Zhang W. Bao W. Liu X.\u2010Y. Yang K. Lin Q. Wen H. &Ramezani R.(2020).Large\u2010scale causal approaches to debiasing post\u2010click conversion rate estimation with multi\u2010task learning. InThe web conference 2020 pp.2775\u20132781.","DOI":"10.1145\/3366423.3380037"},{"key":"e_1_2_8_74_1","doi-asserted-by":"publisher","DOI":"10.1214\/16-AOS1528"}],"container-title":["Journal of Applied Econometrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/jae.3029","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,16]],"date-time":"2024-09-16T02:59:03Z","timestamp":1726455543000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/jae.3029"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,17]]},"references-count":73,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,4]]}},"alternative-id":["10.1002\/jae.3029"],"URL":"https:\/\/doi.org\/10.1002\/jae.3029","archive":["Portico"],"relation":{},"ISSN":["0883-7252","1099-1255"],"issn-type":[{"value":"0883-7252","type":"print"},{"value":"1099-1255","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,17]]},"assertion":[{"value":"2021-12-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-07-06","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}