{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T15:48:15Z","timestamp":1777391295408,"version":"3.51.4"},"reference-count":40,"publisher":"SAGE Publications","issue":"1-2","license":[{"start":{"date-parts":[[2018,6,21]],"date-time":"2018-06-21T00:00:00Z","timestamp":1529539200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Algorithmic Finance"],"published-print":{"date-parts":[[2018,6,21]]},"abstract":"<jats:p>Recent advances in machine learning, artificial intelligence, and the availability of billions of high frequency data signals have made model selection a challenging and pressing need. However, most of the model selection methods available in modern finance are subject to backtest overfitting. This is the probability that one will select a financial strategy that outperforms during backtest, but underperforms in practice. We evaluate the performance of the novel model confidence set (MCS) introduced in Hansen et al. (2011a) in a simple machine learning trading strategy problem. We find that MCS is not robust to multiple testing and that it requires a very high signal-to-noise ratio to be utilizable. More generally, we raise awareness on the limitations of model selection in finance.<\/jats:p>","DOI":"10.3233\/af-180231","type":"journal-article","created":{"date-parts":[[2018,6,22]],"date-time":"2018-06-22T11:12:35Z","timestamp":1529665955000},"page":"53-61","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":5,"title":["How hard is it to pick the right model? MCS and backtest overfitting"],"prefix":"10.1177","volume":"7","author":[{"given":"Diego","family":"Aparicio","sequence":"first","affiliation":[{"name":"Department of Economics, Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marcos","family":"L\u00f3pez de Prado","sequence":"additional","affiliation":[{"name":"True Positive Technologies, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2018,6,21]]},"reference":[{"key":"ref002","author":"Aparicio D.","year":"2016","journal-title":"MIT Working Paper"},{"key":"ref003","author":"Aparicio D.","year":"2017","journal-title":"MIT Working Paper"},{"key":"ref004","unstructured":"AtheyS., 2017. The impact of machine learning on economics, in Economics of Artificial Intelligence, University of Chicago Press."},{"issue":"5","key":"ref005","volume":"61","author":"Bailey D.H.","year":"2014","journal-title":"Notices of the AMS"},{"issue":"4","key":"ref006","first-page":"39","volume":"20","author":"Bailey D.H.","year":"2017","journal-title":"Journal of Computa\u00actional Finance"},{"key":"ref007","doi-asserted-by":"publisher","DOI":"10.3905\/jpm.2014.40.5.094"},{"key":"ref008","first-page":"283","volume-title":"in Proceedings of the 23rd international conference on World wide web","author":"Bakshy E.","year":"2014"},{"key":"ref009","doi-asserted-by":"publisher","DOI":"10.1111\/j.1540-6261.2009.01527.x"},{"key":"ref010","doi-asserted-by":"crossref","unstructured":"BastaniH., BayatiM., 2015. Online decision-making with high-dimensional covariates.","DOI":"10.2139\/ssrn.2661896"},{"key":"ref011","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1995.tb02031.x"},{"key":"ref012","first-page":"1165","author":"Benjamini Y.","year":"2001","journal-title":"Annals of statistics"},{"key":"ref013","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-444-62731-5.00020-8"},{"key":"ref014","first-page":"391","author":"Corradi V.","year":"2011","journal-title":"The Oxford Handbook of Economic Forecasting"},{"key":"ref015","unstructured":"ElliottG., TimmermannA., 2016. Handbook of economic fore\u00accasting, Elsevier."},{"key":"ref016","doi-asserted-by":"crossref","unstructured":"FaustJ., WrightJ.H. 2013. Forecasting inflation, Handbook of economic forecasting 2(Part A), 3\u201356.","DOI":"10.1016\/B978-0-444-53683-9.00001-3"},{"key":"ref017","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-financial-110311-101741"},{"key":"ref018","doi-asserted-by":"publisher","DOI":"10.3982\/ECTA5771"},{"issue":"2","key":"ref019","volume":"79","author":"Hansen P.R.","year":"2011","journal-title":"Econometrica Supplemental Material"},{"key":"ref020","unstructured":"HansenP.R., LundeA., NasonJ.M., 2014. Corrigendum to the model confidence set."},{"key":"ref021","doi-asserted-by":"publisher","DOI":"10.3905\/jpm.2014.40.5.108"},{"key":"ref022","doi-asserted-by":"publisher","DOI":"10.3905\/jpm.2015.42.1.013"},{"key":"ref023","doi-asserted-by":"publisher","DOI":"10.1093\/rfs\/hhv059"},{"key":"ref024","first-page":"65","author":"Holm S.","year":"1979","journal-title":"Scandinavian journal of statistics"},{"key":"ref025","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.0020124"},{"key":"ref026","doi-asserted-by":"publisher","DOI":"10.1177\/0956797611430953"},{"key":"ref027","first-page":"959","volume-title":"In Proceedings of the 13th ACM SIGKDD inter\u00acnational conference on Knowledge discovery and data mining","author":"Kohavi R.","year":"2007"},{"key":"ref001","author":"Lada A.","year":"2018","journal-title":"MIT Working Paper"},{"key":"ref028","doi-asserted-by":"publisher","DOI":"10.1126\/science.1167742"},{"key":"ref029","doi-asserted-by":"publisher","DOI":"10.1017\/S0266466605050036"},{"key":"ref030","doi-asserted-by":"publisher","DOI":"10.2469\/faj.v58.n4.2453"},{"issue":"2","key":"ref031","first-page":"21","volume":"42","author":"Lo A.W.","year":"2016","journal-title":"The Journal of Portfolio Man\u00acagement"},{"key":"ref032","doi-asserted-by":"publisher","DOI":"10.1016\/0304-405X(76)90022-2"},{"key":"ref033","doi-asserted-by":"publisher","DOI":"10.1017\/S0266466608080171"},{"key":"ref034","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.economics.102308.124342"},{"key":"ref035","doi-asserted-by":"publisher","DOI":"10.1111\/j.1468-0262.2005.00615.x"},{"key":"ref036","doi-asserted-by":"publisher","DOI":"10.1037\/a0033242"},{"key":"ref037","doi-asserted-by":"publisher","DOI":"10.1111\/1467-9868.00346"},{"key":"ref038","doi-asserted-by":"publisher","DOI":"10.1093\/rfs\/hhq105"},{"key":"ref039","doi-asserted-by":"publisher","DOI":"10.1257\/jep.28.2.3"},{"key":"ref040","doi-asserted-by":"publisher","DOI":"10.1111\/1468-0262.00152"}],"container-title":["Algorithmic Finance"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/AF-180231","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.3233\/AF-180231","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/AF-180231","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T09:52:56Z","timestamp":1777369976000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.3233\/AF-180231"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,21]]},"references-count":40,"journal-issue":{"issue":"1-2","published-print":{"date-parts":[[2018,6,21]]}},"alternative-id":["10.3233\/AF-180231"],"URL":"https:\/\/doi.org\/10.3233\/af-180231","relation":{},"ISSN":["2158-5571","2157-6203"],"issn-type":[{"value":"2158-5571","type":"print"},{"value":"2157-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,6,21]]}}}