{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,13]],"date-time":"2026-08-13T05:18:52Z","timestamp":1786598332740,"version":"3.56.0"},"reference-count":23,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2012,12,1]],"date-time":"2012-12-01T00:00:00Z","timestamp":1354320000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100003977","name":"Israel Science Foundation","doi-asserted-by":"publisher","award":["1227\/09"],"award-info":[{"award-number":["1227\/09"]}],"id":[{"id":"10.13039\/501100003977","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2012,12]]},"abstract":"<jats:p>Deemed \u201cone of the top ten data mining mistakes\u201d, leakage is the introduction of information about the data mining target that should not be legitimately available to mine from. In addition to our own industry experience with real-life projects, controversies around several major public data mining competitions held recently such as the INFORMS 2010 Data Mining Challenge and the IJCNN 2011 Social Network Challenge are evidence that this issue is as relevant today as it has ever been. While acknowledging the importance and prevalence of leakage in both synthetic competitions and real-life data mining projects, existing literature has largely left this idea unexplored. What little has been said turns out not to be broad enough to cover more complex cases of leakage, such as those where the classical independently and identically distributed (i.i.d.) assumption is violated, that have been recently documented. In our new approach, these cases and others are explained by explicitly defining modeling goals and analyzing the broader framework of the data mining problem. The resulting definition enables us to derive general methodology for dealing with the issue. We show that it is possible to avoid leakage with a simple specific approach to data management followed by what we call a learn-predict separation, and present several ways of detecting leakage when the modeler has no control over how the data have been collected. We also offer an alternative point of view on leakage that is based on causal graph modeling concepts.<\/jats:p>","DOI":"10.1145\/2382577.2382579","type":"journal-article","created":{"date-parts":[[2013,1,2]],"date-time":"2013-01-02T13:23:15Z","timestamp":1357132995000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":549,"title":["Leakage in data mining"],"prefix":"10.1145","volume":"6","author":[{"given":"Shachar","family":"Kaufman","sequence":"first","affiliation":[{"name":"Tel Aviv University, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saharon","family":"Rosset","sequence":"additional","affiliation":[{"name":"Tel Aviv University, Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Claudia","family":"Perlich","sequence":"additional","affiliation":[{"name":"m6d"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ori","family":"Stitelman","sequence":"additional","affiliation":[{"name":"m6d"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2012,12,18]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the International Conference on Database Theory (ICDT'01)","author":"Buneman P."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.2307\/1913236"},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Hastie T. Tibshirani R. and Friedman J. H. 2009. The Elements of Statistical Learning: Data Mining Inference and Prediction. Springer.  Hastie T. Tibshirani R. and Friedman J. H. 2009. The Elements of Statistical Learning: Data Mining Inference and Prediction. Springer.","DOI":"10.1007\/978-0-387-84858-7"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/380995.381033"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:MACH.0000035473.11134.83"},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the 5th WEBKDD Workshop.","author":"Kohavi R."},{"key":"e_1_2_1_7_1","unstructured":"Laiy S. Xiang L. Diao R. Liu Y. Gu H. Xu L. Li H. Wang D. Liu K. Zhao J. etal 2011. Hybrid recommendation models for binary user preference prediction problem. http:\/\/jmir.csail.mit.edu\/proceedings\/papers\/v18\/lai12a\/lai12a.pdf.  Laiy S. Xiang L. Diao R. Liu Y. Gu H. Xu L. Li H. Wang D. Liu K. Zhao J. et al. 2011. Hybrid recommendation models for binary user preference prediction problem. http:\/\/jmir.csail.mit.edu\/proceedings\/papers\/v18\/lai12a\/lai12a.pdf."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1093\/rfs\/3.3.431"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the International Joint Conference on Neural Networks (IJCNN'11)","author":"Narayanan A."},{"key":"e_1_2_1_10_1","unstructured":"Nisbet R. Elder J. and Miner G. 2009. Handbook of Statistical Analysis and Data Mining Applications. Academic Press.   Nisbet R. Elder J. and Miner G. 2009. Handbook of Statistical Analysis and Data Mining Applications. Academic Press."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/82.4.669"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511803161"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1540276.1540289"},{"key":"e_1_2_1_14_1","unstructured":"Pyle D. 1999. Data Preparation for Data Mining. Morgan Kaufmann Publishers.   Pyle D. 1999. Data Preparation for Data Mining. Morgan Kaufmann Publishers."},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Pyle D. 2003. Business Modeling and Data Mining. Morgan Kaufmann Publishers.   Pyle D. 2003. Business Modeling and Data Mining. Morgan Kaufmann Publishers.","DOI":"10.1016\/B978-155860653-1.50005-1"},{"key":"e_1_2_1_16_1","volume-title":"Data Mining: Know it All","author":"Pyle D.","year":"2009"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-1842-5_4"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1345448.1345463"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-009-0158-x"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1281192.1281263"},{"key":"e_1_2_1_21_1","unstructured":"Tukey J. 1977. Exploratory Data Analysis. Addison-Wesley.  Tukey J. 1977. Exploratory Data Analysis. Addison-Wesley."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1018046501280"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/sam.v3:4"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2382577.2382579","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2382577.2382579","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T09:34:38Z","timestamp":1750239278000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2382577.2382579"}},"subtitle":["Formulation, detection, and avoidance"],"short-title":[],"issued":{"date-parts":[[2012,12]]},"references-count":23,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2012,12]]}},"alternative-id":["10.1145\/2382577.2382579"],"URL":"https:\/\/doi.org\/10.1145\/2382577.2382579","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,12]]},"assertion":[{"value":"2011-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-12-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}