{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:33:34Z","timestamp":1750307614058,"version":"3.41.0"},"reference-count":31,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2009,3,1]],"date-time":"2009-03-01T00:00:00Z","timestamp":1235865600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000145","name":"Division of Information and Intelligent Systems","doi-asserted-by":"publisher","award":["NSF-CAREER-IIS-0448264"],"award-info":[{"award-number":["NSF-CAREER-IIS-0448264"]}],"id":[{"id":"10.13039\/100000145","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2009,3]]},"abstract":"<jats:p>In this article we propose a moment-based method for studying models and model selection measures. By focusing on the probabilistic space of classifiers induced by the classification algorithm rather than on that of datasets, we obtain efficient characterizations for computing the moments, which is followed by visualization of the resulting formulae that are too complicated for direct interpretation. By assuming the data to be drawn independently and identically distributed from the underlying probability distribution, and by going over the space of all possible datasets, we establish general relationships between the generalization error, hold-out-set error, cross-validation error, and leave-one-out error. We later exemplify the method and the results by studying the behavior of the errors for the naive Bayes classifier.<\/jats:p>","DOI":"10.1145\/1497577.1497579","type":"journal-article","created":{"date-parts":[[2009,4,6]],"date-time":"2009-04-06T16:34:22Z","timestamp":1239035662000},"page":"1-51","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Semi-analytical method for analyzing models and model selection measures based on moment analysis"],"prefix":"10.1145","volume":"3","author":[{"given":"Amit","family":"Dhurandhar","sequence":"first","affiliation":[{"name":"University of Florida, Gainesville, FL"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alin","family":"Dobra","sequence":"additional","affiliation":[{"name":"University of Florida, Gainesville, FL"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2009,3,23]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Bengio Y. and Grandvalet Y. 2003. No unbiased estimator of the variance of k-fold cross validation. J. Mach. Learn. Res.   Bengio Y. and Grandvalet Y. 2003. No unbiased estimator of the variance of k-fold cross validation. J. Mach. Learn. Res."},{"key":"e_1_2_1_2_1","unstructured":"Bertsimas D. and Popescu I. 1998. Optimal inequalities in probability theory: A convex optimization approach. Tech. rep. Department of Mathematics O.R. Cambridge Massachusetts 02139.  Bertsimas D. and Popescu I. 1998. Optimal inequalities in probability theory: A convex optimization approach. Tech. rep. Department of Mathematics O.R. Cambridge Massachusetts 02139."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/307400.307439"},{"key":"e_1_2_1_4_1","unstructured":"Boucheron S. Bousquet O. and Lugosi G. 2005. Introduction to statistical learning theory. http:\/\/www.kyb.mpg.de\/publications\/pdfs\/pdf2819.pdf.  Boucheron S. Bousquet O. and Lugosi G. 2005. Introduction to statistical learning theory. http:\/\/www.kyb.mpg.de\/publications\/pdfs\/pdf2819.pdf."},{"key":"e_1_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Boyd S. and Vandenberghe L. 2004. Convex Optimization. Cambridge.   Boyd S. and Vandenberghe L. 2004. Convex Optimization. Cambridge.","DOI":"10.1017\/CBO9780511804441"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1032181158"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1998.10473713"},{"key":"e_1_2_1_8_1","unstructured":"Connor-Linton J. 2003. Chi square tutorial. http:\/\/www.georgetown.edu\/faculty\/ballc\/webtools\/web_chi_tut.html.  Connor-Linton J. 2003. Chi square tutorial. http:\/\/www.georgetown.edu\/faculty\/ballc\/webtools\/web_chi_tut.html."},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Devroye L. Gy\u00f6rfi L. and Lugosi G. 1996. A Probabilistic Theory of Pattern Recognition. Springer.  Devroye L. Gy\u00f6rfi L. and Lugosi G. 1996. A Probabilistic Theory of Pattern Recognition. Springer.","DOI":"10.1007\/978-1-4612-0711-5"},{"volume-title":"Measure Theory","author":"Doob J.","key":"e_1_2_1_10_1","unstructured":"Doob , J. 1994. Measure Theory . Springer . Doob, J. 1994. Measure Theory. Springer."},{"key":"e_1_2_1_11_1","unstructured":"Elisseeff A. and Pontil M. 2003. Leave-one-out error and stability of learning algorithms with applications. In Learning Theory and Practice. IOS Press.  Elisseeff A. and Pontil M. 2003. Leave-one-out error and stability of learning algorithms with applications. In Learning Theory and Practice. IOS Press."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.6.1245"},{"volume-title":"The Bootstrap and Edgeworth Expansion","author":"Hall P.","key":"e_1_2_1_13_1","unstructured":"Hall , P. 1992. The Bootstrap and Edgeworth Expansion . Springer . Hall, P. 1992. The Bootstrap and Edgeworth Expansion. Springer."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01733120"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02868641"},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Karlin S. and Shapely L. 1953. Geometry of moment spaces. Memoirs Amer. Math. Soc. 12.  Karlin S. and Shapely L. 1953. Geometry of moment spaces. Memoirs Amer. Math. Soc. 12.","DOI":"10.1090\/memo\/0012"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/267460.267491"},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the 14th International Joint Conference on Artificial Intelligence. Morgan Kaufmann, 1137--1143","author":"Kohavi R.","year":"1995","unstructured":"Kohavi , R. 1995 . A study of cross-validation and bootstrap for accuracy estimation and model selection . In Proceedings of the 14th International Joint Conference on Artificial Intelligence. Morgan Kaufmann, 1137--1143 . Kohavi, R. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence. Morgan Kaufmann, 1137--1143."},{"volume-title":"Proceedings of the 18th Annual Conference on Uncertainty in Artificial Intelligence (UAI-02)","author":"Kutin S.","key":"e_1_2_1_19_1","unstructured":"Kutin , S. and Niyogi , P . 2002. Almost-Everywhere algorithmic stability and generalization error . In Proceedings of the 18th Annual Conference on Uncertainty in Artificial Intelligence (UAI-02) . Morgan Kaufmann Publishers, 275--282. Kutin, S. and Niyogi, P. 2002. Almost-Everywhere algorithmic stability and generalization error. In Proceedings of the 18th Annual Conference on Uncertainty in Artificial Intelligence (UAI-02). Morgan Kaufmann Publishers, 275--282."},{"key":"e_1_2_1_20_1","unstructured":"Langford J. 2005. Filed under: Prediction theory problems. http:\/\/hunch.net\/index.php?p=29.  Langford J. 2005. Filed under: Prediction theory problems. http:\/\/hunch.net\/index.php?p=29."},{"volume-title":"Proceedings of the 16th International Conference on Machine Learning. Morgan Kaufmann, 220--228","author":"Langley P.","key":"e_1_2_1_21_1","unstructured":"Langley , P. and Sage , S . 1999. Tractable average-case analysis of naive Bayesian classifiers . In Proceedings of the 16th International Conference on Machine Learning. Morgan Kaufmann, 220--228 . Langley, P. and Sage, S. 1999. Tractable average-case analysis of naive Bayesian classifiers. In Proceedings of the 16th International Conference on Machine Learning. Morgan Kaufmann, 220--228."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176345593"},{"volume-title":"Proceedings of the International Conference on Machine Learning, 190--198","author":"Moore A.","key":"e_1_2_1_23_1","unstructured":"Moore , A. and Lee , M . 1994. Efficient algorithms for minimizing cross validation error . In Proceedings of the International Conference on Machine Learning, 190--198 . Moore, A. and Lee, M. 1994. Efficient algorithms for minimizing cross validation error. In Proceedings of the International Conference on Machine Learning, 190--198."},{"key":"e_1_2_1_24_1","volume-title":"Survey: Cross-validation in theory and in practice. www.emotivate.com\/CvSurvey.doc.","author":"Plutowski M.","year":"1996","unstructured":"Plutowski , M. 1996 . Survey: Cross-validation in theory and in practice. www.emotivate.com\/CvSurvey.doc. Plutowski, M. 1996. Survey: Cross-validation in theory and in practice. www.emotivate.com\/CvSurvey.doc."},{"key":"e_1_2_1_25_1","unstructured":"Prekopa A. 1989. The discrete moment problem and linear programming. RUTCOR Res. rep.  Prekopa A. 1989. The discrete moment problem and linear programming. RUTCOR Res. rep."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1993.10476299"},{"volume-title":"Statistical Learning Theory","author":"Vapnik V.","key":"e_1_2_1_27_1","unstructured":"Vapnik , V. 1998. Statistical Learning Theory . Wiley & amp; Sons. Vapnik, V. 1998. Statistical Learning Theory. Wiley &amp; Sons."},{"key":"e_1_2_1_28_1","unstructured":"Williamson R. 2001. Srm and vc theory (statistical learning theory). http:\/\/axiom.anu.edu.au\/williams\/papers\/P151.pdf.  Williamson R. 2001. Srm and vc theory (statistical learning theory). http:\/\/axiom.anu.edu.au\/williams\/papers\/P151.pdf."},{"key":"e_1_2_1_29_1","unstructured":"Wolfram-Research. 2009. Mathematica. http:\/\/www.wolfram.com\/.  Wolfram-Research. 2009. Mathematica. http:\/\/www.wolfram.com\/."},{"key":"e_1_2_1_30_1","volume-title":"Sdpsol: A parser\/solver for sdp and maxdet problems with matrix structure","author":"Wu S.-P.","year":"1996","unstructured":"Wu , S.-P. and Boyd , S . 1996 . Sdpsol: A parser\/solver for sdp and maxdet problems with matrix structure . http:\/\/www.stanford.edu\/boyd\/SDPSOL.html. Wu, S.-P. and Boyd, S. 1996. Sdpsol: A parser\/solver for sdp and maxdet problems with matrix structure. http:\/\/www.stanford.edu\/boyd\/SDPSOL.html."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1996.8.7.1421"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1497577.1497579","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1497577.1497579","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T12:45:43Z","timestamp":1750250743000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1497577.1497579"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,3]]},"references-count":31,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2009,3]]}},"alternative-id":["10.1145\/1497577.1497579"],"URL":"https:\/\/doi.org\/10.1145\/1497577.1497579","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"type":"print","value":"1556-4681"},{"type":"electronic","value":"1556-472X"}],"subject":[],"published":{"date-parts":[[2009,3]]},"assertion":[{"value":"2008-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2008-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2009-03-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}