{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T10:17:28Z","timestamp":1791281848173,"version":"4.1.0"},"reference-count":52,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Comput."],"published-print":{"date-parts":[[2012,1]]},"abstract":"<jats:p>We prove the following strong hardness result for learning: Given a distribution of labeled examples from the hypercube such that there exists a monomial consistent with $(1-\\epsilon)$ of the examples it is NP-hard to find a halfspace that is correct on $(1\/2+\\epsilon)$ of the examples for arbitrary constants $\\epsilon&gt;0$. In learning theory terms, weak agnostic learning of monomials is hard, even if one is allowed to output a hypothesis from the much bigger concept class of halfspaces. This hardness result subsumes a long line of previous results, including two recent hardness results for the proper learning of monomials and halfspaces. As an immediate corollary of our result we show that weak agnostic learning of decision lists is NP-hard. Our techniques are quite different from previous hardness proofs for learning. We define distributions on positive and negative examples for monomials whose first few moments match. We use the invariance principle to argue that regular halfspaces (all of whose coefficients have small absolute value relative to the total $\\ell_2$ norm) cannot distinguish between distributions whose first few moments match. For highly nonregular halfspaces, we use a structural lemma from recent work on fooling halfspaces to argue that they are \u201cjunta-like\u201d and one can zero out all but the top few coefficients without affecting the performance of the halfspace. The top few coefficients form the natural list decoding of a halfspace in the context of dictatorship tests\/label cover reductions. We note that unlike previous proofs based on the invariance principle which are only known to give unique games hardness, we are able to reduce from a version of the Label Cover problem that is known to be NP-hard. This has inspired followup work on bypassing the unique games conjecture in some optimal geometric inapproximability results.<\/jats:p>","DOI":"10.1137\/120865094","type":"journal-article","created":{"date-parts":[[2012,12,18]],"date-time":"2012-12-18T13:36:29Z","timestamp":1355837789000},"page":"1558-1590","source":"Crossref","is-referenced-by-count":60,"title":["Agnostic Learning of Monomials by Halfspaces Is Hard"],"prefix":"10.1137","volume":"41","author":[{"given":"Vitaly","family":"Feldman","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Venkatesan","family":"Guruswami","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Prasad","family":"Raghavendra","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2012,12,18]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1016\/S0304-3975(97)00115-1"},{"key":"atypb2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00116829"},{"key":"atypb3","doi-asserted-by":"crossref","unstructured":"B. Applebaum, B. Barak, and D. Xiao,\n                      On basing lower-bounds for learning on worst-case assumptions\n                      , in Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2008, pp. 211-220.","DOI":"10.1109\/FOCS.2008.35"},{"key":"atypb4","doi-asserted-by":"publisher","DOI":"10.1006\/jcss.1997.1472"},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007472513967"},{"key":"atypb6","doi-asserted-by":"publisher","DOI":"10.1016\/S0022-0000(03)00038-2"},{"key":"atypb7","doi-asserted-by":"publisher","DOI":"10.1007\/PL00013833"},{"key":"atypb8","doi-asserted-by":"publisher","DOI":"10.1016\/0020-0190(87)90114-1"},{"key":"atypb9","doi-asserted-by":"crossref","unstructured":"N. Bshouty and L. Burroughs,\n                      Bounds for the minimum disagreement problem with applications to learning theory\n                      , in Proceedings of the 22nd Conference on Computational Learning Theory (COLT), 2002, pp. 271-286.","DOI":"10.1007\/3-540-45435-7_19"},{"key":"atypb10","doi-asserted-by":"publisher","DOI":"10.1016\/j.tcs.2005.10.014"},{"key":"atypb11","doi-asserted-by":"crossref","unstructured":"T. Bylander,\n                      Learning linear threshold functions in the presence of classification noise\n                      , in Proceedings of the 7th Annual ACM Conference on Computational Learning Theory (COLT), 1994, pp. 340-347.","DOI":"10.1145\/180139.181176"},{"key":"atypb12","unstructured":"S. Chatterjee,\n                      A simple invariance theorem\n                      , preprint, 2005; arxiv:math\/0508213v1."},{"key":"atypb13","doi-asserted-by":"crossref","unstructured":"E. Cohen,\n                      Learning noisy perceptrons by a perceptron in polynomial time\n                      , in Proceedings of the 38th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 1997, pp. 514-523.","DOI":"10.1109\/SFCS.1997.646140"},{"key":"atypb14","doi-asserted-by":"crossref","unstructured":"I. Diakonikolas, P. Gopalan, R. Jaiswal, R. A. Servedio, and E. Viola,\n                      Bounded independence fools halfspaces\n                      , in Proceedings of the 50th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2009, pp. 171-180.","DOI":"10.1109\/FOCS.2009.68"},{"key":"atypb15","doi-asserted-by":"crossref","unstructured":"I. Diakonikolas, R. O'Donnell, R. Servedio, and Y. Wu,\n                      Hardness results for agnostically learning low-degree polynomial threshold functions\n                      , in Proceedings of the 22nd Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2011, pp. 1590-1606.","DOI":"10.1137\/1.9781611973082.123"},{"key":"atypb16","doi-asserted-by":"crossref","unstructured":"V. Feldman,\n                      Optimal hardness results for maximizing agreements with monomials\n                      , in Proceedings of the 21st Annual IEEE Conference on Computational Complexity (CCC), 2006, pp. 226-236.","DOI":"10.1109\/CCC.2006.31"},{"key":"atypb17","doi-asserted-by":"crossref","unstructured":"V. Feldman,\n                      A complete characterization of statistical query learning with applications to evolvability\n                      , in Proceedings of the 50th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2009, pp. 375-384.","DOI":"10.1109\/FOCS.2009.35"},{"key":"atypb18","doi-asserted-by":"publisher","DOI":"10.1137\/070684914"},{"key":"atypb19","doi-asserted-by":"publisher","DOI":"10.1006\/inco.1995.1136"},{"key":"atypb20","doi-asserted-by":"publisher","DOI":"10.1109\/72.80230"},{"key":"atypb21","unstructured":"M. Garey and D. S. Johnson,\n                      Computers and Intractability\n                      , W. H. Freeman, San Francisco, CA, 1979."},{"key":"atypb22","unstructured":"C. Gentile and M. K. Warmuth,\n                      Linear hinge loss and average margin\n                      , in Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems (NIPS), 1998, pp. 225-231."},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1137\/070705258"},{"key":"atypb24","doi-asserted-by":"publisher","DOI":"10.1137\/070685798"},{"key":"atypb25","doi-asserted-by":"crossref","unstructured":"V. Guruswami, P. Raghavendra, R. Saket, and Y. Wu,\n                      Bypassing UGC from some optimal geometric inapproximability results\n                      , in Proceedings of the 23rd Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2012, pp. 699-717.","DOI":"10.1137\/1.9781611973099.58"},{"key":"atypb26","doi-asserted-by":"publisher","DOI":"10.1016\/0890-5401(92)90010-D"},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1006\/jcss.1995.1011"},{"key":"atypb28","doi-asserted-by":"publisher","DOI":"10.1016\/0304-3975(78)90006-3"},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1137\/060649057"},{"key":"atypb30","doi-asserted-by":"publisher","DOI":"10.1145\/293347.293351"},{"key":"atypb31","doi-asserted-by":"publisher","DOI":"10.1007\/BF00993468"},{"key":"atypb32","doi-asserted-by":"publisher","DOI":"10.1137\/0222052"},{"key":"atypb33","doi-asserted-by":"publisher","DOI":"10.1016\/S0022-0000(05)80062-5"},{"key":"atypb34","doi-asserted-by":"crossref","unstructured":"S. Khot,\n                      On the power of unique 2-prover 1-round games\n                      , in in Proceedings of the 34th ACM Symposium on Theory of Computing (STOC), 2002, pp. 767-775.","DOI":"10.1145\/509907.510017"},{"key":"atypb35","unstructured":"S. Khot,\n                      New Techniques for Probabilistically Checkable Proofs and Inapproximability Results (Thesis)\n                      , Technical Report TR-673-03, Princeton University, Princeton, NJ, 2003."},{"key":"atypb36","doi-asserted-by":"publisher","DOI":"10.1137\/S0097539705447372"},{"key":"atypb37","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcss.2007.06.019"},{"key":"atypb38","doi-asserted-by":"crossref","unstructured":"S. Khot and R. Saket,\n                      Hardness of minimizing and learning DNF expressions\n                      , in Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2008, pp. 231-240.","DOI":"10.1109\/FOCS.2008.37"},{"key":"atypb39","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-007-5010-1"},{"key":"atypb40","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcss.2008.07.008"},{"key":"atypb41","doi-asserted-by":"crossref","unstructured":"W. S. Lee, P. L. Bartlett, and R. C. Williamson,\n                      On efficient agnostic learning of linear combinations of basis functions\n                      , in Proceedings of the 8th Annual Conference on Computational Learning Theory (COLT), 1995, pp. 369-376.","DOI":"10.1145\/225298.225343"},{"key":"atypb42","doi-asserted-by":"publisher","DOI":"10.1007\/BF00116827"},{"key":"atypb43","doi-asserted-by":"crossref","unstructured":"K. Matulef, R. O'Donnell, R. Rubinfeld, and R. A. Servedio,\n                      Testing halfspaces\n                      , in Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2009, pp. 256-264.","DOI":"10.1137\/1.9781611973068.29"},{"key":"atypb44","doi-asserted-by":"crossref","unstructured":"E. Mossel,\n                      Gaussian bounds for noise correlation of functions\n                      , in Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2008, pp. 156-165.","DOI":"10.1109\/FOCS.2008.44"},{"key":"atypb45","doi-asserted-by":"crossref","unstructured":"E. Mossel, R. O'Donnell, and K. Oleszkiewicz,\n                      Noise stability of functions with low influences: Invariance and optimality\n                      , in Proceedings of the 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2005, pp. 21-30.","DOI":"10.1109\/SFCS.2005.53"},{"key":"atypb46","doi-asserted-by":"crossref","unstructured":"R. O'Donnell and R. A. Servedio,\n                      The Chow parameters problem\n                      , in Proceedings of the 40th Annual ACM Symposium on Theory of Computing (STOC), 2008, pp. 517-526.","DOI":"10.1145\/1374376.1374450"},{"key":"atypb47","doi-asserted-by":"publisher","DOI":"10.1007\/BF00058680"},{"key":"atypb48","doi-asserted-by":"publisher","DOI":"10.1037\/h0042519"},{"key":"atypb49","doi-asserted-by":"publisher","DOI":"10.1007\/BF00116037"},{"key":"atypb50","doi-asserted-by":"publisher","DOI":"10.1007\/s00037-007-0228-7"},{"key":"atypb51","doi-asserted-by":"publisher","DOI":"10.1145\/1968.1972"},{"key":"atypb52","unstructured":"V. Vapnik,\n                      Statistical Learning Theory\n                      , John Wiley, New York, 1998."}],"container-title":["SIAM Journal on Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/120865094","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T19:05:30Z","timestamp":1787339130000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/120865094"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,1]]},"references-count":52,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2012,1]]}},"alternative-id":["10.1137\/120865094"],"URL":"https:\/\/doi.org\/10.1137\/120865094","relation":{},"ISSN":["0097-5397","1095-7111"],"issn-type":[{"value":"0097-5397","type":"print"},{"value":"1095-7111","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,1]]}}}