{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T03:57:40Z","timestamp":1787889460564,"version":"build-2784847793"},"reference-count":45,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2020,11,4]],"date-time":"2020-11-04T00:00:00Z","timestamp":1604448000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,11,15]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>A property, or statistical functional, is said to be elicitable if it minimizes the expected loss for some loss function. The study of which properties are elicitable sheds light on the capabilities and limitations of point estimation and empirical risk minimization. While recent work has sought to identify which properties are elicitable, here we investigate a more nuanced question: how many dimensions are required to indirectly elicit a given property? This number is called the elicitation complexity of the property. We lay the foundation for a general theory of elicitation complexity, which includes several basic results on how elicitation complexity behaves and the complexity of standard properties of interest. Building on this foundation, our main result gives tight complexity bounds for the broad class of Bayes risks. We apply these results to several properties of interest, including variance, entropy, norms and several classes of financial risk measures. The article concludes with a discussion and open questions.<\/jats:p>","DOI":"10.1093\/biomet\/asaa093","type":"journal-article","created":{"date-parts":[[2020,10,22]],"date-time":"2020-10-22T11:10:04Z","timestamp":1603365004000},"page":"857-879","source":"Crossref","is-referenced-by-count":13,"title":["Elicitation complexity of statistical properties"],"prefix":"10.1093","volume":"108","author":[{"given":"Rafael M","family":"Frongillo","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Colorado Boulder, 1111 Engineering Drive, Boulder, Colorado 80309, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ian A","family":"Kash","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Illinois Chicago, 851 S. Morgan (M\/C 152), Room 1120 SEO, Chicago, Illinois 60607, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,11,4]]},"reference":[{"key":"2021111803493837700_asaa093-B1","first-page":"27.1","article-title":"A characterization of scoring rules for linear properties","volume":"23","author":"Abernethy","year":"2012","journal-title":"J. Mach. Learn. Res. Workshop Conf. Proc."},{"key":"2021111803493837700_asaa093-B2","first-page":"4","article-title":"On consistent surrogate risk minimization and property elicitation","volume":"40","author":"Agarwal","year":"2015","journal-title":"Proc. Mach. Learn. Res."},{"key":"2021111803493837700_asaa093-B3","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1007\/s10479-017-2441-3","article-title":"On the dual representation of coherent risk measures","volume":"262","author":"Ang","year":"2018","journal-title":"Ann. Oper. Res."},{"key":"2021111803493837700_asaa093-B4","article-title":"The hardness of approximate optima in lattices, codes, and systems of linear equations","author":"Arora","year":"1993","journal-title":"Proc. 34th IEEE Annu. Sympos. Foundations of Computer Science"},{"key":"2021111803493837700_asaa093-B5","doi-asserted-by":"crossref","first-page":"2664","DOI":"10.1109\/TIT.2005.850145","article-title":"On the optimality of conditional expectation as a Bregman predictor","volume":"51","author":"Banerjee","year":"2005","journal-title":"IEEE Trans. Info. Theory"},{"key":"2021111803493837700_asaa093-B6","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1198\/016214505000000907","article-title":"Convexity, classification, and risk bounds","volume":"101","author":"Bartlett","year":"2006","journal-title":"J. Am. Statist. Assoc."},{"key":"2021111803493837700_asaa093-B7","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1080\/14697688.2014.946955","article-title":"On elicitable risk measures","volume":"15","author":"Bellini","year":"2015","journal-title":"Quant. Finan."},{"key":"2021111803493837700_asaa093-B8","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1111\/j.1467-9965.2007.00311.x","article-title":"An old-new concept of convex risk measures: The optimized certainty equivalent","volume":"17","author":"Ben-Tal","year":"2007","journal-title":"Math. Finan."},{"key":"2021111803493837700_asaa093-B9","first-page":"449","article-title":"Multi-observation elicitation","volume":"65","author":"Casalaina-Martin","year":"2017","journal-title":"Proc. Mach. Learn. Res."},{"key":"2021111803493837700_asaa093-B10","doi-asserted-by":"crossref","first-page":"1095","DOI":"10.1007\/s10463-019-00719-1","article-title":"On the indirect elicitability of the mode and modal interval","volume":"72","author":"Dearborn","year":"2020","journal-title":"Ann. Inst. Statist. Math."},{"key":"2021111803493837700_asaa093-B11","first-page":"1","article-title":"Coherent risk measures on general probability spaces","author":"Delbaen","year":"2002","journal-title":"Advances in Finance and Stochastics"},{"key":"2021111803493837700_asaa093-B12","doi-asserted-by":"crossref","first-page":"31","DOI":"10.21314\/JOR.2015.318","article-title":"What is the best risk measure in practice? A comparison of standard measures","volume":"18","author":"Emmer","year":"2015","journal-title":"J. Risk"},{"key":"2021111803493837700_asaa093-B13","article-title":"Expected shortfall is jointly elicitable with value-at-risk: Implications for backtesting","author":"Fissler","year":"2016","journal-title":"Risk Mag."},{"key":"2021111803493837700_asaa093-B14","doi-asserted-by":"crossref","first-page":"1680","DOI":"10.1214\/16-AOS1439","article-title":"Higher order elicitability and Osband\u2019s principle","volume":"44","author":"Fissler","year":"2016","journal-title":"Ann. Statist."},{"key":"2021111803493837700_asaa093-B15","article-title":"Elicitability of range value at risk","author":"Fissler","year":"2019","journal-title":"arXiv:"},{"key":"2021111803493837700_asaa093-B16","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1214\/19-EJS1552","article-title":"Order-sensitivity and equivariance of scoring functions","volume":"13","author":"Fissler","year":"2019","journal-title":"Electron. J. Statist."},{"key":"2021111803493837700_asaa093-B17","doi-asserted-by":"crossref","DOI":"10.1515\/9783110212075","author":"F\u00f6llmer","year":"2004","journal-title":"Stochastic Finance: An Introduction in Discrete Time"},{"key":"2021111803493837700_asaa093-B18","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1146\/annurev-financial-111914-042031","article-title":"The axiomatic approach to risk measures for capital determination","volume":"7","author":"F\u00f6llmer","year":"2015","journal-title":"Annu. Rev. Finan. Econ."},{"key":"2021111803493837700_asaa093-B19","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1007\/978-3-319-13129-0_29","article-title":"General truthfulness characterizations via convex analysis","author":"Frongillo","year":"2014","journal-title":"Proc. Int. Conf. Web and Internet Economics"},{"key":"2021111803493837700_asaa093-B20","first-page":"710","article-title":"Vector-valued property elicitation","volume":"40","author":"Frongillo","year":"2015","journal-title":"Proc. Mach. Learn. Res."},{"key":"2021111803493837700_asaa093-B21","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1198\/jasa.2011.r10138","article-title":"Making and evaluating point forecasts","volume":"106","author":"Gneiting","year":"2011","journal-title":"J. Am. Statist. Assoc."},{"key":"2021111803493837700_asaa093-B22","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1198\/016214506000001437","article-title":"Strictly proper scoring rules, prediction, and estimation","volume":"102","author":"Gneiting","year":"2007","journal-title":"J. Am. Statist. Assoc."},{"key":"2021111803493837700_asaa093-B23","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1145\/307400.307436","article-title":"Viewing all models as \u201cprobabilistic\u201d","author":"Gr\u00fcnwald","year":"1999","journal-title":"Proc. 12th Annu. Conf. Computational Learning Theory"},{"key":"2021111803493837700_asaa093-B24","first-page":"293","article-title":"That simple device already used by Gauss","author":"Gr\u00fcnwald","year":"2008","journal-title":"Festschrift in Honor of Jorma Rissanen on the Occasion of his 75th Birthday"},{"key":"2021111803493837700_asaa093-B25","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1093\/biomet\/ast048","article-title":"The mode functional is not elicitable","volume":"101","author":"Heinrich","year":"2013","journal-title":"Biometrika"},{"key":"2021111803493837700_asaa093-B26","doi-asserted-by":"crossref","first-page":"629","DOI":"10.1080\/03461238.2018.1426038","article-title":"Multivariate geometric expectiles","volume":"2018","author":"Herrmann","year":"2018","journal-title":"Scand. Actuar. J."},{"key":"2021111803493837700_asaa093-B27","author":"Hrbacek","year":"1999","journal-title":"Introduction to Set Theory"},{"key":"2021111803493837700_asaa093-B28","article-title":"Elicitation and evaluation of statistical forecasts","author":"Lambert","year":"2018"},{"key":"2021111803493837700_asaa093-B29","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1145\/1386790.1386813","article-title":"Eliciting properties of probability distributions","author":"Lambert","year":"2008","journal-title":"Proc. 9th ACM Conf. Electronic Commerce"},{"key":"2021111803493837700_asaa093-B30","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1145\/1566374.1566391","article-title":"Eliciting truthful answers to multiple-choice questions","author":"Lambert","year":"2009","journal-title":"Proc. 10th ACM Conf. Electronic Commerce"},{"key":"2021111803493837700_asaa093-B31","doi-asserted-by":"crossref","first-page":"819","DOI":"10.2307\/1911031","article-title":"Asymmetric least squares estimation and testing","volume":"55","author":"Newey","year":"1987","journal-title":"Econometrica"},{"key":"2021111803493837700_asaa093-B32","author":"Osband","year":"1985","journal-title":"Providing Incentives for Better Cost Forecasting"},{"key":"2021111803493837700_asaa093-B33","first-page":"1475","article-title":"Convex calibrated surrogates for low-rank loss matrices with applications to subset ranking losses","author":"Ramaswamy","year":"2013","journal-title":"Advances in Neural Information Processing Systems 26 (NIPS 2013)"},{"key":"2021111803493837700_asaa093-B34","first-page":"397","article-title":"Convex calibration dimension for multiclass loss matrices","volume":"17","author":"Ramaswamy","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"2021111803493837700_asaa093-B35","first-page":"68","article-title":"Convexity properties of entropy functions and analysis of diversity","volume":"5","author":"Rao","year":"1984","journal-title":"IMS Lect. Not. Monog. Ser."},{"key":"2021111803493837700_asaa093-B36","first-page":"151","article-title":"Superquantiles and their applications to risk, random variables, and regression","author":"Rockafellar","year":"2013","journal-title":"Theory Driven by Influential Applications"},{"key":"2021111803493837700_asaa093-B37","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s10479-016-2129-0","article-title":"Superquantile\/CVaR risk measures: Second-order theory","volume":"262","author":"Rockafellar","year":"2018","journal-title":"Ann. Oper. Res."},{"key":"2021111803493837700_asaa093-B38","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1016\/j.ejor.2013.10.046","article-title":"Superquantile regression with applications to buffered reliability, uncertainty quantification, and conditional value-at-risk","volume":"234","author":"Rockafellar","year":"2014","journal-title":"Eur. J. Oper. Res."},{"key":"2021111803493837700_asaa093-B39","first-page":"33","article-title":"The fundamental risk quadrangle in risk management, optimization and statistical estimation","volume":"18","author":"Rockafellar","year":"2013","journal-title":"Surv. Oper. Res. Manag. Sci."},{"key":"2021111803493837700_asaa093-B40","doi-asserted-by":"crossref","first-page":"783","DOI":"10.1080\/01621459.1971.10482346","article-title":"Elicitation of personal probabilities and expectations","volume":"66","author":"Savage","year":"1971","journal-title":"J. Am. Statist. Assoc."},{"key":"2021111803493837700_asaa093-B41","author":"Steinwart","year":"2008","journal-title":"Support Vector Machines"},{"key":"2021111803493837700_asaa093-B42","first-page":"482","article-title":"Elicitation and identification of properties","volume":"35","author":"Steinwart","year":"2014","journal-title":"Proc. Mach. Learn. Res."},{"key":"2021111803493837700_asaa093-B43","doi-asserted-by":"crossref","first-page":"1337","DOI":"10.1111\/mafi.12270","article-title":"Risk functionals with convex level sets","volume":"30","author":"Wang","year":"2018","journal-title":"Math. Finan."},{"key":"2021111803493837700_asaa093-B44","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/j.spl.2015.02.004","article-title":"Elicitable distortion risk measures: A concise proof","volume":"100","author":"Wang","year":"2015","journal-title":"Statist. Prob. Lett."},{"key":"2021111803493837700_asaa093-B45","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1111\/mafi.12080","article-title":"Coherence and elicitability","volume":"26","author":"Ziegel","year":"2016","journal-title":"Math. Finan."}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/biomet\/advance-article-pdf\/doi\/10.1093\/biomet\/asaa093\/36332707\/asaa093.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/108\/4\/857\/41153633\/asaa093.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/108\/4\/857\/41153633\/asaa093.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,10]],"date-time":"2023-10-10T10:24:18Z","timestamp":1696933458000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/108\/4\/857\/5955754"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,4]]},"references-count":45,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,11,4]]},"published-print":{"date-parts":[[2021,11,15]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asaa093","relation":{},"ISSN":["0006-3444","1464-3510"],"issn-type":[{"value":"0006-3444","type":"print"},{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,12,1]]},"published":{"date-parts":[[2020,11,4]]}}}