{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T09:03:53Z","timestamp":1773738233452,"version":"3.50.1"},"reference-count":13,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2020,12,2]],"date-time":"2020-12-02T00:00:00Z","timestamp":1606867200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGecom Exch."],"published-print":{"date-parts":[[2020,12,2]]},"abstract":"<jats:p>We summarize our recent work that uses machine learning techniques as a complement to theoretical modeling, rather than a substitute for it. The key concepts are those of the completeness and restrictiveness of a model. A theory's completeness is how much it improves predictions over a naive baseline, relative to how much improvement is possible. When a theory is relatively incomplete, machine learning algorithms can help reveal regularities that the theory doesn't capture, and thus lead to the construction of theories that make more accurate predictions. Restrictiveness measures a theory's ability to match arbitrary hypothetical data: A very unrestrictive theory will be complete on almost any data, so the fact that it is complete on the actual data is not very instructive. We algorithmically quantify restrictiveness by measuring how well the theory approximates randomly generated behaviors. Finally, we propose \"algorithmic experimental design\" as a method to help select which experiments to run.<\/jats:p>","DOI":"10.1145\/3440959.3440962","type":"journal-article","created":{"date-parts":[[2020,12,2]],"date-time":"2020-12-02T23:58:00Z","timestamp":1606953480000},"page":"4-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Machine learning for evaluating and improving theories"],"prefix":"10.1145","volume":"18","author":[{"given":"Drew","family":"Fudenberg","sequence":"first","affiliation":[{"name":"MIT"}]},{"given":"Annie","family":"Liang","sequence":"additional","affiliation":[{"name":"University of Pennsylvania"}]}],"member":"320","published-online":{"date-parts":[[2020,12,2]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.3982\/ECTA7139"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1162\/0033553041502225"},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Fudenberg D. Gao W. and Liang A. 2020. Quantifying the restrictiveness of theories. Working Paper; available at http:\/\/economics.mit.edu\/faculty\/drewf\/working.  Fudenberg D. Gao W. and Liang A. 2020. Quantifying the restrictiveness of theories. Working Paper; available at http:\/\/economics.mit.edu\/faculty\/drewf\/working.","DOI":"10.2139\/ssrn.3580408"},{"key":"e_1_2_1_4_1","unstructured":"Fudenberg D. Kleinberg J. Liang A. and Mullainathan S. 2019. Measuring the com- pleteness of theories. Working Paper; available at http:\/\/economics.mit.edu\/faculty\/drewf\/ working.  Fudenberg D. Kleinberg J. Liang A. and Mullainathan S. 2019. Measuring the com- pleteness of theories. Working Paper; available at http:\/\/economics.mit.edu\/faculty\/drewf\/ working."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.20180654"},{"key":"e_1_2_1_6_1","doi-asserted-by":"crossref","unstructured":"Goldstein W. M. and Einhorn H. J. 1987. Expression theory and the preference reversal phenomena. Psychological review 94 2 236.  Goldstein W. M. and Einhorn H. J. 1987. Expression theory and the preference reversal phenomena. Psychological review 94 2 236.","DOI":"10.1037\/0033-295X.94.2.236"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/2969033.2969125"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2046684.2046692"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-2681(95)90015-2"},{"key":"e_1_2_1_10_1","first-page":"1313","article-title":"Unraveling in guessing games: An experimental study","volume":"85","author":"Nagel R.","year":"1995","journal-title":"American Economic Review"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/0167-2681(94)90103-1"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1006\/game.1995.1031"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2600057.2602907"}],"container-title":["ACM SIGecom Exchanges"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3440959.3440962","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3440959.3440962","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:51Z","timestamp":1750197771000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3440959.3440962"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,2]]},"references-count":13,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,12,2]]}},"alternative-id":["10.1145\/3440959.3440962"],"URL":"https:\/\/doi.org\/10.1145\/3440959.3440962","relation":{},"ISSN":["1551-9031"],"issn-type":[{"value":"1551-9031","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,2]]},"assertion":[{"value":"2020-12-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}