{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,6]],"date-time":"2026-01-06T05:14:56Z","timestamp":1767676496256},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,7]]},"abstract":"<jats:p>Assessing the progress made in AI and contributions to the state of the art is of major concern to\n\nthe community. Recently, Frechette et al. [2016]\n\nadvocated performing such analysis via the Shapley\n\nvalue, a concept from coalitional game theory. In\n\nthis paper, we argue that while this general idea is\n\nsound, it unfairly penalizes older algorithms that\n\nadvanced the state of the art when introduced, but\n\nwere then outperformed by modern counterparts.\n\nDriven by this observation, we introduce the temporal Shapley value, a measure that addresses this\n\nproblem while maintaining the desirable properties\n\nof the (classical) Shapley value. We use the tempo-\n\nral Shapley value to analyze the progress made in\n\n(i) the different versions of the Quicksort algorithm;\n\n(ii) the annual SAT competitions 2007\u20132014; (iii)\n\nan annual competition of Constraint Programming,\n\nnamely the MiniZinc challenge 2014\u20132016. Our\n\nanalysis reveals novel insights into the development\n\nmade in these important areas of research over time.<\/jats:p>","DOI":"10.24963\/ijcai.2018\/716","type":"proceedings-article","created":{"date-parts":[[2018,7,5]],"date-time":"2018-07-05T01:49:10Z","timestamp":1530755350000},"page":"5165-5171","source":"Crossref","is-referenced-by-count":4,"title":["Quantifying Algorithmic Improvements over Time"],"prefix":"10.24963","author":[{"given":"Lars","family":"Kotthoff","sequence":"first","affiliation":[{"name":"University of Wyoming"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandre","family":"Fr\u00e9chette","sequence":"additional","affiliation":[{"name":"University of British Columbia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tomasz","family":"Michalak","sequence":"additional","affiliation":[{"name":"University of Warsaw"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Talal","family":"Rahwan","sequence":"additional","affiliation":[{"name":"Masdar Institute of Science and Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Holger H.","family":"Hoos","sequence":"additional","affiliation":[{"name":"Universiteit Leiden"},{"name":"University of British Columbia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kevin","family":"Leyton-Brown","sequence":"additional","affiliation":[{"name":"University of British Columbia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"10584","event":{"number":"27","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"acronym":"IJCAI-2018","name":"Twenty-Seventh International Joint Conference on Artificial Intelligence {IJCAI-18}","start":{"date-parts":[[2018,7,13]]},"theme":"Artificial Intelligence","location":"Stockholm, Sweden","end":{"date-parts":[[2018,7,19]]}},"container-title":["Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2018,7,5]],"date-time":"2018-07-05T01:55:27Z","timestamp":1530755727000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2018\/716"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2018,7]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2018\/716","relation":{},"subject":[],"published":{"date-parts":[[2018,7]]}}}