{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T11:09:33Z","timestamp":1783422573731,"version":"3.54.6"},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2025,2,12]],"date-time":"2025-02-12T00:00:00Z","timestamp":1739318400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Digit. Gov.: Res. Pract."],"published-print":{"date-parts":[[2025,3,31]]},"abstract":"<jats:p>The recent developments in artificial intelligence (AI) are revolutionizing professional practices across various professional fields, including evaluation. With its advanced automation and learning capabilities, AI is bringing significant changing to the way organizations and societies function. Evaluation is no exception to this trend, even though evaluators are adopting AI at a slower pace. This article examines ongoing applications that already improve and enhance the evaluation practice. We advance our discussion by exploring the potential impact of AI on the policy cycle. Subsequently, we analyze the potential incorporation of evaluation into autonomous AI systems that could design, implement, and evaluate public policies with minimal to no human supervision.<\/jats:p>","DOI":"10.1145\/3696009","type":"journal-article","created":{"date-parts":[[2024,9,13]],"date-time":"2024-09-13T08:15:25Z","timestamp":1726215325000},"page":"1-10","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Artificial Intelligence and the Future of Evaluation: From Augmented to Automated Evaluation"],"prefix":"10.1145","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8301-2151","authenticated-orcid":false,"given":"Steve","family":"Jacob","sequence":"first","affiliation":[{"name":"Universit\u00e9 Laval, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,2,12]]},"reference":[{"key":"e_1_3_1_2_1","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.35179"},{"key":"e_1_3_1_3_1","unstructured":"M. Ashoori and J. Weisz. 2019. In AI we trust? Factors that influence trustworthiness of AI-infused decision-making processes. arXiv:1912.02675. Retrieved from https:\/\/arxiv.org\/abs\/1912.02675"},{"key":"e_1_3_1_4_1","first-page":"74","article-title":"The culture of evaluation in the age of big data: The need for a new evaluation paradigm for the 4th industrial revolution","volume":"1","author":"Bamberger M.","year":"2020","unstructured":"M. Bamberger. 2020. The culture of evaluation in the age of big data: The need for a new evaluation paradigm for the 4th industrial revolution. Evaluation Matters, Q1 (2020), 74\u201385. https:\/\/idev.afdb.org\/sites\/default\/files\/Evaluations\/2020-07\/Evaluation%20Matters%20Q1-2020-En.pdf","journal-title":"Evaluation Matters"},{"key":"e_1_3_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3327174"},{"key":"e_1_3_1_6_1","doi-asserted-by":"publisher","DOI":"10.4324\/9781003512493"},{"key":"e_1_3_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-69128-8_2"},{"key":"e_1_3_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-822596-7.00009-7"},{"key":"e_1_3_1_9_1","unstructured":"DataFoundation. 2023. Issues in Evaluation: Using AI for Evaluation - Potential Risks Likely Benefits and What We Do Next. Retrieved from https:\/\/datafoundation.org\/news\/past-events\/275\/275-Issues-in-Evaluation-Using-AI-for-Evaluation-Potential-Risks-Likely-Benefits-What-We-Do-Next"},{"key":"e_1_3_1_10_1","doi-asserted-by":"publisher","DOI":"10.1177\/09520767231198737"},{"key":"e_1_3_1_11_1","doi-asserted-by":"publisher","DOI":"10.1002\/ev.20557"},{"key":"e_1_3_1_12_1","doi-asserted-by":"publisher","DOI":"10.1596\/37117"},{"key":"e_1_3_1_13_1","article-title":"All about autonomous intelligence: A comprehensive overview","author":"G\u00fclen K.","year":"2023","unstructured":"K. G\u00fclen. 2023. All about autonomous intelligence: A comprehensive overview. Dataconomy. Retrieved June 1st, 2024 from https:\/\/dataconomy.com\/2023\/03\/09\/what-is-autonomous-artificial-intelligence","journal-title":"Dataconomy"},{"key":"e_1_3_1_14_1","doi-asserted-by":"publisher","DOI":"10.1080\/10919392.2015.1125187"},{"key":"e_1_3_1_15_1","doi-asserted-by":"publisher","DOI":"10.4337\/9781800884892.00021"},{"key":"e_1_3_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-90434-0_15-1"},{"key":"e_1_3_1_17_1","doi-asserted-by":"publisher","DOI":"10.1111\/capa.12571"},{"key":"e_1_3_1_18_1","doi-asserted-by":"publisher","DOI":"10.3390\/info13110543"},{"key":"e_1_3_1_19_1","doi-asserted-by":"publisher","DOI":"10.1002\/ev.20561"},{"key":"e_1_3_1_20_1","unstructured":"Ministry of Finance - Finland. 2019. AuroraAI \u2013 towards a human-centric society. Development and Implementation Plan 2019\u20132023 Based on the Preliminary Study on the Aurora National Artificial Intelligence Programme. Retrieved June 1st 2024 from https:\/\/vm.fi\/documents\/10623\/cevelopment+and+implementation+plan+2019-2023.pdf"},{"key":"e_1_3_1_21_1","unstructured":"MIT Sloan Teaching & Learning Technologies. 2023. How to Use ChatGPT's Advanced Data Analysis Feature. Retrieved June 1st 2024 from https:\/\/mitsloanedtech.mit.edu\/ai\/tools\/data-analysis\/how-to-use-chatgpts-advanced-data-analysis-feature\/"},{"key":"e_1_3_1_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/ev.20558"},{"key":"e_1_3_1_23_1","doi-asserted-by":"publisher","DOI":"10.4337\/9781800884892.00008"},{"key":"e_1_3_1_24_1","doi-asserted-by":"publisher","DOI":"10.1177\/0952076718780537"},{"key":"e_1_3_1_25_1","article-title":"Sizing the prize: What's the real value of AI for your business and how can you capitalise?","author":"Rao A.","year":"2017","unstructured":"A. Rao and G. Verweij. 2017. Sizing the prize: What's the real value of AI for your business and how can you capitalise? PwC. Retrieved June 1st, 2024 from https:\/\/www.pwc.com\/gx\/en\/issues\/data-and-analytics\/publications\/artificial-intelligence-study.html","journal-title":"PwC"},{"key":"e_1_3_1_26_1","doi-asserted-by":"publisher","DOI":"10.1002\/ev.20562"},{"key":"e_1_3_1_27_1","unstructured":"M. Schemmer A. Bartos P. Spitzer P. Hemmer N. K\u00fchl J. Liebschner and G. Satzger. 2023. Towards effective human-AI decision-making: The role of human learning in appropriate reliance on AI advice. arXiv:2310.02108. Retrieved from https:\/\/arxiv.org\/abs\/2310.02108"},{"key":"e_1_3_1_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2020.06.026"},{"key":"e_1_3_1_29_1","first-page":"B5","article-title":"Thanks to generative AI, profits are up and costs are down at these businesses","volume":"20","author":"Silcoff S.","year":"2023","unstructured":"S. Silcoff and J. Castaldo. 2023. Thanks to generative AI, profits are up and costs are down at these businesses. The Globe and Mail, 20: B5\u20136.","journal-title":"The Globe and Mail"},{"key":"e_1_3_1_30_1","doi-asserted-by":"publisher","DOI":"10.3390\/ijerph19063641"},{"key":"e_1_3_1_31_1","unstructured":"United States Agency for International Development (USAID). 2022. Managing Machine Learning Projects in International Development: A Practical Guide. Retrieved from https:\/\/www.usaid.gov\/sites\/default\/files\/2022-05\/Vital_Wave_USAID-AIML-FieldGuide_FINAL_VERSION_1.pdf"},{"key":"e_1_3_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.giq.2020.101509"},{"key":"e_1_3_1_33_1","doi-asserted-by":"publisher","DOI":"10.1080\/01900692.2018.1498103"},{"key":"e_1_3_1_34_1","doi-asserted-by":"publisher","DOI":"10.1093\/PPMGOV\/GVAB006"},{"key":"e_1_3_1_35_1","article-title":"Unlocking real-time evidence for practitioners. How Evaluation and Data Analytics Are Generating On-Demand, Actionable Evidence for Front-Line Practitioners at First Place for Youth and Gemma Services","author":"York P.","year":"2021","unstructured":"P. York. 2021. Unlocking real-time evidence for practitioners. How Evaluation and Data Analytics Are Generating On-Demand, Actionable Evidence for Front-Line Practitioners at First Place for Youth and Gemma Services. Project Evident. Retrieved June 1st, 2024 from https:\/\/projectevident.org\/case-study\/generating-on-demand-actionable-evidence-first-place-for-youth-and-gemma-services","journal-title":"Project Evident"},{"key":"e_1_3_1_36_1","doi-asserted-by":"publisher","DOI":"10.4324\/9781003512493-11"}],"container-title":["Digital Government: Research and Practice"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696009","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3696009","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:04:30Z","timestamp":1750291470000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696009"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,12]]},"references-count":35,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,3,31]]}},"alternative-id":["10.1145\/3696009"],"URL":"https:\/\/doi.org\/10.1145\/3696009","relation":{},"ISSN":["2691-199X","2639-0175"],"issn-type":[{"value":"2691-199X","type":"print"},{"value":"2639-0175","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,12]]},"assertion":[{"value":"2023-12-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}