{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:13:45Z","timestamp":1750220025907,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":37,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,9,5]],"date-time":"2022-09-05T00:00:00Z","timestamp":1662336000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NSERC","award":["RGPIN-2021-03500"],"award-info":[{"award-number":["RGPIN-2021-03500"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,9,5]]},"DOI":"10.1145\/3555858.3555880","type":"proceedings-article","created":{"date-parts":[[2022,11,4]],"date-time":"2022-11-04T15:48:39Z","timestamp":1667576919000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Charting the Uncharted with GUR: How AI Playtesting Can Supplement Expert Evaluation"],"prefix":"10.1145","author":[{"given":"Atiya","family":"Nova","sequence":"first","affiliation":[{"name":"Ontario Tech University, Canada"}]},{"given":"Stevie","family":"Sansalone","sequence":"additional","affiliation":[{"name":"Ontario Tech University, Canada"}]},{"given":"Raquel","family":"Robinson","sequence":"additional","affiliation":[{"name":"Ontario Tech University, Canada"}]},{"given":"Pejman","family":"Mirza-Babaei","sequence":"additional","affiliation":[{"name":"Ontario Tech University, Canada"}]}],"member":"320","published-online":{"date-parts":[[2022,11,4]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Playtesting: What is Beyond Personas. ArXiv abs\/2107.11965(2021).","author":"Ariyurek Sinan","year":"2021","unstructured":"Sinan Ariyurek , Elif S\u00fcrer , and Aysu\u00a0Betin Can . 2021 . Playtesting: What is Beyond Personas. ArXiv abs\/2107.11965(2021). Sinan Ariyurek, Elif S\u00fcrer, and Aysu\u00a0Betin Can. 2021. Playtesting: What is Beyond Personas. ArXiv abs\/2107.11965(2021)."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1162\/desi.2008.24.3.85"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2017.08.141"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2967934.2968103"},{"key":"e_1_3_2_1_5_1","volume-title":"Understanding Inspection Methods: Lessons from an Assessment of Heuristic Evaluation. (01","author":"Cockton Gilbert","year":"2001","unstructured":"Gilbert Cockton and Alan Woolrych . 2001. Understanding Inspection Methods: Lessons from an Assessment of Heuristic Evaluation. (01 2001 ). https:\/\/doi.org\/10.1007\/978-1-4471-0353-0_11 10.1007\/978-1-4471-0353-0_11 Gilbert Cockton and Alan Woolrych. 2001. Understanding Inspection Methods: Lessons from an Assessment of Heuristic Evaluation. (01 2001). https:\/\/doi.org\/10.1007\/978-1-4471-0353-0_11"},{"key":"e_1_3_2_1_6_1","unstructured":"Dan Curtis. 2018. Blighttown - Dark Souls Wiki Guide. https:\/\/www.ign.com\/wikis\/dark-souls\/Blighttown  Dan Curtis. 2018. Blighttown - Dark Souls Wiki Guide. https:\/\/www.ign.com\/wikis\/dark-souls\/Blighttown"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Fernando de Mesentier\u00a0Silva Igor Borovikov John\u00a0F. Kolen Navid Aghdaie and Kazi\u00a0A. Zaman. 2018. Exploring Gameplay With AI Agents. In AIIDE.  Fernando de Mesentier\u00a0Silva Igor Borovikov John\u00a0F. Kolen Navid Aghdaie and Kazi\u00a0A. Zaman. 2018. Exploring Gameplay With AI Agents. In AIIDE.","DOI":"10.1609\/aiide.v14i1.13034"},{"key":"e_1_3_2_1_8_1","volume-title":"Evaluation system for game playability using emotion sensor based on ai. Sensors and Materials 33, 9","author":"Deng Shuh-Yeuan","year":"2021","unstructured":"Shuh-Yeuan Deng and Kuo-Kuang Fan . 2021. Evaluation system for game playability using emotion sensor based on ai. Sensors and Materials 33, 9 ( 2021 ). https:\/\/doi.org\/10.18494\/sam.2021.3479 10.18494\/sam.2021.3479 Shuh-Yeuan Deng and Kuo-Kuang Fan. 2021. Evaluation system for game playability using emotion sensor based on ai. Sensors and Materials 33, 9 (2021). https:\/\/doi.org\/10.18494\/sam.2021.3479"},{"volume-title":"Online Communities and Social Computing, A.\u00a0Ant Ozok and Panayiotis Zaphiris (Eds.)","author":"Desurvire Heather","key":"e_1_3_2_1_9_1","unstructured":"Heather Desurvire and Charlotte Wiberg . 2009. Game Usability Heuristics (PLAY) for Evaluating and Designing Better Games: The Next Iteration . In Online Communities and Social Computing, A.\u00a0Ant Ozok and Panayiotis Zaphiris (Eds.) . Springer Berlin Heidelberg , Berlin, Heidelberg , 557\u2013566. Heather Desurvire and Charlotte Wiberg. 2009. Game Usability Heuristics (PLAY) for Evaluating and Designing Better Games: The Next Iteration. In Online Communities and Social Computing, A.\u00a0Ant Ozok and Panayiotis Zaphiris (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 557\u2013566."},{"key":"#cr-split#-e_1_3_2_1_10_1.1","doi-asserted-by":"crossref","unstructured":"Heather Desurvire and Charlotte Wiberg. 2010. User Experience Design for Inexperienced Gamers: GAP - Game Approachability Principles. 131-147. https:\/\/doi.org\/10.1007\/978-1-84882-963-3_8 10.1007\/978-1-84882-963-3_8","DOI":"10.1007\/978-1-84882-963-3_8"},{"key":"#cr-split#-e_1_3_2_1_10_1.2","doi-asserted-by":"crossref","unstructured":"Heather Desurvire and Charlotte Wiberg. 2010. User Experience Design for Inexperienced Gamers: GAP - Game Approachability Principles. 131-147. https:\/\/doi.org\/10.1007\/978-1-84882-963-3_8","DOI":"10.1007\/978-1-84882-963-3_8"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CIG.2009.5286500"},{"volume-title":"Games User Research","author":"Drachen Anders","key":"e_1_3_2_1_12_1","unstructured":"Anders Drachen , Pejman Mirza-Babaei , and Lennart Nacke . 2018. Games User Research . Oxford University Press, Inc. , USA. Anders Drachen, Pejman Mirza-Babaei, and Lennart Nacke. 2018. Games User Research. Oxford University Press, Inc., USA."},{"key":"e_1_3_2_1_13_1","unstructured":"GamingRevenant. 2018. Far cry 1: Walkthrough - volcano [level 20] (realistic mode) 4K UHD - 60fps max settings. https:\/\/www.youtube.com\/watch?v=1fD7xcp7KuU&ab_channel=GamingRevenant  GamingRevenant. 2018. Far cry 1: Walkthrough - volcano [level 20] (realistic mode) 4K UHD - 60fps max settings. https:\/\/www.youtube.com\/watch?v=1fD7xcp7KuU&ab_channel=GamingRevenant"},{"key":"e_1_3_2_1_14_1","volume-title":"Automated playtesting in collectible card games using evolutionary algorithms: A case study in hearthstone. Knowledge-Based Systems 153 (Aug","author":"Garcia-Sanchez Pablo","year":"2018","unstructured":"Pablo Garcia-Sanchez , Alberto TONDA , Antonio Mora , Giovanni Squillero , and Juan\u00a0Julian Merelo . 2018. Automated playtesting in collectible card games using evolutionary algorithms: A case study in hearthstone. Knowledge-Based Systems 153 (Aug . 2018 ), 133\u2013146. https:\/\/doi.org\/10.1016\/j.knosys.2018.04.030 10.1016\/j.knosys.2018.04.030 Pablo Garcia-Sanchez, Alberto TONDA, Antonio Mora, Giovanni Squillero, and Juan\u00a0Julian Merelo. 2018. Automated playtesting in collectible card games using evolutionary algorithms: A case study in hearthstone. Knowledge-Based Systems 153 (Aug. 2018), 133\u2013146. https:\/\/doi.org\/10.1016\/j.knosys.2018.04.030"},{"key":"e_1_3_2_1_15_1","volume-title":"Video-game industry revenues grew so much during the pandemic that they reportedly exceeded sports and film combined. Business Insider","author":"Gilbert Ben","year":"2020","unstructured":"Ben Gilbert . 2020. Video-game industry revenues grew so much during the pandemic that they reportedly exceeded sports and film combined. Business Insider ( 2020 ). https:\/\/www.businessinsider.com\/video-game-industry-revenues-exceed-sports-and-film-combined-idc-2020-12 Ben Gilbert. 2020. Video-game industry revenues grew so much during the pandemic that they reportedly exceeded sports and film combined. Business Insider (2020). https:\/\/www.businessinsider.com\/video-game-industry-revenues-exceed-sports-and-film-combined-idc-2020-12"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1155\/2012\/625476"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.intcom.2006.10.001"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/3060832.3060973"},{"key":"e_1_3_2_1_19_1","unstructured":"David\u00a0E. Kieras. 2003. GOMS Models for Task Analysis.  David\u00a0E. Kieras. 2003. GOMS Models for Task Analysis."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2994310.2994364"},{"volume-title":"Usability Inspection Methods","author":"Nielsen Jakob","key":"e_1_3_2_1_21_1","unstructured":"Jakob Nielsen and Robert\u00a0 L. Mack . 1994. Usability Inspection Methods . Wiley , New York . Jakob Nielsen and Robert\u00a0L. Mack. 1994. Usability Inspection Methods. Wiley, New York."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/97243.97281"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450337.3483495"},{"key":"e_1_3_2_1_24_1","unstructured":"Jeffrey Parkin. 2018. Dark souls remastered guide: Blighttown Map. https:\/\/www.polygon.com\/dark-souls-remastered-guide\/2018\/7\/2\/17478910\/blighttown-map-items-npc  Jeffrey Parkin. 2018. Dark souls remastered guide: Blighttown Map. https:\/\/www.polygon.com\/dark-souls-remastered-guide\/2018\/7\/2\/17478910\/blighttown-map-items-npc"},{"key":"e_1_3_2_1_25_1","unstructured":"Jack Pooley. 2017. 13 terrible levels in otherwise awesome video games. https:\/\/whatculture.com\/gaming\/13-terrible-levels-in-otherwise-awesome-video-games  Jack Pooley. 2017. 13 terrible levels in otherwise awesome video games. https:\/\/whatculture.com\/gaming\/13-terrible-levels-in-otherwise-awesome-video-games"},{"key":"e_1_3_2_1_26_1","unstructured":"Jack Pooley. 2021. 10 convoluted video game levels everyone got lost in. https:\/\/whatculture.com\/gaming\/10-convoluted-video-game-levels-everyone-got-lost-in?page=9  Jack Pooley. 2021. 10 convoluted video game levels everyone got lost in. https:\/\/whatculture.com\/gaming\/10-convoluted-video-game-levels-everyone-got-lost-in?page=9"},{"key":"e_1_3_2_1_27_1","unstructured":"Mikko Rajanen and Dorina Rajanen. 2018. Heuristic evaluation in game and gamification development. In GamiFIN.  Mikko Rajanen and Dorina Rajanen. 2018. Heuristic evaluation in game and gamification development. In GamiFIN."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3474658"},{"key":"e_1_3_2_1_29_1","volume-title":"Ropossum: An Authoring Tool for Designing, Optimizing and Solving Cut the Rope Levels. In AIIDE.","author":"Shaker Noor","year":"2013","unstructured":"Noor Shaker , Mohammad\u00a0Hossein Shaker , and Julian Togelius . 2013 . Ropossum: An Authoring Tool for Designing, Optimizing and Solving Cut the Rope Levels. In AIIDE. Noor Shaker, Mohammad\u00a0Hossein Shaker, and Julian Togelius. 2013. Ropossum: An Authoring Tool for Designing, Optimizing and Solving Cut the Rope Levels. In AIIDE."},{"key":"e_1_3_2_1_30_1","volume-title":"Artificial Playfulness: A Tool for Automated Agent-Based Playtesting. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA \u201919)","author":"Nova Atiya","year":"2019","unstructured":"Samantha\u00a0. Stahlke, Atiya Nova , and Pejman Mirza-Babaei . 2019 . Artificial Playfulness: A Tool for Automated Agent-Based Playtesting. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA \u201919) . Association for Computing Machinery, New York, NY, USA, 1\u20136. https:\/\/doi.org\/10.1145\/3290607.3313039 10.1145\/3290607.3313039 Samantha\u00a0. Stahlke, Atiya Nova, and Pejman Mirza-Babaei. 2019. Artificial Playfulness: A Tool for Automated Agent-Based Playtesting. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA \u201919). Association for Computing Machinery, New York, NY, USA, 1\u20136. https:\/\/doi.org\/10.1145\/3290607.3313039"},{"volume-title":"Artificial Players in the Design Process: Developing an Automated Testing Tool for Game Level and World Design","author":"Stahlke Samantha","key":"e_1_3_2_1_31_1","unstructured":"Samantha Stahlke , Atiya Nova , and Pejman Mirza-Babaei . 2020. Artificial Players in the Design Process: Developing an Automated Testing Tool for Game Level and World Design . Association for Computing Machinery , New York, NY, USA , 267\u2013280. https:\/\/doi.org\/10.1145\/3410404.3414249 10.1145\/3410404.3414249 Samantha Stahlke, Atiya Nova, and Pejman Mirza-Babaei. 2020. Artificial Players in the Design Process: Developing an Automated Testing Tool for Game Level and World Design. Association for Computing Machinery, New York, NY, USA, 267\u2013280. https:\/\/doi.org\/10.1145\/3410404.3414249"},{"key":"e_1_3_2_1_32_1","unstructured":"Ryan Taljonick and Connor Sheridan. 2017. Remember these 11 frustrating video game levels and try not to smash your controller. https:\/\/www.gamesradar.com\/frustrating-levels-nearly-made-us-break-our-controllers\/  Ryan Taljonick and Connor Sheridan. 2017. Remember these 11 frustrating video game levels and try not to smash your controller. https:\/\/www.gamesradar.com\/frustrating-levels-nearly-made-us-break-our-controllers\/"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Wei-Siong Tan Dahai Liu and R. Bishu. 2009. Web evaluation: heuristic evaluation vs. user testing.International Journal of Industrial Ergonomics 39 (2009) 621\u2013627.  Wei-Siong Tan Dahai Liu and R. Bishu. 2009. Web evaluation: heuristic evaluation vs. user testing.International Journal of Industrial Ergonomics 39 (2009) 621\u2013627.","DOI":"10.1016\/j.ergon.2008.02.012"},{"key":"e_1_3_2_1_34_1","volume-title":"Cognitive Architectures and Autonomy: A Comparative Review. Journal of Artificial General Intelligence 3 (01","author":"Th\u00f3risson Kristinn","year":"2012","unstructured":"Kristinn Th\u00f3risson and Helgi Helgason . 2012. Cognitive Architectures and Autonomy: A Comparative Review. Journal of Artificial General Intelligence 3 (01 2012 ), 1\u201330. https:\/\/doi.org\/10.2478\/v10229-011-0015-3 10.2478\/v10229-011-0015-3 Kristinn Th\u00f3risson and Helgi Helgason. 2012. Cognitive Architectures and Autonomy: A Comparative Review. Journal of Artificial General Intelligence 3 (01 2012), 1\u201330. https:\/\/doi.org\/10.2478\/v10229-011-0015-3"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.entcom.2013.02.002"},{"key":"e_1_3_2_1_36_1","volume-title":"CHI \u201911 Extended Abstracts on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI EA \u201911)","author":"White R.","year":"1979","unstructured":"Gareth\u00a0 R. White , Pejman Mirza-babaei, Graham McAllister , and Judith Good . 2011. Weak Inter-Rater Reliability in Heuristic Evaluation of Video Games . In CHI \u201911 Extended Abstracts on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI EA \u201911) . Association for Computing Machinery , New York, NY, USA , 1441\u20131446. https:\/\/doi.org\/10.1145\/ 1979 742.1979788 10.1145\/1979742.1979788 Gareth\u00a0R. White, Pejman Mirza-babaei, Graham McAllister, and Judith Good. 2011. Weak Inter-Rater Reliability in Heuristic Evaluation of Video Games. In CHI \u201911 Extended Abstracts on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI EA \u201911). Association for Computing Machinery, New York, NY, USA, 1441\u20131446. https:\/\/doi.org\/10.1145\/1979742.1979788"}],"event":{"name":"FDG22: 17th International Conference on the Foundations of Digital Games","acronym":"FDG22","location":"Athens Greece"},"container-title":["Proceedings of the 17th International Conference on the Foundations of Digital Games"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555858.3555880","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3555858.3555880","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:51:37Z","timestamp":1750182697000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555858.3555880"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,5]]},"references-count":37,"alternative-id":["10.1145\/3555858.3555880","10.1145\/3555858"],"URL":"https:\/\/doi.org\/10.1145\/3555858.3555880","relation":{},"subject":[],"published":{"date-parts":[[2022,9,5]]},"assertion":[{"value":"2022-11-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}