{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,24]],"date-time":"2025-08-24T01:17:42Z","timestamp":1755998262337,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":13,"publisher":"ACM","license":[{"start":{"date-parts":[[2017,11,13]],"date-time":"2017-11-13T00:00:00Z","timestamp":1510531200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"RAEng\/Leverhulme Trust Senior Research Fellowship Scheme","award":["Hastie\/LTSRF1617\/13\/37"],"award-info":[{"award-number":["Hastie\/LTSRF1617\/13\/37"]}]},{"name":"EPSRC","award":["EP\/M005429\/1, EP\/N017536\/1"],"award-info":[{"award-number":["EP\/M005429\/1, EP\/N017536\/1"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2017,11,13]]},"DOI":"10.1145\/3139491.3139504","type":"proceedings-article","created":{"date-parts":[[2017,11,6]],"date-time":"2017-11-06T13:30:29Z","timestamp":1509975029000},"page":"25-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["A review of evaluation techniques for social dialogue systems"],"prefix":"10.1145","author":[{"given":"Amanda Cercas","family":"Curry","sequence":"first","affiliation":[{"name":"Heriot-Watt University, UK"}]},{"given":"Helen","family":"Hastie","sequence":"additional","affiliation":[{"name":"Heriot-Watt University, UK"}]},{"given":"Verena","family":"Rieser","sequence":"additional","affiliation":[{"name":"Heriot-Watt University, UK"}]}],"member":"320","published-online":{"date-parts":[[2017,11,13]]},"reference":[{"volume-title":"Data-Driven Methods for Adaptive Spoken Dialogue Systems","author":"Hastie Helen","key":"e_1_3_2_1_1_1","unstructured":"Helen Hastie . 2012. Metrics and evaluation of spoken dialogue systems . In Data-Driven Methods for Adaptive Spoken Dialogue Systems . Springer New York , 131\u2013150. Helen Hastie. 2012. Metrics and evaluation of spoken dialogue systems. In Data-Driven Methods for Adaptive Spoken Dialogue Systems. Springer New York, 131\u2013150."},{"key":"e_1_3_2_1_2_1","volume-title":"Adversarial Evaluation of Dialogue Models. CoRR abs\/1701.08198","author":"Kannan Anjuli","year":"2017","unstructured":"Anjuli Kannan and Oriol Vinyals . 2017. Adversarial Evaluation of Dialogue Models. CoRR abs\/1701.08198 ( 2017 ). http:\/\/arxiv.org\/abs\/1701.08198 Anjuli Kannan and Oriol Vinyals. 2017. Adversarial Evaluation of Dialogue Models. CoRR abs\/1701.08198 (2017). http:\/\/arxiv.org\/abs\/1701.08198"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1127"},{"key":"e_1_3_2_1_4_1","first-page":"74","article-title":"ROUGE: A package for automatic evaluation of summaries. In Text summarization branches out: Proceedings of the ACL-04 workshop","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin . 2004 . ROUGE: A package for automatic evaluation of summaries. In Text summarization branches out: Proceedings of the ACL-04 workshop . Barcelona, Spain , 74 \u2013 81 . http:\/\/aclweb.org\/anthology\/W04-1013 Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text summarization branches out: Proceedings of the ACL-04 workshop. Barcelona, Spain, 74\u201381. http:\/\/aclweb.org\/anthology\/W04-1013","journal-title":"Barcelona, Spain"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1230"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"crossref","unstructured":"Ryan Lowe Michael Noseworthy Iulian Serban Nicolas Angelard-Gontier Yoshua Bengio and Joelle Pineau. 2017. Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses. In (under review).  Ryan Lowe Michael Noseworthy Iulian Serban Nicolas Angelard-Gontier Yoshua Bengio and Joelle Pineau. 2017. Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses. In (under review).","DOI":"10.18653\/v1\/P17-1103"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-3634"},{"volume-title":"Spoken Dialogue Technology: Toward the Conversational User Interface","author":"McTear Mike","key":"e_1_3_2_1_8_1","unstructured":"Mike McTear . 2004. Spoken Dialogue Technology: Toward the Conversational User Interface . Springer , London . Mike McTear. 2004. Spoken Dialogue Technology: Toward the Conversational User Interface. Springer, London."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.3115\/1073083.1073135"},{"key":"e_1_3_2_1_10_1","series-title":"Book Series: Theory and Applications of Natural Language Processing","volume-title":"Reinforcement Learning for Adaptive Dialogue Systems: A Data-driven Methodology for Dialogue Management and Natural Language Generation","author":"Rieser Verena","unstructured":"Verena Rieser and Oliver Lemon . 2011. Reinforcement Learning for Adaptive Dialogue Systems: A Data-driven Methodology for Dialogue Management and Natural Language Generation . Book Series: Theory and Applications of Natural Language Processing , Springer , Berlin\/ Heidelberg . Verena Rieser and Oliver Lemon. 2011. Reinforcement Learning for Adaptive Dialogue Systems: A Data-driven Methodology for Dialogue Management and Natural Language Generation. Book Series: Theory and Applications of Natural Language Processing, Springer, Berlin\/Heidelberg."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-1020"},{"key":"e_1_3_2_1_12_1","volume-title":"A neural conversational model. arXiv preprint arXiv:1506.05869","author":"Vinyals Oriol","year":"2015","unstructured":"Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869 ( 2015 ). Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869 (2015)."},{"key":"e_1_3_2_1_13_1","volume-title":"Rudnicky","author":"Yu Zhou","year":"2016","unstructured":"Zhou Yu , Ziyu Xu , Alan W. Black , and Alexander I . Rudnicky . 2016 . Strategy and Policy Learning for Non-Task-Oriented Conversational Systems. In Proceedings of the SIGDIAL. Abstract 1 Introduction 2 Automatic Metrics 2.1 Word-Overlap Metrics 2.2 Machine Learning Methods for Dialogue Evaluation 2.3 Reward-based Metrics 3 Conclusion and Discussion References Zhou Yu, Ziyu Xu, Alan W. Black, and Alexander I. Rudnicky. 2016. Strategy and Policy Learning for Non-Task-Oriented Conversational Systems. In Proceedings of the SIGDIAL. Abstract 1 Introduction 2 Automatic Metrics 2.1 Word-Overlap Metrics 2.2 Machine Learning Methods for Dialogue Evaluation 2.3 Reward-based Metrics 3 Conclusion and Discussion References"}],"event":{"name":"ICMI '17: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"],"location":"Glasgow UK","acronym":"ICMI '17"},"container-title":["Proceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interactions with Artificial Agents"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3139491.3139504","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3139491.3139504","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:14:01Z","timestamp":1750212841000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3139491.3139504"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,11,13]]},"references-count":13,"alternative-id":["10.1145\/3139491.3139504","10.1145\/3139491"],"URL":"https:\/\/doi.org\/10.1145\/3139491.3139504","relation":{},"subject":[],"published":{"date-parts":[[2017,11,13]]},"assertion":[{"value":"2017-11-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}