{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T00:50:01Z","timestamp":1774399801900,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":42,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T00:00:00Z","timestamp":1603065600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Australian Research Council Project","award":["DP180102687"],"award-info":[{"award-number":["DP180102687"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,10,19]]},"DOI":"10.1145\/3340531.3412043","type":"proceedings-article","created":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T05:31:03Z","timestamp":1603085463000},"page":"85-94","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Do People and Neural Nets Pay Attention to the Same Words"],"prefix":"10.1145","author":[{"given":"Valeria","family":"Bolotova","sequence":"first","affiliation":[{"name":"RMIT University, Melbourne, VIC, Australia"}]},{"given":"Vladislav","family":"Blinov","sequence":"additional","affiliation":[{"name":"Ural Federal University, Melbourne, VIC, Australia"}]},{"given":"Yukun","family":"Zheng","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}]},{"given":"W. Bruce","family":"Croft","sequence":"additional","affiliation":[{"name":"University of Massachusetts Amherst, Amherst, MA, USA"}]},{"given":"Falk","family":"Scholer","sequence":"additional","affiliation":[{"name":"RMIT University, Melbourne, VIC, Australia"}]},{"given":"Mark","family":"Sanderson","sequence":"additional","affiliation":[{"name":"RMIT University, Melbourne, VIC, Australia"}]}],"member":"320","published-online":{"date-parts":[[2020,10,19]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli.07-034-R2"},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1080\/02724980443000476"},{"key":"e_1_3_2_2_3_1","volume-title":"In Proc. of IIiX. ACM, 195--204","author":"Bando Lorena Leal","year":"2010","unstructured":"Lorena Leal Bando , Falk Scholer , and Andrew Turpin . 2010 . Constructing querybiased summaries: a comparison of human and system generated snippets . In In Proc. of IIiX. ACM, 195--204 . Lorena Leal Bando, Falk Scholer, and Andrew Turpin. 2010. Constructing querybiased summaries: a comparison of human and system generated snippets. In In Proc. of IIiX. ACM, 195--204."},{"key":"e_1_3_2_2_4_1","volume-title":"A taxonomy of web search. ACM Sigir forum 36, 2","author":"Broder Andrei","year":"2002","unstructured":"Andrei Broder . 2002. A taxonomy of web search. ACM Sigir forum 36, 2 ( 2002 ), 3--10. Andrei Broder. 2002. A taxonomy of web search. ACM Sigir forum 36, 2 (2002), 3--10."},{"key":"e_1_3_2_2_5_1","volume-title":"Chilton and Jaime Teevan","author":"Lydia","year":"2011","unstructured":"Lydia B. Chilton and Jaime Teevan . 2011 . Addressing People's Information Needs Directly in a Web Search Result Page. In In Proc. of WWW. 27--36. Lydia B. Chilton and Jaime Teevan. 2011. Addressing People's Information Needs Directly in a Web Search Result Page. In In Proc. of WWW. 27--36."},{"key":"e_1_3_2_2_6_1","volume-title":"Manning","author":"Clark Kevin","year":"2019","unstructured":"Kevin Clark , Urvashi Khandelwal , Omer Levy , and Christopher D . Manning . 2019 . What Does BERT Look at? An Analysis of BERT?s Attention. In Proc. ACL. 276--286. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What Does BERT Look at? An Analysis of BERT?s Attention. In Proc. ACL. 276--286."},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277767"},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2970398.2970438"},{"key":"e_1_3_2_2_9_1","volume-title":"Proc. SIGCHI. 407--416","author":"Cutrell Edward","year":"2007","unstructured":"Edward Cutrell and Zhiwei Guan . 2007 . What are you looking for?: an eyetracking study of information usage in web search . In Proc. SIGCHI. 407--416 . Edward Cutrell and Zhiwei Guan. 2007. What are you looking for?: an eyetracking study of information usage in web search. In Proc. SIGCHI. 407--416."},{"key":"e_1_3_2_2_10_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In In Proc. of NAACL. 4171--4186","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In In Proc. of NAACL. 4171--4186 . Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In In Proc. of NAACL. 4171--4186."},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1008992.1009079"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2637002.2637011"},{"key":"e_1_3_2_2_13_1","first-page":"1","article-title":"Online evaluation for information retrieval","volume":"10","author":"Hofmann Katja","year":"2016","unstructured":"Katja Hofmann , Lihong Li , Filip Radlinski , 2016 . Online evaluation for information retrieval . FnTIR 10 , 1 (2016), 1 -- 117 . Katja Hofmann, Lihong Li, Filip Radlinski, et al. 2016. Online evaluation for information retrieval. FnTIR 10, 1 (2016), 1--117.","journal-title":"FnTIR"},{"key":"e_1_3_2_2_14_1","volume-title":"Evaluating the impact of snippet highlighting in search. UIIR-2009","author":"Iofciu Tereza","year":"2009","unstructured":"Tereza Iofciu , Nick Craswell , and Milad Shokouhi . 2009. Evaluating the impact of snippet highlighting in search. UIIR-2009 ( 2009 ), 44. Tereza Iofciu, Nick Craswell, and Milad Shokouhi. 2009. Evaluating the impact of snippet highlighting in search. UIIR-2009 (2009), 44."},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1070"},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2007.289"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03193604"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2682862.2682877"},{"key":"e_1_3_2_2_19_1","volume-title":"Revealing the Dark Secrets of BERT. In In Proc. of EMNLP-IJCNLP. 4365--4374","author":"Kovaleva Olga","year":"2019","unstructured":"Olga Kovaleva , Alexey Romanov , Anna Rogers , and Anna Rumshisky . 2019 . Revealing the Dark Secrets of BERT. In In Proc. of EMNLP-IJCNLP. 4365--4374 . Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. Revealing the Dark Secrets of BERT. In In Proc. of EMNLP-IJCNLP. 4365--4374."},{"key":"e_1_3_2_2_20_1","volume-title":"Towards Better Measurement of Attention and Satisfaction in Mobile Search. In In Proc. of SIGIR. 113--122","author":"Lagun Dmitry","year":"2014","unstructured":"Dmitry Lagun , Chih-Hung Hsieh , Dale Webster , and Vidhya Navalpakkam . 2014 . Towards Better Measurement of Attention and Satisfaction in Mobile Search. In In Proc. of SIGIR. 113--122 . Dmitry Lagun, Chih-Hung Hsieh, Dale Webster, and Vidhya Navalpakkam. 2014. Towards Better Measurement of Attention and Satisfaction in Mobile Search. In In Proc. of SIGIR. 113--122."},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.20794"},{"key":"e_1_3_2_2_22_1","volume-title":"A Study of BERT for Non-Factoid Question-Answering under Passage Length Constraints. ArXiv abs\/1908.06780","author":"Mass Yosi","year":"2019","unstructured":"Yosi Mass , Haggai Roitman , Shai Erera , Or Rivlin , Bar Weiner , and David Konopnicki . 2019. A Study of BERT for Non-Factoid Question-Answering under Passage Length Constraints. ArXiv abs\/1908.06780 ( 2019 ). Yosi Mass, Haggai Roitman, Shai Erera, Or Rivlin, Bar Weiner, and David Konopnicki. 2019. A Study of BERT for Non-Factoid Question-Answering under Passage Length Constraints. ArXiv abs\/1908.06780 (2019)."},{"key":"e_1_3_2_2_23_1","volume-title":"Proc. NeurIPS. 14014--14024","author":"Michel Paul","year":"2019","unstructured":"Paul Michel , Omer Levy , and Graham Neubig . 2019 . Are Sixteen Heads Really Better than One? . In Proc. NeurIPS. 14014--14024 . Paul Michel, Omer Levy, and Graham Neubig. 2019. Are Sixteen Heads Really Better than One?. In Proc. NeurIPS. 14014--14024."},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767768"},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3295750.3298946"},{"key":"e_1_3_2_2_26_1","volume-title":"The effect of word frequency, word predictability, and font difficulty on the eye movements of young and older readers. Psychology and aging 21, 3","author":"Rayner Keith","year":"2006","unstructured":"Keith Rayner , Erik D Reichle , Michael J Stroud , Carrick C Williams , and Alexander Pollatsek . 2006. The effect of word frequency, word predictability, and font difficulty on the eye movements of young and older readers. Psychology and aging 21, 3 ( 2006 ), 448. Keith Rayner, Erik D Reichle, Michael J Stroud, Carrick C Williams, and Alexander Pollatsek. 2006. The effect of word frequency, word predictability, and font difficulty on the eye movements of young and older readers. Psychology and aging 21, 3 (2006), 448."},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23708-9_4"},{"key":"e_1_3_2_2_28_1","volume-title":"Neural Machine Translation of Rare Words with Subword Units. In In Proc. of ACL. 1715--1725","author":"Sennrich Rico","year":"2016","unstructured":"Rico Sennrich , Barry Haddow , and Alexandra Birch . 2016 . Neural Machine Translation of Rare Words with Subword Units. In In Proc. of ACL. 1715--1725 . Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural Machine Translation of Rare Words with Subword Units. In In Proc. of ACL. 1715--1725."},{"key":"e_1_3_2_2_29_1","volume-title":"Robertson","author":"Jones K Sp\u00e4rck","year":"2000","unstructured":"K Sp\u00e4rck Jones , Steve Walker , and Stephen E . Robertson . 2000 . A probabilistic model of information retrieval: development and comparative experiments: Part 2. Information processing & management 36, 6 (2000), 809--840. K Sp\u00e4rck Jones, Steve Walker, and Stephen E. Robertson. 2000. A probabilistic model of information retrieval: development and comparative experiments: Part 2. Information processing & management 36, 6 (2000), 809--840."},{"key":"e_1_3_2_2_30_1","volume-title":"In Proc. of ACL-08: HLT. 719--727","author":"Surdeanu Mihai","year":"2008","unstructured":"Mihai Surdeanu , Massimiliano Ciaramita , and Hugo Zaragoza . 2008 . Learning to rank answers on large online QA collections . In In Proc. of ACL-08: HLT. 719--727 . Mihai Surdeanu, Massimiliano Ciaramita, and Hugo Zaragoza. 2008. Learning to rank answers on large online QA collections. In In Proc. of ACL-08: HLT. 719--727."},{"key":"e_1_3_2_2_31_1","volume-title":"An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation. In In Proc. of WMT","author":"Tang Gongbo","year":"2018","unstructured":"Gongbo Tang , Rico Sennrich , and Joakim Nivre . 2018 . An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation. In In Proc. of WMT 2018. 26--35. Gongbo Tang, Rico Sennrich, and Joakim Nivre. 2018. An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation. In In Proc. of WMT 2018. 26--35."},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.290947"},{"key":"e_1_3_2_2_33_1","volume-title":"Proc. NIPS.","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , Llion Jones , Aidan N. Gomez , ?ukasz Kaiser, and Illia Polosukhin . 2017 . Attention is All You Need . In Proc. NIPS. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, ?ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proc. NIPS."},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-3007"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.5121\/mlaij.2016.3103"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1580"},{"key":"e_1_3_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23782"},{"key":"e_1_3_2_2_38_1","volume-title":"Proc. SIGIR. 889--892","author":"Williams Kyle","year":"2016","unstructured":"Kyle Williams , Julia Kiseleva , Aidan C. Crook , Imed Zitouni , Ahmed Hassan Awadallah , and Madian Khabsa . 2016 . Is This Your Final Answer? Evaluating the Effect of Answers on Good Abandonment in Mobile Search . In Proc. SIGIR. 889--892 . Kyle Williams, Julia Kiseleva, Aidan C. Crook, Imed Zitouni, Ahmed Hassan Awadallah, and Madian Khabsa. 2016. Is This Your Final Answer? Evaluating the Effect of Answers on Good Abandonment in Mobile Search. In Proc. SIGIR. 889--892."},{"key":"e_1_3_2_2_39_1","doi-asserted-by":"crossref","unstructured":"Gary S Wolverton and David Zola. 1983. The temporal characteristics of visual information extraction during reading. In Eye movements in reading. 41--51.  Gary S Wolverton and David Zola. 1983. The temporal characteristics of visual information extraction during reading. In Eye movements in reading. 41--51.","DOI":"10.1016\/B978-0-12-583680-7.50008-4"},{"key":"e_1_3_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-30671-1_9"},{"key":"e_1_3_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2505515.2505720"},{"key":"e_1_3_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331231"}],"event":{"name":"CIKM '20: The 29th ACM International Conference on Information and Knowledge Management","location":"Virtual Event Ireland","acronym":"CIKM '20","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 29th ACM International Conference on Information & Knowledge Management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3340531.3412043","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3340531.3412043","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:29Z","timestamp":1750197749000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3340531.3412043"}},"subtitle":["Studying Eye-tracking Data for Non-factoid QA Evaluation"],"short-title":[],"issued":{"date-parts":[[2020,10,19]]},"references-count":42,"alternative-id":["10.1145\/3340531.3412043","10.1145\/3340531"],"URL":"https:\/\/doi.org\/10.1145\/3340531.3412043","relation":{},"subject":[],"published":{"date-parts":[[2020,10,19]]},"assertion":[{"value":"2020-10-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}