{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T06:14:52Z","timestamp":1775283292367,"version":"3.50.1"},"reference-count":23,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2011,1,3]],"date-time":"2011-01-03T00:00:00Z","timestamp":1294012800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGIR Forum"],"published-print":{"date-parts":[[2011,1,3]]},"abstract":"<jats:p>All search in the real-world is inherently interactive. Information retrieval (IR) has a firm tradition of using simulation to evaluate IR systems as embodied by the Cranfield paradigm. However, to a large extent, such system evaluations ignore user interaction. Simulations provide a way to go beyond this limitation. With an increasing number of researchers using simulation to evaluate interactive IR systems, it is now timely to discuss, develop and advance this powerful methodology within the field of IR. During the SimInt 2010 workshop around 40 participants discussed and presented their views on the simulation of interaction. The main conclusion and general consensus was that simulation offers great potential for the field of IR; and that simulations of user interaction can make explicit the user and the user interface while maintaining the advantages of the Cranfield paradigm.<\/jats:p>","DOI":"10.1145\/1924475.1924484","type":"journal-article","created":{"date-parts":[[2011,1,24]],"date-time":"2011-01-24T14:58:13Z","timestamp":1295881093000},"page":"35-47","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":40,"title":["Report on the SIGIR 2010 workshop on the simulation of interaction"],"prefix":"10.1145","volume":"44","author":[{"given":"Leif","family":"Azzopardi","sequence":"first","affiliation":[{"name":"University of Glasgow, Scotland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kalervo","family":"J\u00e4rvelin","sequence":"additional","affiliation":[{"name":"University of Tampere, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jaap","family":"Kamps","sequence":"additional","affiliation":[{"name":"University of Amsterdam, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mark D.","family":"Smucker","sequence":"additional","affiliation":[{"name":"University of Waterloo, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2011,1,3]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"25","volume-title":"Azzopardi et al. {3}","author":"Alonso O.","unstructured":"O. Alonso and J. Pedersen . Recovering temporal context for relevance assessments . In Azzopardi et al. {3} , pages 25 -- 26 . O. Alonso and J. Pedersen. Recovering temporal context for relevance assessments. In Azzopardi et al. {3}, pages 25--26."},{"key":"e_1_2_1_2_1","first-page":"27","volume-title":"Azzopardi et al. {3}","author":"Arvola P.","unstructured":"P. Arvola and J. Kek\u00e4l\u00e4inen . Simulating user interaction in result document browsing . In Azzopardi et al. {3} , pages 27 -- 28 . P. Arvola and J. Kek\u00e4l\u00e4inen. Simulating user interaction in result document browsing. In Azzopardi et al. {3}, pages 27--28."},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the SIGIR 2010 Workshop on the Simulation of Interaction: Automated Evaluation of Interactive IR (SimInt 2010)","author":"Azzopardi L.","year":"2010","unstructured":"L. Azzopardi , K. J\u00e4rvelin , J. Kamps , and M. D. Smucker , editors . Proceedings of the SIGIR 2010 Workshop on the Simulation of Interaction: Automated Evaluation of Interactive IR (SimInt 2010) , 2010 . ACM Press. L. Azzopardi, K. J\u00e4rvelin, J. Kamps, and M. D. Smucker, editors. Proceedings of the SIGIR 2010 Workshop on the Simulation of Interaction: Automated Evaluation of Interactive IR (SimInt 2010), 2010. ACM Press."},{"key":"e_1_2_1_4_1","first-page":"19","volume-title":"Azzopardi et al. {3}","author":"Clough P.","unstructured":"P. Clough , J. Gonzalo , and J. Karlgren . Creating re-useable log files for interactive CLIR . In Azzopardi et al. {3} , pages 19 -- 20 . P. Clough, J. Gonzalo, and J. Karlgren. Creating re-useable log files for interactive CLIR. In Azzopardi et al. {3}, pages 19--20."},{"key":"e_1_2_1_5_1","first-page":"1","volume-title":"Azzopardi et al. {3}","author":"Cole M. J.","unstructured":"M. J. Cole . Simulation of the IIR user: Beyond the automagic . In Azzopardi et al. {3} , pages 1 -- 2 . M. J. Cole. Simulation of the IIR user: Beyond the automagic. In Azzopardi et al. {3}, pages 1--2."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/0020-0271(73)90004-1"},{"key":"e_1_2_1_7_1","first-page":"9","volume-title":"Azzopardi et al. {3}","author":"Geva S.","unstructured":"S. Geva and T. Chappell . Focused relevance feedback evaluation . In Azzopardi et al. {3} , pages 9 -- 10 . S. Geva and T. Chappell. Focused relevance feedback evaluation. In Azzopardi et al. {3}, pages 9--10."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/133160.133167"},{"key":"e_1_2_1_9_1","first-page":"21","volume-title":"Azzopardi et al. {3}","author":"Huurnink B.","unstructured":"B. Huurnink , K. Hofmann , and M. de Rijke . Simulating searches from transaction logs . In Azzopardi et al. {3} , pages 21 -- 22 . B. Huurnink, K. Hofmann, and M. de Rijke. Simulating searches from transaction logs. In Azzopardi et al. {3}, pages 21--22."},{"key":"e_1_2_1_10_1","first-page":"11","volume-title":"Azzopardi et al. {3}","author":"Jethani C.","unstructured":"C. Jethani and M. D. Smucker . Modeling the time to judge document relevance . In Azzopardi et al. {3} , pages 11 -- 12 . C. Jethani and M. D. Smucker. Modeling the time to judge document relevance. In Azzopardi et al. {3}, pages 11--12."},{"key":"e_1_2_1_11_1","first-page":"13","volume-title":"Session track at TREC","author":"Kanoulas E.","year":"2010","unstructured":"E. Kanoulas , P. Clough , B. Carterette , and M. Sanderson . Session track at TREC 2010 . In Azzopardi et al. {3}, pages 13 -- 14 . E. Kanoulas, P. Clough, B. Carterette, and M. Sanderson. Session track at TREC 2010. In Azzopardi et al. {3}, pages 13--14."},{"key":"e_1_2_1_12_1","first-page":"3","volume-title":"Azzopardi et al. {3}","author":"Kato T.","unstructured":"T. Kato , M. Matsushita , and N. Kando . Bridging evaluations: Inspiration from dialog system research . In Azzopardi et al. {3} , pages 3 -- 4 . T. Kato, M. Matsushita, and N. Kando. Bridging evaluations: Inspiration from dialog system research. In Azzopardi et al. {3}, pages 3--4."},{"key":"e_1_2_1_13_1","first-page":"29","volume-title":"Azzopardi et al. {3}","author":"Keskustalo H.","unstructured":"H. Keskustalo and K. J\u00e4rvelin . Query and browsing-based interaction simulation in test collections . In Azzopardi et al. {3} , pages 29 -- 30 . H. Keskustalo and K. J\u00e4rvelin. Query and browsing-based interaction simulation in test collections. In Azzopardi et al. {3}, pages 29--30."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-007-9043-7"},{"key":"e_1_2_1_15_1","first-page":"15","volume-title":"Azzopardi et al. {3}","author":"Keskustalo H.","unstructured":"H. Keskustalo , K. J\u00e4rvelin , and A. Pirkola . Graph-based query session exploration based on facet analysis . In Azzopardi et al. {3} , pages 15 -- 16 . H. Keskustalo, K. J\u00e4rvelin, and A. Pirkola. Graph-based query session exploration based on facet analysis. In Azzopardi et al. {3}, pages 15--16."},{"key":"e_1_2_1_16_1","first-page":"5","volume-title":"Azzopardi et al. {3}","author":"Mulwa C.","unstructured":"C. Mulwa , W. Li , S. Lawless , and G. Jones . A proposal for the evaluation of adaptive information retrieval systems using simulated interaction . In Azzopardi et al. {3} , pages 5 -- 6 . C. Mulwa, W. Li, S. Lawless, and G. Jones. A proposal for the evaluation of adaptive information retrieval systems using simulated interaction. In Azzopardi et al. {3}, pages 5--6."},{"key":"e_1_2_1_17_1","first-page":"23","volume-title":"Azzopardi et al. {3}","author":"Nanas N.","unstructured":"N. Nanas , U. Kruschwitz , M.-D. Albakour , M. Fasli , D. Song , Y. Kim , U. C. Beresi , and A. D. Roeck . A methodology for simulated experiments in interactive search . In Azzopardi et al. {3} , pages 23 -- 24 . N. Nanas, U. Kruschwitz, M.-D. Albakour, M. Fasli, D. Song, Y. Kim, U. C. Beresi, and A. D. Roeck. A methodology for simulated experiments in interactive search. In Azzopardi et al. {3}, pages 23--24."},{"key":"e_1_2_1_18_1","first-page":"31","volume-title":"Azzopardi et al. {3}","author":"Preminger M.","unstructured":"M. Preminger . Evaluating a visualization approach by user simulation . In Azzopardi et al. {3} , pages 31 -- 32 . M. Preminger. Evaluating a visualization approach by user simulation. In Azzopardi et al. {3}, pages 31--32."},{"key":"e_1_2_1_19_1","first-page":"33","volume-title":"Azzopardi et al. {3}","author":"Stober S.","unstructured":"S. Stober and A. Nuernberger . Automatic evaluation of user adaptive interfaces for information organization and exploration . In Azzopardi et al. {3} , pages 33 -- 34 . S. Stober and A. Nuernberger. Automatic evaluation of user adaptive interfaces for information organization and exploration. In Azzopardi et al. {3}, pages 33--34."},{"key":"e_1_2_1_20_1","first-page":"236","volume-title":"SIGIR 1980","author":"Tague J.","year":"1980","unstructured":"J. Tague , M. Nelson , and H. Wu . Problems in the simulation of bibliographic retrieval systems . In SIGIR 1980 , pages 236 -- 255 , 1980 . J. Tague, M. Nelson, and H. Wu. Problems in the simulation of bibliographic retrieval systems. In SIGIR 1980, pages 236--255, 1980."},{"key":"e_1_2_1_21_1","first-page":"7","volume-title":"Azzopardi et al. {3}","author":"Tunkelang D.","unstructured":"D. Tunkelang . Using QPP to simulate query refinement . In Azzopardi et al. {3} , pages 7 -- 8 . D. Tunkelang. Using QPP to simulate query refinement. In Azzopardi et al. {3}, pages 7--8."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1080343.1080347"},{"key":"e_1_2_1_23_1","first-page":"17","volume-title":"Azzopardi et al. {3}","author":"Zhang P.","unstructured":"P. Zhang , U. C. Beresi , D. Song , and Y. Hou . A probabilistic automaton for the dynamic relevance judgement of users . In Azzopardi et al. {3} , pages 17 -- 18 . P. Zhang, U. C. Beresi, D. Song, and Y. Hou. A probabilistic automaton for the dynamic relevance judgement of users. In Azzopardi et al. {3}, pages 17--18."}],"container-title":["ACM SIGIR Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1924475.1924484","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1924475.1924484","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T10:59:53Z","timestamp":1750244393000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1924475.1924484"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,1,3]]},"references-count":23,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2011,1,3]]}},"alternative-id":["10.1145\/1924475.1924484"],"URL":"https:\/\/doi.org\/10.1145\/1924475.1924484","relation":{},"ISSN":["0163-5840"],"issn-type":[{"value":"0163-5840","type":"print"}],"subject":[],"published":{"date-parts":[[2011,1,3]]},"assertion":[{"value":"2011-01-03","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}