{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:17:34Z","timestamp":1750220254354,"version":"3.41.0"},"reference-count":15,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2021,6,1]],"date-time":"2021-06-01T00:00:00Z","timestamp":1622505600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGIR Forum"],"published-print":{"date-parts":[[2021,6]]},"abstract":"<jats:p>The Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021) was held in conjunction with the ACM SIGIR Conference on Human Information Interaction &amp; Retrieval (CHIIR 2021) in Canberra, Australia, as a virtual event. WEPIR 2021 followed on from the first and second WEPIRs held at CHIIR 2018 and 2019. The purpose of the workshop was again to bring together researchers from different backgrounds, interested in advancing the evaluation of personalisation in information retrieval. The workshop focused on further development of a common understanding of the challenges, requirements and practical limitations of personalisation in information retrieval and its evaluation.<\/jats:p>","DOI":"10.1145\/3476415.3476422","type":"journal-article","created":{"date-parts":[[2021,7,16]],"date-time":"2021-07-16T16:07:00Z","timestamp":1626451620000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Report on the CHIIR 2021 third workshop on evaluation of personalisation in information retrieval (WEPIR 2021)"],"prefix":"10.1145","volume":"55","author":[{"given":"Gareth J. F.","family":"Jones","sequence":"first","affiliation":[{"name":"Dublin City University, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nicholas J.","family":"Belkin","sequence":"additional","affiliation":[{"name":"Rutgers University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Noriko","family":"Kando","sequence":"additional","affiliation":[{"name":"National Institute of Informatics, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gabriella","family":"Pasi","sequence":"additional","affiliation":[{"name":"University of Milano-Bicocca, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,7,16]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2641383.2641390"},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the 23rd Text REtrieval Conference, TREC 2014","author":"Carterette Ben","year":"2015","unstructured":"Ben Carterette , Evangelos Kanoulos , Mark Hall , and Peter Clough . Overview of the trec 2014 session track. In Ellen M. Voorheen, editor , Proceedings of the 23rd Text REtrieval Conference, TREC 2014 , Gaithersburg, MD , 2015 . National Institute of Standards and Technology. URL http:\/\/trec.nist.gov\/pubs\/trec23\/papers\/overview-session.pdf. Ben Carterette, Evangelos Kanoulos, Mark Hall, and Peter Clough. Overview of the trec 2014 session track. In Ellen M. Voorheen, editor, Proceedings of the 23rd Text REtrieval Conference, TREC 2014, Gaithersburg, MD, 2015. National Institute of Standards and Technology. URL http:\/\/trec.nist.gov\/pubs\/trec23\/papers\/overview-session.pdf."},{"key":"e_1_2_1_4_1","volume-title":"Adaptation and Personalization","author":"Conlan Owen","year":"2016","unstructured":"Owen Conlan , Liadh Kelly , Kevin Koidl , Seamus Lawless , Killian Levacher , and Athanasios Staikopoulos , editors. EvalUMAP2016 : Towards Comparative Evaluation in the User Modelling , Adaptation and Personalization , Halifax, Canada , 2016 . Owen Conlan, Liadh Kelly, Kevin Koidl, Seamus Lawless, Killian Levacher, and Athanasios Staikopoulos, editors. EvalUMAP2016: Towards Comparative Evaluation in the User Modelling, Adaptation and Personalization, Halifax, Canada, 2016."},{"key":"e_1_2_1_5_1","first-page":"123","volume-title":"The TREC interactive tracks: Putting the user into search","author":"Dumais Susan","year":"2005","unstructured":"Susan Dumais and Nicholas J. Belkin . The TREC interactive tracks: Putting the user into search . In Ellen M. Voorhees and Donna K. Harman, editors, TREC. Experiment and evaluation in information retrieval, pages 123 -- 152 . MIT Press , Cambridge, MA, 2005 . Susan Dumais and Nicholas J. Belkin. The TREC interactive tracks: Putting the user into search. In Ellen M. Voorhees and Donna K. Harman, editors, TREC. Experiment and evaluation in information retrieval, pages 123 -- 152. MIT Press, Cambridge, MA, 2005."},{"key":"e_1_2_1_6_1","volume-title":"Rowman & Littlefield","author":"Falk John H","year":"2018","unstructured":"John H Falk and Lynn D Dierking . Learning from museums . Rowman & Littlefield , 2018 . John H Falk and Lynn D Dierking. Learning from museums. Rowman & Littlefield, 2018."},{"key":"e_1_2_1_7_1","first-page":"88","volume-title":"Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021","author":"Ghafourian Yasin","year":"2021","unstructured":"Yasin Ghafourian , Petr Knoth , and Allan Hanbury . Information retrieval evaluation in knowledge acquisition tasks . In Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021 ), pages 88 -- 95 , Canberra, Australia, Virtual event , 2021 . Yasin Ghafourian, Petr Knoth, and Allan Hanbury. Information retrieval evaluation in knowledge acquisition tasks. In Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021), pages 88 -- 95, Canberra, Australia, Virtual event, 2021."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274784.3274796"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3458537.3458541"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2010056"},{"key":"e_1_2_1_11_1","volume-title":"Prochzta Sen. Overview of the CLEF 2017 personalised information retrieval pilot lab (PIRCLEF 2017). In Proceedings of CLEF 2017","author":"Pasi Gabriella","year":"2017","unstructured":"Gabriella Pasi , Gareth J. F. Jones , Stefania Marrara , Camilla Sanvitto , Debasis Ganguly , and Prochzta Sen. Overview of the CLEF 2017 personalised information retrieval pilot lab (PIRCLEF 2017). In Proceedings of CLEF 2017 , Dublin, Ireland , 2017 . Springer. Gabriella Pasi, Gareth J. F. Jones, Stefania Marrara, Camilla Sanvitto, Debasis Ganguly, and Prochzta Sen. Overview of the CLEF 2017 personalised information retrieval pilot lab (PIRCLEF 2017). In Proceedings of CLEF 2017, Dublin, Ireland, 2017. Springer."},{"key":"e_1_2_1_12_1","volume-title":"Prochzta Sen. Overview of the CLEF 2018 personalised information retrieval pilot lab (PIR-CLEF 2018). In Proceedings of CLEF 2018","author":"Pasi Gabriella","year":"2018","unstructured":"Gabriella Pasi , Gareth J. F. Jones , Keith Curtis , Stefania Marrara , Camilla Sanvitto , Debasis Ganguly , and Prochzta Sen. Overview of the CLEF 2018 personalised information retrieval pilot lab (PIR-CLEF 2018). In Proceedings of CLEF 2018 , Avignon, France , 2018 . Springer. Gabriella Pasi, Gareth J. F. Jones, Keith Curtis, Stefania Marrara, Camilla Sanvitto, Debasis Ganguly, and Prochzta Sen. Overview of the CLEF 2018 personalised information retrieval pilot lab (PIR-CLEF 2018). In Proceedings of CLEF 2018, Avignon, France, 2018. Springer."},{"key":"e_1_2_1_13_1","first-page":"417","volume-title":"Camilla Sanvitto. Overview of the CLEF 2019 personalised information retrieval lab (2019). In Proceedings of CLEF 2019","author":"Pasi Gabriella","year":"2019","unstructured":"Gabriella Pasi , Gareth J. F. Jones , Lorraine Goeuriot , Liadh Kelly , Stefania Marrara , and Camilla Sanvitto. Overview of the CLEF 2019 personalised information retrieval lab (2019). In Proceedings of CLEF 2019 , pages 417 -- 424 , Lugano, Switzerland , 2019 . Springer. Gabriella Pasi, Gareth J. F. Jones, Lorraine Goeuriot, Liadh Kelly, Stefania Marrara, and Camilla Sanvitto. Overview of the CLEF 2019 personalised information retrieval lab (2019). In Proceedings of CLEF 2019, pages 417--424, Lugano, Switzerland, 2019. Springer."},{"key":"e_1_2_1_14_1","first-page":"13","volume-title":"Proceedings of The Seventh International Workshop on Evaluating Information Access - A Satellite Workshop of NTCIR-12","author":"Sanvitto Camilla","year":"2016","unstructured":"Camilla Sanvitto , Debasis Ganguly , Gareth J. F. Jones , and Gabriella Pasi . A laboratory-based method for the evaluation of personalised search. In Emine Yilmaz and Charles L. A. Clarke, editors , Proceedings of The Seventh International Workshop on Evaluating Information Access - A Satellite Workshop of NTCIR-12 , pages 13 -- 16 , Tokyo, Japan , 2016 . Camilla Sanvitto, Debasis Ganguly, Gareth J. F. Jones, and Gabriella Pasi. A laboratory-based method for the evaluation of personalised search. In Emine Yilmaz and Charles L. A. Clarke, editors, Proceedings of The Seventh International Workshop on Evaluating Information Access - A Satellite Workshop of NTCIR-12, pages 13 -- 16, Tokyo, Japan, 2016."},{"key":"e_1_2_1_15_1","first-page":"79","volume-title":"Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021","author":"Shoji Yoshiyuki","year":"2021","unstructured":"Yoshiyuki Shoji , Kenro Aihara , Martin J. D\u00fcrst , Noriko Kando , Takuya Nakaya , Hiroaki Ohshima , Takehiro Yamamoto , and Yusuke Yamamoto . Location-based reminder for memorizing what visitors learn at a museum . In Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021 ), pages 79 -- 87 , Canberra, Australia, Virtual event , 2021 . Yoshiyuki Shoji, Kenro Aihara, Martin J. D\u00fcrst, Noriko Kando, Takuya Nakaya, Hiroaki Ohshima, Takehiro Yamamoto, and Yusuke Yamamoto. Location-based reminder for memorizing what visitors learn at a museum. In Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021), pages 79 -- 87, Canberra, Australia, Virtual event, 2021."},{"key":"e_1_2_1_16_1","first-page":"96","volume-title":"Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021","author":"Ueta Masaki","year":"2021","unstructured":"Masaki Ueta , Tomoya Hashiguchi , Hun-Long Pham , Yoshiyuki Shoji , Noriko Kando , Yusuke Yamamoto , Takehiro Yamamoto , and Hiroaki Ohshima . Quiz generation on the electronic guide application for improving learning experience in the museum . In Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021 ), pages 96 -- 104 , Canberra, Australia, Virtual event , 2021 . Masaki Ueta, Tomoya Hashiguchi, Hun-Long Pham, Yoshiyuki Shoji, Noriko Kando, Yusuke Yamamoto, Takehiro Yamamoto, and Hiroaki Ohshima. Quiz generation on the electronic guide application for improving learning experience in the museum. In Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021), pages 96 -- 104, Canberra, Australia, Virtual event, 2021."}],"container-title":["ACM SIGIR Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3476415.3476422","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3476415.3476422","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:45Z","timestamp":1750188645000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3476415.3476422"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6]]},"references-count":15,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,6]]}},"alternative-id":["10.1145\/3476415.3476422"],"URL":"https:\/\/doi.org\/10.1145\/3476415.3476422","relation":{},"ISSN":["0163-5840"],"issn-type":[{"type":"print","value":"0163-5840"}],"subject":[],"published":{"date-parts":[[2021,6]]},"assertion":[{"value":"2021-07-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}