{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,25]],"date-time":"2026-01-25T04:34:26Z","timestamp":1769315666800,"version":"3.49.0"},"reference-count":42,"publisher":"Emerald","issue":"3","license":[{"start":{"date-parts":[[1997,8,1]],"date-time":"1997-08-01T00:00:00Z","timestamp":870393600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[1997,8,1]]},"abstract":"<jats:p>The paper describes the ideas and assumptions underlying the development of a new method for the evaluation and testing of interactive information retrieval (IR) systems, and reports on the initial tests of the proposed method. The method is designed to collect different types of empirical data, i.e. cognitive data as well as traditional systems performance data. The method is based on the novel concept of a \u2018simulated work task situation\u2019 or scenario and the involvement of real end users. The method is also based on a mixture of simulated and real information needs, and involves a group of test persons as well as assessments made by individual panel members. The relevance assessments are made with reference to the concepts of topical as well as situational relevance. The method takes into account the dynamic nature of information needs which are assumed to develop over time for the same user, a variability which is presumed to be strongly connected to the processes of relevance assessment.<\/jats:p>","DOI":"10.1108\/eum0000000007198","type":"journal-article","created":{"date-parts":[[2004,5,12]],"date-time":"2004-05-12T08:32:11Z","timestamp":1084350731000},"page":"225-250","source":"Crossref","is-referenced-by-count":149,"title":["The development of a method for the evaluation of interactive information retrieval systems"],"prefix":"10.1108","volume":"53","author":[{"given":"Pia","family":"Borlund","sequence":"first","affiliation":[]},{"given":"Peter","family":"Ingwersen","sequence":"additional","affiliation":[]}],"member":"140","reference":[{"key":"p_1","doi-asserted-by":"publisher","DOI":"10.1145\/160688.160692"},{"key":"p_2","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(94)00047-7"},{"key":"p_3","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199601)47:1<85::AID-ASI8>3.0.CO;2-Z"},{"key":"p_4","doi-asserted-by":"publisher","DOI":"10.1108\/eb026960"},{"key":"p_5","volume-title":"Design.","volume":"1","author":"Cleverdon C.W.","year":"1966"},{"key":"p_6","doi-asserted-by":"publisher","DOI":"10.1002\/asi.4630260604"},{"key":"p_7","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(87)90016-1"},{"key":"p_8","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(90)90050-C"},{"key":"p_9","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(93)90026-A"},{"key":"p_10","doi-asserted-by":"publisher","DOI":"10.1515\/libr.1995.45.3-4.160"},{"key":"p_11","unstructured":"Balch Petersen, H. and Borlund, P. Pilotunders\u00f8gelse af IR-teknikker: eksakt match versus partial match. [Pilot test of IR techniques: exact match vs. partial match]. UnpublishedMSc thesis,Danmarks Biblioteksskole, Copenhagen, 1995."},{"key":"p_12","volume-title":"Understanding computers and cognition","author":"Winograd T.","year":"1986"},{"key":"p_13","volume-title":"Information retrieval interaction","author":"Ingwersen P.","year":"1992"},{"key":"p_14","doi-asserted-by":"publisher","DOI":"10.1145\/160688.160760"},{"key":"p_15","doi-asserted-by":"publisher","DOI":"10.1145\/215206.215352"},{"key":"p_16","series-title":"May 5","volume-title":"What makes the question? The Listener, 62","author":"Mackay D.M.","year":"1960"},{"key":"p_17","doi-asserted-by":"publisher","DOI":"10.5860\/crl_29_03_178"},{"key":"p_18","volume-title":"M\u00fcnchen-Pullach: Verlag Dokumentation Saur KG","author":"Wersig G.","year":"1971"},{"key":"p_19","doi-asserted-by":"publisher","DOI":"10.1108\/eb026722"},{"key":"p_20","doi-asserted-by":"publisher","DOI":"10.1016\/0143-6236(84)90070-X"},{"key":"p_21","doi-asserted-by":"publisher","DOI":"10.1108\/eb026727"},{"key":"p_22","first-page":"47","volume":"27","author":"Saracevic T.","year":"1990","journal-title":"ASIS Proceedings"},{"key":"p_23","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199404)45:3<207::AID-ASI10>3.0.CO;2-1"},{"key":"p_24","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199210)43:9<602::AID-ASI3>3.0.CO;2-Q"},{"key":"p_25","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199404)45:3<160::AID-ASI6>3.0.CO;2-4"},{"key":"p_26","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199404)45:3<135::AID-ASI3>3.0.CO;2-1"},{"key":"p_27","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199404)45:3<124::AID-ASI2>3.0.CO;2-8"},{"key":"p_28","volume-title":"Information science: integration in perspective.","author":"Saracevic T.","year":"1996"},{"key":"p_29","doi-asserted-by":"publisher","DOI":"10.1016\/0020-0271(73)90096-X"},{"key":"p_30","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199404)45:3<201::AID-ASI9>3.0.CO;2-W"},{"key":"p_31","volume-title":"Relevance: communication and cognition","author":"Sperber D.","year":"1986"},{"key":"p_32","doi-asserted-by":"publisher","DOI":"10.1108\/eb026436"},{"key":"p_33","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199404)45:3<149::AID-ASI5>3.0.CO;2-J"},{"key":"p_34","doi-asserted-by":"publisher","DOI":"10.1108\/eb026895"},{"key":"p_35","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199404)45:3<142::AID-ASI4>3.0.CO;2-6"},{"key":"p_36","volume-title":"Cal.: Dialog","author":"Dialog","year":"1993"},{"key":"p_37","first-page":"31","author":"Tenopir C.","year":"1994","journal-title":"On-line"},{"key":"p_38","first-page":"272","volume-title":"eds. Proceedings of the 17th ACM Sigir Conference on Research and Development of Information Retrieval. Dublin","volume":"1994","author":"Morita M.","year":"1994"},{"key":"p_39","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4471-2099-5_8"},{"key":"p_40","first-page":"101","volume-title":"eds. Proceedings of the 17th ACM Sigir Conference on Research and Development of Information Retrieval. Dublin","volume":"1994","author":"Ingwersen P.","year":"1994"},{"key":"p_41","first-page":"51","volume-title":"eds. Proceedings of the 17th ACM Sigir Conference on Research and Development of Information Retrieval. Dublin","volume":"1994","author":"Ellis D.","year":"1994"},{"key":"p_42","doi-asserted-by":"publisher","DOI":"10.1145\/215206.215355"}],"container-title":["Journal of Documentation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/EUM0000000007198\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/EUM0000000007198\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T00:52:48Z","timestamp":1753404768000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/jd\/article\/53\/3\/225-250\/211037"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1997,8,1]]},"references-count":42,"journal-issue":{"issue":"3","published-print":{"date-parts":[[1997,8,1]]}},"alternative-id":["10.1108\/EUM0000000007198"],"URL":"https:\/\/doi.org\/10.1108\/eum0000000007198","relation":{},"ISSN":["0022-0418"],"issn-type":[{"value":"0022-0418","type":"print"}],"subject":[],"published":{"date-parts":[[1997,8,1]]}}}