{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T03:18:34Z","timestamp":1740107914642,"version":"3.37.3"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2023,4,28]],"date-time":"2023-04-28T00:00:00Z","timestamp":1682640000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,4,28]],"date-time":"2023-04-28T00:00:00Z","timestamp":1682640000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"DFG","award":["MA 3964\/10-1"],"award-info":[{"award-number":["MA 3964\/10-1"]}]},{"name":"Projekt DEAL"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Digit Libr"],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Retrievability measures the influence a retrieval system has on the access to information in a given collection of items. This measure can help in making an evaluation of the search system based on which insights can be drawn. In this paper, we investigate the retrievability in an integrated search system consisting of items from various categories, particularly focussing on datasets, publications and variables in a real-life digital library. The traditional metrics, that is, the Lorenz curve and Gini coefficient, are employed to visualise the diversity in retrievability scores of the three retrievable document types (specifically datasets, publications, and variables). Our results show a significant popularity bias with certain items being retrieved more often than others. Particularly, it has been shown that certain datasets are more likely to be retrieved than other datasets in the same category. In contrast, the retrievability scores of items from the variable or publication category are more evenly distributed. We have observed that the distribution of document retrievability is more diverse for datasets as compared to publications and variables.<\/jats:p>","DOI":"10.1007\/s00799-023-00363-4","type":"journal-article","created":{"date-parts":[[2023,4,28]],"date-time":"2023-04-28T14:02:11Z","timestamp":1682690531000},"page":"287-301","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Retrievability in an integrated retrieval system: an extended study"],"prefix":"10.1007","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5962-5983","authenticated-orcid":false,"given":"Dwaipayan","family":"Roy","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zeljko","family":"Carevic","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6656-1658","authenticated-orcid":false,"given":"Philipp","family":"Mayr","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,4,28]]},"reference":[{"key":"363_CR1","doi-asserted-by":"publisher","unstructured":"Adali, S., Emery, R.: A uniform framework for integrating knowledge in heterogeneous knowledge systems. In: Proceedings of the Eleventh International Conference on Data Engineering, Taipei, Taiwan, 6\u201310 March 1995. IEEE Computer Society, pp. 513\u2013520 (1995). https:\/\/doi.org\/10.1109\/ICDE.1995.380362","DOI":"10.1109\/ICDE.1995.380362"},{"issue":"1","key":"363_CR2","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1145\/2215676.2215686","volume":"46","author":"J Arguello","year":"2012","unstructured":"Arguello, J.: Federated search in heterogeneous environments. SIGIR Forum 46(1), 78\u201379 (2012). https:\/\/doi.org\/10.1145\/2215676.2215686","journal-title":"SIGIR Forum"},{"key":"363_CR3","doi-asserted-by":"publisher","unstructured":"Azzopardi, L., Vinay, V.: Retrievability: an evaluation measure for higher order information access tasks. In: Shanahan JG., Amer-Yahia S., Manolescu I., et\u00a0al. (eds) Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM 2008, Napa Valley, California, USA, 26\u201330 Oct 2008. ACM, pp. 561\u2013570 (2008). https:\/\/doi.org\/10.1145\/1458082.1458157","DOI":"10.1145\/1458082.1458157"},{"key":"363_CR4","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1007\/978-3-642-16175-9_4","volume-title":"Improving Access to Large Patent Corpora","author":"R Bache","year":"2010","unstructured":"Bache, R., Azzopardi, L.: Improving Access to Large Patent Corpora, pp. 103\u2013121. Springer-Verlag, Berlin, Heidelberg (2010). https:\/\/doi.org\/10.1007\/978-3-642-16175-9_4"},{"key":"363_CR5","doi-asserted-by":"publisher","unstructured":"Bashir, S., Rauber, A.: Analyzing document retrievability in patent retrieval settings. In: International Conference on Database and Expert Systems Applications, pp. 753\u2013760. Springer (2009a). https:\/\/doi.org\/10.1007\/978-3-642-03573-9_63","DOI":"10.1007\/978-3-642-03573-9_63"},{"key":"363_CR6","doi-asserted-by":"publisher","unstructured":"Bashir, S., Rauber, A.: Identification of low\/high retrievable patents using content-based features. In: Proceedings of the 2nd International Workshop on Patent Information Retrieval. Association for Computing Machinery, New York, NY, USA, PaIR \u201909, pp. 9\u201316 (2009b). https:\/\/doi.org\/10.1145\/1651343.1651346","DOI":"10.1145\/1651343.1651346"},{"key":"363_CR7","doi-asserted-by":"publisher","unstructured":"Bashir, S., Rauber, A.: Improving retrievability of patents with cluster-based pseudo-relevance feedback documents selection. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management. Association for Computing Machinery, New York, NY, USA, CIKM \u201909, pp. 1863\u20131866 (2009c). https:\/\/doi.org\/10.1145\/1645953.1646250","DOI":"10.1145\/1645953.1646250"},{"issue":"8","key":"363_CR8","doi-asserted-by":"publisher","first-page":"1515","DOI":"10.1002\/asi.21549","volume":"62","author":"S Bashir","year":"2011","unstructured":"Bashir, S., Rauber, A.: On the relationship between query characteristics and ir functions retrieval bias. J. Am. Soc. Inf. Sci. Technol. 62(8), 1515\u20131532 (2011). https:\/\/doi.org\/10.1002\/asi.21549","journal-title":"J. Am. Soc. Inf. Sci. Technol."},{"issue":"2","key":"363_CR9","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1145\/382979.383040","volume":"19","author":"J Callan","year":"2001","unstructured":"Callan, J., Connell, M.: Query-based sampling of text databases. ACM Trans. Inf. Syst. (TOIS) 19(2), 97\u2013130 (2001). https:\/\/doi.org\/10.1145\/382979.383040","journal-title":"ACM Trans. Inf. Syst. (TOIS)"},{"key":"363_CR10","doi-asserted-by":"publisher","unstructured":"Carevic, Z., Sch\u00fcller, S., Mayr, P., et\u00a0al.: Contextualised browsing in a digital library\u2019s living lab. In: Proceedings of the 18th ACM\/IEEE on Joint Conference on Digital Libraries, pp. 89\u201398 (2018). https:\/\/doi.org\/10.1145\/3197026.3197054","DOI":"10.1145\/3197026.3197054"},{"key":"363_CR11","doi-asserted-by":"publisher","unstructured":"Carevic, Z., Roy, D., Mayr, P.: Characteristics of dataset retrieval sessions: experiences from a real-life digital library. In: International Conference on Theory and Practice of Digital Libraries, pp. 185\u2013193. Springer (2020). https:\/\/doi.org\/10.1007\/978-3-030-54956-5_14","DOI":"10.1007\/978-3-030-54956-5_14"},{"key":"363_CR12","doi-asserted-by":"publisher","unstructured":"Carmel, D., Yom-Tov, E.: Estimating the Query Difficulty for Information Retrieval. Synthesis Lectures on Information Concepts, Retrieval, and Services. Morgan & Claypool Publishers (2010). https:\/\/doi.org\/10.2200\/S00235ED1V01Y201004ICR015","DOI":"10.2200\/S00235ED1V01Y201004ICR015"},{"key":"363_CR13","doi-asserted-by":"publisher","unstructured":"Carmel, D., Yom-Tov, E., Darlow, A., et\u00a0al.: What makes a query difficult? In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, SIGIR \u201906, pp. 390\u2013397 (2006). https:\/\/doi.org\/10.1145\/1148170.1148238","DOI":"10.1145\/1148170.1148238"},{"key":"363_CR14","unstructured":"Cole, M., Liu, J., Belkin, N., et\u00a0al.: Usefulness as the criterion for evaluation of interactive information retrieval. in: Proc HCIR, pp. 1\u20134 (2009)"},{"key":"363_CR15","doi-asserted-by":"publisher","unstructured":"Friedrich, T.: Looking for data. PhD thesis, Humboldt-Universit\u00e4t zu Berlin, Philosophische Fakult\u00e4t (2020). https:\/\/doi.org\/10.18452\/22173","DOI":"10.18452\/22173"},{"issue":"3","key":"363_CR16","doi-asserted-by":"publisher","first-page":"306","DOI":"10.2307\/1937992","volume":"54","author":"JL Gastwirth","year":"1972","unstructured":"Gastwirth, J.L.: The estimation of the Lorenz curve and Gini index. Rev. Econ. Stat. 54(3), 306\u2013316 (1972). (http:\/\/www.jstor.org\/stable\/1937992)","journal-title":"Rev. Econ. Stat."},{"issue":"5","key":"363_CR17","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1002\/asi.24165","volume":"70","author":"K Gregory","year":"2019","unstructured":"Gregory, K., Groth, P., Cousijn, H., et al.: Searching data: a review of observational data retrieval practices in selected disciplines. J. Assoc. Inf. Sci. Technol. 70(5), 419\u2013432 (2019). https:\/\/doi.org\/10.1002\/asi.24165","journal-title":"J. Assoc. Inf. Sci. Technol."},{"key":"363_CR18","doi-asserted-by":"publisher","unstructured":"Hienert, D., Mutschke, P.: A usefulness-based approach for measuring the local and global effect of IIR services. In: Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval, CHIIR \u201916, pp. 153\u2013162 (2016). https:\/\/doi.org\/10.1145\/2854946.2854962","DOI":"10.1145\/2854946.2854962"},{"key":"363_CR19","doi-asserted-by":"publisher","unstructured":"Hienert, D., Kern, D., Boland, K., et\u00a0al.: A digital library for research data and related information in the social sciences. In: 2019 ACM\/IEEE Joint Conference on Digital Libraries (JCDL), pp. 148\u2013157. IEEE, Champaign, IL, USA (2019). https:\/\/doi.org\/10.1109\/JCDL.2019.00030","DOI":"10.1109\/JCDL.2019.00030"},{"key":"363_CR20","doi-asserted-by":"publisher","unstructured":"Kacprzak, E., Koesten, L.M., Ib\u00e1\u00f1ez, L.D., et\u00a0al.: A query log analysis of dataset search. In: International Conference on Web Engineering, pp. 429\u2013436. Springer (2017). https:\/\/doi.org\/10.1007\/978-3-319-60131-1_29","DOI":"10.1007\/978-3-319-60131-1_29"},{"key":"363_CR21","doi-asserted-by":"publisher","unstructured":"Kacprzak, E., Koesten, L., Tennison, J., et\u00a0al.: Characterising dataset search queries. In: Companion Proceedings of the The Web Conference 2018. International World Wide Web Conferences Steering Committee, WWW \u201918, pp. 1485\u20131488 (2018). https:\/\/doi.org\/10.1145\/3184558.3191597","DOI":"10.1145\/3184558.3191597"},{"key":"363_CR22","doi-asserted-by":"publisher","unstructured":"Kern, D., Mathiak, B.: Are there any differences in data set retrieval compared to well-known literature retrieval? In: International Conference on Theory and Practice of Digital Libraries, pp. 197\u2013208. Springer (2015). https:\/\/doi.org\/10.1007\/978-3-319-24592-8_15","DOI":"10.1007\/978-3-319-24592-8_15"},{"issue":"2","key":"363_CR23","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1007\/s00799-021-00303-0","volume":"22","author":"T Kr\u00e4mer","year":"2021","unstructured":"Kr\u00e4mer, T., Papenmeier, A., Carevic, Z., et al.: Data-seeking behaviour in the social sciences. Int. J. Digit. Libr. 22(2), 175\u2013195 (2021). https:\/\/doi.org\/10.1007\/s00799-021-00303-0","journal-title":"Int. J. Digit. Libr."},{"key":"363_CR24","doi-asserted-by":"publisher","unstructured":"Kunze, S.R., Auer, S.: Dataset retrieval. In: 2013 IEEE Seventh International Conference on Semantic Computing, Irvine, CA, USA, 16\u201318 Sep 2013. IEEE Computer Society, pp. 1\u20138 (2013). https:\/\/doi.org\/10.1109\/ICSC.2013.12","DOI":"10.1109\/ICSC.2013.12"},{"key":"363_CR25","doi-asserted-by":"publisher","unstructured":"Lalmas, M.: Aggregated search. In: Advanced Topics in Information Retrieval, The Information Retrieval Series, vol. 33, pp. 109\u2013123. Springer (2011). https:\/\/doi.org\/10.1007\/978-3-642-20946-8_5","DOI":"10.1007\/978-3-642-20946-8_5"},{"key":"363_CR26","unstructured":"Nikkhoo, H.K.: The impact of near-duplicate documents on information retrieval evaluation. In: Masters thesis. University of Waterloo (2011). http:\/\/hdl.handle.net\/10012\/5750"},{"key":"363_CR27","doi-asserted-by":"publisher","unstructured":"Roy, D., Carevic, Z., Mayr, P.: Studying retrievability of publications and datasets in an integrated retrieval system. In: JCDL \u201922: The ACM\/IEEE Joint Conference on Digital Libraries in 2022, Cologne, Germany, 20\u2013 24 June 2022. ACM, p. 8 (2022). https:\/\/doi.org\/10.1145\/3529372.3530931","DOI":"10.1145\/3529372.3530931"},{"issue":"1","key":"363_CR28","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1007\/s00799-017-0215-9","volume":"19","author":"T Samar","year":"2018","unstructured":"Samar, T., Traub, M.C., Ossenbruggen, J., et al.: Quantifying retrieval bias in web archive search. Int. J. Digit. Libr. 19(1), 57\u201375 (2018). https:\/\/doi.org\/10.1007\/s00799-017-0215-9","journal-title":"Int. J. Digit. Libr."},{"key":"363_CR29","doi-asserted-by":"publisher","unstructured":"Sparck Jones, K., Walker, S., Robertson, S.: A probabilistic model of information retrieval: development and comparative experiments: part 1. Inf. Process. Manag. 36(6), 779\u2013808 (2000). https:\/\/doi.org\/10.1016\/S0306-4573(00)00015-7","DOI":"10.1016\/S0306-4573(00)00015-7"},{"key":"363_CR30","doi-asserted-by":"publisher","unstructured":"Traub, M.C., Samar, T., van Ossenbruggen, J., et\u00a0al.: Querylog-based assessment of retrievability bias in a large newspaper corpus. In: Proceedings of the 16th ACM\/IEEE-CS on Joint Conference on Digital Libraries, JCDL 2016, Newark, NJ, USA, 19\u201323 June 2016. ACM, pp. 7\u201316 (2016). https:\/\/doi.org\/10.1145\/2910896.2910907","DOI":"10.1145\/2910896.2910907"},{"key":"363_CR31","unstructured":"Tsereteli, T., Kartal, Y.S., Ponzetto, S.P., et\u00a0al.: Overview of the SV-ident 2022 shared task on survey variable identification in social science publications. In: Proceedings of the Third Workshop on Scholarly Document Processing. Association for Computational Linguistics, Gyeongju, Republic of Korea, pp. 229\u2013246 (2022). https:\/\/aclanthology.org\/2022.sdp-1.29"},{"key":"363_CR32","doi-asserted-by":"publisher","unstructured":"Webber, W., Moffat, A., Zobel, J.: A similarity measure for indefinite rankings. ACM Trans. Inf. Syst. (2010). https:\/\/doi.org\/10.1145\/1852102.1852106","DOI":"10.1145\/1852102.1852106"},{"key":"363_CR33","doi-asserted-by":"publisher","unstructured":"Wilkie, C., Azzopardi, L.: Best and fairest: an empirical analysis of retrieval system bias. In: Proceedings of the 36th European Conference on IR Research on Advances in Information Retrieval, vol. 8416, pp. 13\u201325. Springer-Verlag, Berlin, Heidelberg, ECIR 2014 (2014a). https:\/\/doi.org\/10.1007\/978-3-319-06028-6_2","DOI":"10.1007\/978-3-319-06028-6_2"},{"key":"363_CR34","doi-asserted-by":"publisher","unstructured":"Wilkie, C., Azzopardi, L.: A retrievability analysis: exploring the relationship between retrieval bias and retrieval performance. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. Association for Computing Machinery, New York, NY, USA, CIKM \u201914, pp. 81\u201390 (2014b). https:\/\/doi.org\/10.1145\/2661829.2661948","DOI":"10.1145\/2661829.2661948"},{"key":"363_CR35","doi-asserted-by":"publisher","unstructured":"Wilkie, C., Azzopardi, L.: A topical approach to retrievability bias estimation. In: Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval. Association for Computing Machinery, New York, NY, USA, ICTIR \u201916, pp. 119\u2013122 (2016). https:\/\/doi.org\/10.1145\/2970398.2970437","DOI":"10.1145\/2970398.2970437"},{"key":"363_CR36","doi-asserted-by":"publisher","unstructured":"Wilkie, C., Azzopardi, L.: Algorithmic bias: do good systems make relevant documents more retrievable? In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. Association for Computing Machinery, New York, NY, USA, CIKM \u201917, pp. 2375\u20132378 (2017). https:\/\/doi.org\/10.1145\/3132847.3133135","DOI":"10.1145\/3132847.3133135"}],"container-title":["International Journal on Digital Libraries"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00799-023-00363-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00799-023-00363-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00799-023-00363-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,26]],"date-time":"2024-06-26T06:09:26Z","timestamp":1719382166000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00799-023-00363-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,28]]},"references-count":36,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["363"],"URL":"https:\/\/doi.org\/10.1007\/s00799-023-00363-4","relation":{},"ISSN":["1432-5012","1432-1300"],"issn-type":[{"type":"print","value":"1432-5012"},{"type":"electronic","value":"1432-1300"}],"subject":[],"published":{"date-parts":[[2023,4,28]]},"assertion":[{"value":"6 December 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 March 2023","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 March 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 April 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Philipp Mayr is on the Editorial Board of the \u201cInternational Journal on Digital Libraries\u201d and guest co-editor of the special issue \u201cJCDL 2022\u201d. In this case, the co-editors are handling the review process.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}