{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,4,17]],"date-time":"2025-04-17T04:17:14Z","timestamp":1744863434496,"version":"3.40.4"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,4,16]],"date-time":"2025-04-16T00:00:00Z","timestamp":1744761600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,4,16]],"date-time":"2025-04-16T00:00:00Z","timestamp":1744761600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["JP19K20629","JP15H01721"],"award-info":[{"award-number":["JP19K20629","JP15H01721"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Discov Computing"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>In an academic paper search to confirm the novelty of a research project, it is important to improve the recall score for the number of search results that users can check to comprehensively collect research papers related to the user\u2019s information need. However, it may be necessary to check papers in the lower ranks to cover the relevant papers comprehensively when using single search method. To improve the recall score for the number of papers that users can check, we considered that it would be effective to integrate ranking results using multiple search methods with different approaches. This is based on the idea that relevant papers that do not appear in the higher ranks for one method can be found using other methods, and that effect would be amplified by integrating more ranking results. As the methods to be integrated, we used the ranking methods in the vector space model, the query likelihood model, and a newly proposed method. Our method is based on a topic-based Boolean search that uses the topic analysis result from latent Dirichlet allocation, and ranked papers in descending order the number of times they are included in each search result. We performed an evaluation using the NTCIR- 1 and -\u20092 datasets, and confirmed that our topic-based search method showed different trends from rankings based on conventional vector space model and query likelihood model. Furthermore, we showed the best performance by the re-ranking using these three methods comparing other combinations.<\/jats:p>","DOI":"10.1007\/s10791-025-09527-z","type":"journal-article","created":{"date-parts":[[2025,4,16]],"date-time":"2025-04-16T18:28:23Z","timestamp":1744828103000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Ranking method for an academic paper search with an emphasis on comprehensiveness"],"prefix":"10.1007","volume":"28","author":[{"given":"Satoshi","family":"Fukuda","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yoichi","family":"Tomiura","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,4,16]]},"reference":[{"key":"9527_CR1","unstructured":"Mai G, Janowicz K, Yan B. Combining text embedding and knowledge graph embedding techniques for academic search engines. In: Joint proceedings of ISWC 2018 workshops SemDeep-4 and NLIWOD 2018;4:77\u201388"},{"key":"9527_CR2","doi-asserted-by":"crossref","unstructured":"Takaku M, Egusa Y. Simple document-by-document search tool fuwatto search using web API. In: Proceedings of the 16th international conference on asia-pacific digital libraries, 2014:312\u2013319","DOI":"10.1007\/978-3-319-12823-8_32"},{"key":"9527_CR3","unstructured":"Nascimento C, Laender AHF, Silva AS, Gonalves MA. Simple document-by-document search tool fuwatto search using web API. In: Proceedings of the 11th annual international ACM\/IEEE joint conference on digital libraries, 2011;297\u2013306"},{"key":"9527_CR4","doi-asserted-by":"crossref","unstructured":"Hassan HAM. Personalized research paper recommendation using deep learning. In: Proceedings of the 25th conference on user modeling, adaptation and personalization, 2017;327\u2013330","DOI":"10.1145\/3079628.3079708"},{"key":"9527_CR5","unstructured":"Alzoghbi A, Ayala VAA, Fischer PM, Lausen G. Learning-to-rank in research paper CBF recommendation: leveraging irrelevant papers. In: Proceedings of the 3rd workshop new trends content-based recommender syst, 2016;43\u201346"},{"key":"9527_CR6","doi-asserted-by":"crossref","unstructured":"Amami M, Pasi G, Stella F, Faiz R. An LDA-based approach to scientific paper recommendation. In: International conference on applications of natural language to information systems, 2016;200\u2013210","DOI":"10.1007\/978-3-319-41754-7_17"},{"key":"9527_CR7","doi-asserted-by":"crossref","unstructured":"Verberne S, Sappelli M, Kraaij W. Query term suggestion in academic search. In: Proceedings of the 36th european conference on IR research, 2014;560\u2013566","DOI":"10.1007\/978-3-319-06028-6_57"},{"key":"9527_CR8","doi-asserted-by":"crossref","unstructured":"Kim Y, Seo J, Croft WB. Automatic Boolean query suggestion for professional search. In: Proceedings of the 34th international ACM SIGIR conference on research and development in information retrieval, 2011;825\u2013834","DOI":"10.1145\/2009916.2010026"},{"key":"9527_CR9","doi-asserted-by":"crossref","unstructured":"Scells H, Zuccon G, Koopman B, Clark J. Automatic Boolean query formulation for systematic review literature search. In: Proceedings of the web conference, 2020;1071\u20131081","DOI":"10.1145\/3366423.3380185"},{"key":"9527_CR10","doi-asserted-by":"crossref","unstructured":"Wang S, Scells H, Koopman B, Zuccon G. Can ChatGPT write a good boolean query for systematic review literature search? In: Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, 2023;1426-1436","DOI":"10.1145\/3539618.3591703"},{"key":"9527_CR11","doi-asserted-by":"crossref","unstructured":"Budau L, Ensan F. Fully automated scholarly search for biomedical systematic literature reviews. IEEE Access 2024;12","DOI":"10.1109\/ACCESS.2024.3405529"},{"issue":"10","key":"9527_CR12","doi-asserted-by":"publisher","first-page":"5339","DOI":"10.1097\/GOX.0000000000005339","volume":"11","author":"L Alaniz","year":"2023","unstructured":"Alaniz L, Vu C, Pfaff MJ. The utility of artificial intelligence for systematic reviews and Boolean query formulation and translation. Plastic Reconstruc SurgeryGlobal Open. 2023;11(10):5339.","journal-title":"Plastic Reconstruc SurgeryGlobal Open"},{"key":"9527_CR13","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1016\/j.ins.2019.04.019","volume":"492","author":"HK Azad","year":"2019","unstructured":"Azad HK, Deepak A. A new approach for query expansion using wikipedia and wordnet. Inf Sci. 2019;492:147\u201363.","journal-title":"Inf Sci"},{"key":"9527_CR14","doi-asserted-by":"crossref","unstructured":"Xiong C, Callan J. Query expansion with freebase. In: Proceedings of the 2015 international conference on the theory of information retrieval, 2015;111\u2013120","DOI":"10.1145\/2808194.2809446"},{"key":"9527_CR15","doi-asserted-by":"crossref","unstructured":"Gabn J, Ares ME, Parapar J. Keyword embeddings for query suggestion. In: Proceedings of the european conference on information retrieval, 2023;346\u2013360","DOI":"10.1007\/978-3-031-28244-7_22"},{"key":"9527_CR16","doi-asserted-by":"crossref","unstructured":"Salton G, McGill M. Introduction to modern information retrieval 1983.","DOI":"10.1145\/182.358466"},{"key":"9527_CR17","unstructured":"Kosmopoulos A, Androutsopoulos I, Paliouras G. Biomedical semantic indexing using dense word vectors in BioASQ. Biomedical Semantics 2015."},{"issue":"2","key":"9527_CR18","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1145\/984321.984322","volume":"22","author":"C Zhai","year":"2004","unstructured":"Zhai C, Lafferty J. A study of smoothing methods for language models applied to information retrieval. ACM Trans Inf Syst. 2004;22(2):179\u2013214.","journal-title":"ACM Trans Inf Syst"},{"key":"9527_CR19","doi-asserted-by":"crossref","unstructured":"Wei X, Croft WB. LDA-based document models for ad-hoc retrieval. In: Proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval, 2006;178\u2013185:2006","DOI":"10.1145\/1148170.1148204"},{"key":"9527_CR20","doi-asserted-by":"crossref","unstructured":"Ganguly D, Roy D, Mitra M, Jones GJF. A word embedding based generalized language model for information retrieval. In: Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, 2015;795\u2013798","DOI":"10.1145\/2766462.2767780"},{"key":"9527_CR21","doi-asserted-by":"crossref","unstructured":"Griffiths TL, Steyvers M. Finding scientific topics. In: Proceedings of the National Academy of Sciences of the United States of America. 2004;101:5228\u201353.","DOI":"10.1073\/pnas.0307752101"},{"issue":"4598","key":"9527_CR22","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1126\/science.220.4598.671","volume":"220","author":"S Kirkpatrick","year":"1983","unstructured":"Kirkpatrick S, Gelatt CD Jr, Vecchi MP. Optimization by simulated annealing. Science. 1983;220(4598):671\u201380.","journal-title":"Science"},{"key":"9527_CR23","doi-asserted-by":"crossref","unstructured":"Amami M, Faiz R, Stella F, Pasi G. A graph based approach to scientific paper recommendation. In: Proceedings of the international conference on web intelligence, 2017;777\u2013782","DOI":"10.1145\/3106426.3106479"},{"issue":"4","key":"9527_CR24","doi-asserted-by":"publisher","first-page":"966","DOI":"10.1016\/j.ipm.2013.03.001","volume":"49","author":"J Parapar","year":"2013","unstructured":"Parapar J, Bellogn A, Castells P, Barreiro A. Relevance-based language modelling for recommender systems. Inf Process Manag. 2013;49(4):966\u201380.","journal-title":"Inf Process Manag"},{"key":"9527_CR25","doi-asserted-by":"crossref","unstructured":"Wang C, Blei DM. Collaborative topic modeling for recommending scientific articles. In: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, 2011;448\u2013456","DOI":"10.1145\/2020408.2020480"},{"key":"9527_CR26","doi-asserted-by":"publisher","first-page":"25303","DOI":"10.1109\/ACCESS.2023.3251189","volume":"11","author":"A Hadhiatma","year":"2023","unstructured":"Hadhiatma A, Azhari A, Suyanto Y. A scientific paper recommendation framework based on multi-topic communities and modified pagerank. IEEE Access. 2023;11:25303\u201317.","journal-title":"IEEE Access"},{"issue":"12","key":"9527_CR27","doi-asserted-by":"publisher","first-page":"15657","DOI":"10.1007\/s10639-023-11817-2","volume":"28","author":"B Smail","year":"2023","unstructured":"Smail B, Aliane H, Abdeldjalil O. Using an explicit query and a topic model for scientific article recommendation. Educ Inf Technol. 2023;28(12):15657\u201370.","journal-title":"Educ Inf Technol"},{"issue":"1","key":"9527_CR28","doi-asserted-by":"publisher","first-page":"123","DOI":"10.2298\/CSIS160229042W","volume":"14","author":"Y Wang","year":"2017","unstructured":"Wang Y, Choi I-C, Liu H. Generalized ensemble model for document ranking in information retrieval. Comput Sci Inf Syst. 2017;14(1):123\u201351.","journal-title":"Comput Sci Inf Syst"},{"key":"9527_CR29","doi-asserted-by":"crossref","unstructured":"Xu J, Li H. Adarank: a boosting algorithm for information retrieval. In: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, 2007;391\u2013398","DOI":"10.1145\/1277741.1277809"},{"key":"9527_CR30","doi-asserted-by":"publisher","first-page":"115","DOI":"10.7551\/mitpress\/1113.003.0010","volume-title":"Advances in large-margin classifiers","author":"R Herbrich","year":"2000","unstructured":"Herbrich R, Graepel T, Obermayer K. Large margin rank boundaries for ordinal regression. In: Bartlett PJ, Schlkopf B, Schuurmans D, Smola AJ, editors. Advances in large-margin classifiers. Cambridge: The MIT Press; 2000. p. 115\u201332."},{"key":"9527_CR31","unstructured":"Hoi SCH, Jin R. Semi-supervised ensemble ranking. In: Proceedings of the 23rd national conference on artificial intelligence, 2008;634\u2013639"},{"issue":"3","key":"9527_CR32","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1016\/j.ipm.2010.09.003","volume":"47","author":"S Chen","year":"2011","unstructured":"Chen S, Wang F, Song Y, Zhang C. Semi-supervised ranking aggregation. Inf Process Manage. 2011;47(3):415\u201325.","journal-title":"Inf Process Manage"},{"issue":"6","key":"9527_CR33","doi-asserted-by":"publisher","first-page":"1232","DOI":"10.1002\/asi.21296","volume":"61","author":"F Wei","year":"2010","unstructured":"Wei F, Li W, Liu S. irank: A rank-learn-combine framework for unsupervised ensemble ranking. Am Soc Inf Sci Technol. 2010;61(6):1232\u201343.","journal-title":"Am Soc Inf Sci Technol"},{"key":"9527_CR34","doi-asserted-by":"crossref","unstructured":"Meng C, Arabzadeh N, Askari A, Aliannejadi M, Rijke M. Ranked list truncation for large language model-based re-ranking. In: Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval, 2024;141-151","DOI":"10.1145\/3626772.3657864"},{"key":"9527_CR35","doi-asserted-by":"crossref","unstructured":"Sun W, Yan L, Ma X, Wang S, Ren P, Chen Z, Yin D, Ren Z. Is ChatGPT good at search? Investigating large language models as re-ranking agents. arXiv preprint arXiv:2304.09542 2023.","DOI":"10.18653\/v1\/2023.emnlp-main.923"},{"key":"9527_CR36","unstructured":"Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. In: 1st International conference on learning representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings. http:\/\/arxiv.org\/abs\/1301.3781 2013"},{"key":"9527_CR37","unstructured":"Kando N, Kuriyama K, Nozue T, Eguchi K, Kato H, Hidaka S, Adachi J. The NTCIR Workshop: the first evaluation workshop on japanese text retrieval and cross-lingual information retrieval. In: Proceedings of the 4th international workshop on information retrieval with Asian languages 1999."},{"key":"9527_CR38","unstructured":"Kando N. Overview of the second NTCIR workshop. In: Proceedings of the 2nd NTCIR workshop on research in Chinese and Japanese text retrieval and text summarization, 2001;35\u201343"},{"key":"9527_CR39","unstructured":"Kuriyama K, Yoshioka M, Kando N. The effect of cross-lingual pooling on evaluation. In: Proceedings of the 2nd NTCIR workshop on research in Chinese and Japanese text retrieval and text summarization, 2001;297\u2013310"},{"issue":"1","key":"9527_CR40","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1023\/A:1012778807438","volume":"5","author":"K Kuriyama","year":"2002","unstructured":"Kuriyama K, Kando N, Nozue T, Eguchi K. Pooling for a large-scale test collection: an analysis of the search results from the first ntcir workshop. Inf Retrieval. 2002;5(1):41\u201359.","journal-title":"Inf Retrieval"}],"container-title":["Discover Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10791-025-09527-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10791-025-09527-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10791-025-09527-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,16]],"date-time":"2025-04-16T18:28:35Z","timestamp":1744828115000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10791-025-09527-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,16]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["9527"],"URL":"https:\/\/doi.org\/10.1007\/s10791-025-09527-z","relation":{},"ISSN":["2948-2992"],"issn-type":[{"value":"2948-2992","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,16]]},"assertion":[{"value":"8 April 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 March 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 April 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study involved a single participant, Satoshi Fukuda, who was the first author of this paper. The experiment was conducted in accordance with ethical guidelines. As the sole participant, the first author provided informed consent for participation and data collection. No external participants were recruited, and no personally identifiable information was disclosed. Given the nature of the study, formal ethical approval was not required.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no conflict of interest.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"40"}}