{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T00:29:57Z","timestamp":1777854597975,"version":"3.51.4"},"reference-count":82,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2023,6,26]],"date-time":"2023-06-26T00:00:00Z","timestamp":1687737600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2023,6,26]],"date-time":"2023-06-26T00:00:00Z","timestamp":1687737600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["PD\/BD\/2020.04698.BD."],"award-info":[{"award-number":["PD\/BD\/2020.04698.BD."]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100018319","name":"computer science and artificial intelligence laboratory, massachusetts institute of technology","doi-asserted-by":"publisher","award":["UIDB\/00027\/2020"],"award-info":[{"award-number":["UIDB\/00027\/2020"]}],"id":[{"id":"10.13039\/100018319","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Information Science"],"published-print":{"date-parts":[[2026,2]]},"abstract":"<jats:p>The assignment of appropriate reviewers to academic articles, known as the reviewer assignment problem (RAP), has become a crucial issue in academia. While there has been much research on RAP, there has not yet been a systematic literature review (SLR) examining the various approaches, techniques, algorithms and discoveries related to this topic. To conduct the SLR, we identified and evaluated relevant articles from four databases using defined inclusion and exclusion criteria. We analysed the selected articles and extracted information, and assessed their quality. Our review identified 67 articles on RAP published in conferences and journals up to mid-2022. As one of the main challenges in RAP is acquiring open data, we have studied the data sources used by researchers and found that most studies use real data from conferences, bibliographic databases and online academic search engines. RAP is divided into two main phases: (1) finding\/recommending expert reviewers and (2) assigning reviewers to submitted manuscripts. In Phase 1, we have identified that decision support systems, recommendation systems, and machine learning-oriented approaches are more commonly used due to better results. In Phase 2, heuristics and metaheuristics are the approaches that present better results and are consequently more commonly used by researchers. Based on the analysed studies, we have identified potential areas for future research that could lead to improved results. Specifically, we suggest exploring the application of deep neural networks for calculating the degree of correspondence and using the Boolean satisfiability problem to optimise the attribution process.<\/jats:p>","DOI":"10.1177\/01655515231176668","type":"journal-article","created":{"date-parts":[[2023,6,26]],"date-time":"2023-06-26T07:35:36Z","timestamp":1687764936000},"page":"39-59","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":5,"title":["Investigating the reviewer assignment problem: A systematic literature review"],"prefix":"10.1177","volume":"52","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1557-654X","authenticated-orcid":false,"given":"Ana Carolina","family":"Ribeiro","sequence":"first","affiliation":[{"name":"Department of Informatics Engineering (DEI), Faculty of Engineering of the University of Porto (FEUP), University of Porto, Portugal; Artificial Intelligence and Computer Science Laboratory (LIACC), University of Porto, Portugal"}]},{"given":"Amanda","family":"Sizo","sequence":"additional","affiliation":[{"name":"Department of Informatics Engineering (DEI), Faculty of Engineering of the University of Porto (FEUP), University of Porto, Portugal; Artificial Intelligence and Computer Science Laboratory (LIACC), University of Porto, Portugal"}]},{"given":"Lu\u00eds Paulo","family":"Reis","sequence":"additional","affiliation":[{"name":"Department of Informatics Engineering (DEI), Faculty of Engineering of the University of Porto (FEUP), University of Porto, Portugal; Artificial Intelligence and Computer Science Laboratory (LIACC), University of Porto, Portugal"}]}],"member":"179","published-online":{"date-parts":[[2023,6,26]]},"reference":[{"key":"e_1_3_5_2_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1540-5915.1999.tb00910.x"},{"key":"e_1_3_5_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-020-03519-0"},{"key":"e_1_3_5_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2020.114331"},{"key":"e_1_3_5_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2007.17"},{"key":"e_1_3_5_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-013-0445-5"},{"key":"e_1_3_5_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cor.2012.08.005"},{"key":"e_1_3_5_8_2","unstructured":"Goldsmith J Sloan RH. The AI conference paper assignment problem. In: AAAI workshop \u2013 technical report 2007 pp. 53\u201357. https:\/\/www.cs.uic.edu\/~sloan\/my-papers\/GodlsmithSloanPaperAssignment.pdf"},{"key":"e_1_3_5_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458127"},{"key":"e_1_3_5_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cor.2003.11.003"},{"key":"e_1_3_5_11_2","doi-asserted-by":"publisher","DOI":"10.1002\/aris.2011.1440450112"},{"key":"e_1_3_5_12_2","unstructured":"Fletcher RH Fletcher SW. The effectiveness of journal peer review. 2003. https:\/\/www.bmj.com\/sites\/default\/files\/attachments\/resources\/2011\/07\/fletcher.pdf"},{"key":"e_1_3_5_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compind.2015.11.001"},{"key":"e_1_3_5_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/SMC.2014.6974397"},{"key":"e_1_3_5_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3407982.3408026"},{"key":"e_1_3_5_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2011.09.004"},{"key":"e_1_3_5_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.joi.2020.101022"},{"key":"e_1_3_5_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2019.01.035"},{"key":"e_1_3_5_19_2","doi-asserted-by":"crossref","unstructured":"Dumais ST Nielsen J. Automating the assignment of submitted manuscripts to reviewer. In: SIGIR Forum (ACM Special Interest Group on Information Retrieval) 1992 pp. 233\u2013244. https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/133160.133205","DOI":"10.1145\/133160.133205"},{"key":"e_1_3_5_20_2","first-page":"718","volume-title":"Lecture notes in artificial intelligence","author":"Wang F","year":"2008","unstructured":"Wang F, Shi N, Chen B. A survey on reviewer assignment problem. In: Nguyen NT, Borzemski L, Grzech A, et al. (eds) Lecture notes in artificial intelligence. Berlin: Springer-Verlag, 2008, pp. 718\u2013727."},{"key":"e_1_3_5_21_2","doi-asserted-by":"publisher","DOI":"10.1142\/S0219622010003993"},{"key":"e_1_3_5_22_2","doi-asserted-by":"publisher","DOI":"10.4103\/0256-4602.78092"},{"key":"e_1_3_5_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/IoT-SIU.2019.8777727"},{"key":"e_1_3_5_24_2","article-title":"PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews","volume":"372","author":"Page MJ","year":"2021","unstructured":"Page MJ, Moher D, Bossuyt PM, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 2021; 372: n160.","journal-title":"BMJ"},{"key":"e_1_3_5_25_2","article-title":"The PRISMA 2020 statement: an updated guideline for reporting systematic reviews","volume":"372","author":"Page MJ","year":"2021","unstructured":"Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372: n71.","journal-title":"BMJ"},{"key":"e_1_3_5_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-020-03690-4"},{"key":"e_1_3_5_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbiomech.2017.08.025"},{"key":"e_1_3_5_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2894680"},{"key":"e_1_3_5_29_2","first-page":"25","volume-title":"RecSys 2014 \u2013 Proceedings of the 8th ACM conference on recommender systems 2014","author":"Liu X","unstructured":"Liu X, Suel T, Memon N. A robust model for paper-reviewer assignment categories and subject descriptors. In: RecSys 2014 \u2013 Proceedings of the 8th ACM conference on recommender systems 2014, San Jose, CA, 6\u201310 October 2014, pp. 25\u201332. New York: ACM."},{"key":"e_1_3_5_30_2","doi-asserted-by":"publisher","DOI":"10.36478\/jeasci.2019.3378.3382"},{"key":"e_1_3_5_31_2","first-page":"86","volume-title":"Proceedings of the 27th conference on uncertainty in artificial intelligence, UAI 2011","author":"Charlin L","unstructured":"Charlin L, Zemel R, Boutilier C. A framework for optimizing paper matching. In: Proceedings of the 27th conference on uncertainty in artificial intelligence, UAI 2011, virtual event, 27\u201330 July 2011, pp. 86\u201395. New York: ACM."},{"key":"e_1_3_5_32_2","unstructured":"Charlin L Zemel RS. The Toronto Paper Matching System: an automated paper-reviewer assignment system. In: Proceedings of the 30th international conference on machine learning 2013. http:\/\/cmt.research.microsoft.com\/cmt\/"},{"key":"e_1_3_5_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2020.106483"},{"key":"e_1_3_5_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2019.01.007"},{"key":"e_1_3_5_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2723727"},{"key":"e_1_3_5_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2013.13"},{"key":"e_1_3_5_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2011.04.039"},{"key":"e_1_3_5_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/1639714.1639787"},{"key":"e_1_3_5_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-63579-8_28"},{"key":"e_1_3_5_40_2","doi-asserted-by":"publisher","DOI":"10.1080\/01969722.2020.1871227"},{"key":"e_1_3_5_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2017.09.020"},{"key":"e_1_3_5_42_2","doi-asserted-by":"crossref","unstructured":"Maleszka M Maleszka B Kr\u00f3l D et al. A Modular diversity based reviewer recommendation system. In: Intelligent information and database systems 2020 pp. 550\u2013561. http:\/\/www.springerlink.com\/content\/38p748j5q2475127\/","DOI":"10.1007\/978-981-15-3380-8_48"},{"key":"e_1_3_5_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/TETC.2018.2861214"},{"key":"e_1_3_5_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2013.09.144"},{"key":"e_1_3_5_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/11779568_78"},{"key":"e_1_3_5_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAAI.2016.7932314"},{"key":"e_1_3_5_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3487664.3487673"},{"key":"e_1_3_5_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3361719"},{"key":"e_1_3_5_49_2","first-page":"3","volume-title":"Proceedings \u2013 9th international workshop on semantic and social media adaptation and personalization, SMAP2014","author":"Saloun P","unstructured":"Saloun P, Ondrejka A, Zelinka I. Similarity of authors\u2019 profiles and its usage for reviewers\u2019 recommendation. In: Proceedings \u2013 9th international workshop on semantic and social media adaptation and personalization, SMAP2014, Corfu, 6\u20137 November 2014, pp. 3\u20138. New York: IEEE."},{"key":"e_1_3_5_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3053053"},{"key":"e_1_3_5_51_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2019.06.074"},{"issue":"6","key":"e_1_3_5_52_2","article-title":"Context-aware reviewer assignment for trust enhanced peer review","volume":"10","author":"Li L","year":"2015","unstructured":"Li L, Wang Y, Liu G, et al. Context-aware reviewer assignment for trust enhanced peer review. PLoS ONE 2015; 10(6): e0130493.","journal-title":"PLoS ONE"},{"key":"e_1_3_5_53_2","doi-asserted-by":"publisher","DOI":"10.4028\/www.scientific.net\/AMR.452-453.369"},{"key":"e_1_3_5_54_2","doi-asserted-by":"crossref","unstructured":"Kolasa T Kr\u00f3l D (2010) ACO-GA Approach to Paper-Reviewer Assignment Problem in CMS. In: J\u0229drzejowicz P. Nguyen N.T. Howlet R.J. Jain L.C. (eds) Agent and Multi-Agent Systems: Technologies and Applications. KES-AMSTA 2010. Lecture Notes in Computer Science() vol 6071. Springer Berlin Heidelberg. https:\/\/doi.org\/10.1007\/978-3-642-13541-5_37","DOI":"10.1007\/978-3-642-13541-5_37"},{"key":"e_1_3_5_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCAA.2018.8777730"},{"key":"e_1_3_5_56_2","first-page":"4","volume-title":"2020 international conference on intelligent engineering and management (ICIEM)","author":"Mittal K","unstructured":"Mittal K, Jain A, Singh Vaisla K. A novel method for reviewer assignment problem based on reviewers\u2019 profile and fuzzy graph connectivity measure. In: 2020 international conference on intelligent engineering and management (ICIEM), London, 17\u201319 June 2020, pp. 4\u20139. New York: IEEE."},{"key":"e_1_3_5_57_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2007.11.011"},{"key":"e_1_3_5_58_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2016.05.041"},{"key":"e_1_3_5_59_2","doi-asserted-by":"publisher","DOI":"10.1177\/0165551518806116"},{"key":"e_1_3_5_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/2629436"},{"key":"e_1_3_5_61_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11484"},{"key":"e_1_3_5_62_2","unstructured":"Jecmen S Shah NB Zhang H et al. Mitigating manipulation in peer review via randomized reviewer assignments. arXiv 2020. https:\/\/arxiv.org\/abs\/2006.16437"},{"key":"e_1_3_5_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/SMC52423.2021.9658668"},{"key":"e_1_3_5_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICICT.2007.375347"},{"key":"e_1_3_5_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/IALP.2018.8629124"},{"key":"e_1_3_5_66_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-55705-2_11"},{"key":"e_1_3_5_67_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41550-020-1038-y"},{"key":"e_1_3_5_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458230"},{"key":"e_1_3_5_69_2","doi-asserted-by":"publisher","DOI":"10.1007\/11504894_109"},{"key":"e_1_3_5_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.2016.7579022"},{"key":"e_1_3_5_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/1281192.1281247"},{"key":"e_1_3_5_72_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1049"},{"key":"e_1_3_5_73_2","first-page":"383","article-title":"PRATO: an automated taxonomy-based reviewer-proposal assignment system","volume":"13","author":"Warade S","year":"2018","unstructured":"Warade S, Aghav J, Petitpierre C, et al. PRATO: an automated taxonomy-based reviewer-proposal assignment system. Interdiscip J Inf Knowl Manag 2018; 13: 383\u2013396.","journal-title":"Interdiscip J Inf Knowl Manag"},{"key":"e_1_3_5_74_2","first-page":"113","article-title":"Reviewer assignment for scientific articles using memetic algorithms","volume":"39","author":"Schirrer A","year":"2007","unstructured":"Schirrer A, Doerner KF, Hartl RF. Reviewer assignment for scientific articles using memetic algorithms. Oper Res Comput Sci Interfaces Ser 2007; 39: 113\u2013134.","journal-title":"Oper Res Comput Sci Interfaces Ser"},{"key":"e_1_3_5_75_2","doi-asserted-by":"crossref","unstructured":"Merelo-Guerv\u00f3s JJ Castillo-Valdivieso P (2004) Conference Paper Assignment Using a Combined Greedy\/Evolutionary Algorithm. In: et al. Parallel Problem Solving from Nature - PPSN VIII. PPSN 2004. Lecture Notes in Computer Science vol 3242. Springer Berlin Heidelberg. https:\/\/doi.org\/10.1007\/978-3-540-30217-9_61","DOI":"10.1007\/978-3-540-30217-9_61"},{"key":"e_1_3_5_76_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cie.2014.02.014"},{"key":"e_1_3_5_77_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49130-1_13"},{"key":"e_1_3_5_78_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-013-1187-1"},{"key":"e_1_3_5_79_2","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.1040.0290"},{"key":"e_1_3_5_80_2","unstructured":"Taylor CJ On the optimal assignment of conference papers to reviewers. Proofs 2006; vol 1 1\u20134."},{"key":"e_1_3_5_81_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10479-018-2919-7"},{"key":"e_1_3_5_82_2","volume-title":"Decision support system for business intelligence","author":"Sauter VL.","year":"2010","unstructured":"Sauter VL. Decision support system for business intelligence. Hoboken, NJ: John Wiley & Sons, 2010."},{"key":"e_1_3_5_83_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCNC.2018.8390343"}],"container-title":["Journal of Information Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/01655515231176668","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/01655515231176668","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/01655515231176668","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T23:10:10Z","timestamp":1777504210000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/01655515231176668"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,26]]},"references-count":82,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,2]]}},"alternative-id":["10.1177\/01655515231176668"],"URL":"https:\/\/doi.org\/10.1177\/01655515231176668","relation":{},"ISSN":["0165-5515","1741-6485"],"issn-type":[{"value":"0165-5515","type":"print"},{"value":"1741-6485","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,26]]}}}