{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,19]],"date-time":"2026-10-19T06:53:48Z","timestamp":1792392828019,"version":"4.4.0"},"reference-count":41,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"\n Background<\/jats:title>\n Literature reviews (LRs) identify, evaluate, and synthesize relevant papers to a particular research question to advance understanding and support decision-making. However, LRs, especially traditional systematic reviews, are slow, resource-intensive, and become outdated quickly.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n LiteRev is an advanced and enhanced version of an existing automation tool designed to assist researchers in conducting LRs through the implementation of cutting-edge technologies such as natural language processing and machine learning techniques. In this paper, we present a comprehensive explanation of LiteRev\u2019s capabilities, its methodology, and an evaluation of its accuracy and efficiency to a manual LR, highlighting the benefits of using LiteRev.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Based on the user\u2019s query, LiteRev performs an automated search on a wide range of open-access databases and retrieves relevant metadata on the resulting papers, including abstracts or full texts when available. These abstracts (or full texts) are text processed and represented as a term frequency-inverse document frequency matrix. Using dimensionality reduction (pairwise controlled manifold approximation) and clustering (hierarchical density-based spatial clustering of applications with noise) techniques, the corpus is divided into different topics described by a list of the most important keywords. The user can then select one or several topics of interest, enter additional keywords to refine its search, or provide key papers to the research question. Based on these inputs, LiteRev performs a k-nearest neighbor (k-NN) search and suggests a list of potentially interesting papers. By tagging the relevant ones, the user triggers new k-NN searches until no additional paper is suggested for screening. To assess the performance of LiteRev, we ran it in parallel to a manual LR on the burden and care for acute and early HIV infection in sub-Saharan Africa. We assessed the performance of LiteRev using true and false predictive values, recall, and work saved over sampling.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n LiteRev extracted, processed, and transformed text into a term frequency-inverse document frequency matrix of 631 unique papers from PubMed. The topic modeling module identified 16 topics and highlighted 2 topics of interest to the research question. Based on 18 key papers, the k-NNs module suggested 193 papers for screening out of 613 papers in total (31.5% of the whole corpus) and correctly identified 64 relevant papers out of the 87 papers found by the manual abstract screening (recall rate of 73.6%). Compared to the manual full text screening, LiteRev identified 42 relevant papers out of the 48 papers found manually (recall rate of 87.5%). This represents a total work saved over sampling of 56%.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n We presented the features and functionalities of LiteRev, an automation tool that uses natural language processing and machine learning methods to streamline and accelerate LRs and support researchers in getting quick and in-depth overviews on any topic of interest.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/39736","type":"journal-article","created":{"date-parts":[[2023,9,15]],"date-time":"2023-09-15T10:00:46Z","timestamp":1694772046000},"page":"e39736","source":"Crossref","is-referenced-by-count":22,"title":["An Automated Literature Review Tool (LiteRev) for Streamlining and Accelerating Research Using Natural Language Processing and Machine Learning: Descriptive Performance Evaluation Study"],"prefix":"10.2196","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4652-4750","authenticated-orcid":false,"given":"Erol","family":"Orel","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0406-5218","authenticated-orcid":false,"given":"Iza","family":"Ciglenecki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1544-415X","authenticated-orcid":false,"given":"Amaury","family":"Thiabaud","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5875-8066","authenticated-orcid":false,"given":"Alexander","family":"Temerev","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1137-6826","authenticated-orcid":false,"given":"Alexandra","family":"Calmy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8191-2789","authenticated-orcid":false,"given":"Olivia","family":"Keiser","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6418-2945","authenticated-orcid":false,"given":"Aziza","family":"Merzouki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2023,9,15]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1111\/hir.12276"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2021.106589"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.7326\/0003-4819-147-4-200708210-00179"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-020-0897-3"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1002\/jrsm.1237"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1186\/s13643-015-0066-7"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1145\/2745802.2745824"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2020.01.008"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.2196\/24418"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1186\/s13643-018-0740-7"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.2196\/18747"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1097\/QAI.0000000000002168"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1016\/S2352-3018(19)30226-7"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.7448\/IAS.20.1.21579"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1086\/429411"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.7448\/IAS.20.1.21708"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1177\/0956462414531244"},{"key":"ref18","unstructured":"BHIVA\/BASHH\/BIA adult HIV testing guidelines 2020BHIVA British HIV Association20202021-04-14https:\/\/www.bhiva.org\/HIV-testing-guidelines"},{"key":"ref19","unstructured":"BransonBMOwenSMWesolowskiLGBennettBWernerBGWroblewskiKEPentellaMAAssociation of Public Health LaboratorieLaboratory testing for the diagnosis of HIV infection: updated recommendationsCenters for Disease Control and Prevention20142021-04-14http:\/\/stacks.cdc.gov\/view\/cdc\/23447"},{"key":"ref20","unstructured":"Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection: recommendations for a public health approach, 2nd edWorld Health Organization2021-04-13http:\/\/www.who.int\/hiv\/pub\/arv\/arv-2016\/en\/"},{"key":"ref21","unstructured":"Consolidated guidelines on HIV testing services, 2019World Health Organization20202023-08-03https:\/\/www.who.int\/publications\/i\/item\/978-92-4-155058-1"},{"key":"ref22","unstructured":"RehurekRSojkaPGensim-Python framework for vector space modellingNLP Centre, Faculty of Informatics, Masaryk University, Brno, Czech Republic20112023-08-03https:\/\/scholar.google.com\/citations?view_op=view_citation&hl=th&user=9vG_kV0AAAAJ&citation_for_view=9vG_kV0AAAAJ:RGFaLdJalmkC"},{"key":"ref23","unstructured":"HonnibalMMontaniIspaCy 2: natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsingSentometrics Research20172023-08-03https:\/\/sentometrics-research.com\/publication\/72\/"},{"key":"ref24","first-page":"2825","volume":"12","author":"Pedregosa, F","year":"2011","journal-title":"J Mach Learn Res"},{"issue":"1","key":"ref25","first-page":"9129","volume":"22","author":"Wang, Y","year":"2021","journal-title":"J Mach Learn Res"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00205"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330701"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611973440.96"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.1016\/0377-0427(87)90125-7"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2019.09.013"},{"key":"ref31","author":"Peters, M","year":"2020","journal-title":"JBI Manual for Evidence Synthesis"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.1186\/s13643-016-0384-4"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2010.004325"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.1197\/jamia.M1929"},{"key":"ref35","unstructured":"MarshallCSuttonAO'KeefeHJohnsonEThe Systematic Review Toolbox20222023-08-03http:\/\/www.systematicreviewtools.com\/"},{"key":"ref36","unstructured":"Comprehensive guide to machine learning software for text screeningGitHub2023-08-03https:\/\/github.com\/Rensvandeschoot\/software-overview-machine-learning-for-screening-text"},{"key":"ref37","unstructured":"StoelLMouritsGvan de SchootRProcedure and results for the initial selection of software for systematically screening large amounts of textual data implementing active learningThe Open Science Platform2023-08-03https:\/\/osf.io\/g3nkz\/"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-020-00287-7"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9587-0"},{"key":"ref40","unstructured":"BrookerJSynnotAMcDonaldSElliottJTurnerTHodderRWeeksLRiedJMacLehoseHAklEKahaleLHiltonJFlemyngELassersonTThomasJSkoetzNClarkJFeatherstoneRNoel-StorrAMartinNMarshallRSimmondsMBaggMFusar-PoliPSinghAChouRGuidance for the production and publication of Cochrane living systematic reviews: Cochrane Reviews in living modeCochrane Collaboration20192023-08-03https:\/\/community.cochrane.org\/sites\/default\/files\/uploads\/inline-files\/Transform\/201912_LSR_Revised_Guidance.pdf"},{"key":"ref41","unstructured":"CigleneckiIKeiserOOrelECalmyABurden and care for acute and early HIV infection in sub-Saharan Africa: a scoping review protocolOSF Home2023-08-03https:\/\/osf.io\/c3qht\/"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2023\/1\/e39736","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:37:15Z","timestamp":1782841035000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2023\/1\/e39736"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,15]]},"references-count":41,"URL":"https:\/\/doi.org\/10.2196\/39736","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.39736","asserted-by":"object"}]},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,15]]}}}