{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T11:53:23Z","timestamp":1791978803551,"version":"4.4.0"},"reference-count":19,"publisher":"Wiley","issue":"5","license":[{"start":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T00:00:00Z","timestamp":1790553600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"},{"start":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T00:00:00Z","timestamp":1790553600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Cancer Innovation"],"published-print":{"date-parts":[[2026,10]]},"abstract":"ABSTRACT<\/jats:title>\n \n Background<\/jats:title>\n Systematic reviews (SRs), often conducted for evidence synthesis, require extensive multistage screening, with Stage\u2010I title\/abstract screening being especially time consuming. Covidence systematic review software, a widely used SR platform, has offered artificial intelligence (AI)\u2010assisted Stage\u2010I screening function since 2022. However, this function's performance has not yet been formally compared across heterogeneous, large\u2010scale oncology\u2010focused SRs.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n We selected two completed SRs conducted by the Program in Evidence\u2010based Care (PEBC), Ontario, Canada. The first SR (Axilla\u2010BC,\n N<\/jats:italic>\n \u2009=\u20098774) supported a clinical practice guideline in breast cancer and the second SR (PET\u2010utility,\n N<\/jats:italic>\n \u2009=\u20097267) informed the provincial Positron Emission Tomography (PET) steering committee in Ontario. Using each SR, we conducted a simulation\u2010based study by drawing subsets of 500, 1000, and 2000 articles and running 30, 30, and 10 simulation trials, respectively, to emulate Covidence's AI\u2010assisted Stage\u2010I screening. For each trial, we calculated the workload and time savings at 95% and 100% sensitivity for relevant articles and at 100% sensitivity for finally included articles as primary outcomes. Secondary outcomes comprised missed finally included articles at 95% sensitivity for relevant articles. Wilcoxon rank\u2010sum tests were used to compare outcomes between these two SRs.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n When pooling all subset sizes and the\n N<\/jats:italic>\n \u2009=\u20091000 subsets, workload savings at 95% sensitivity were significantly higher for Axilla\u2010BC than for PET\u2010utility (median = 37.4%\n vs.<\/jats:italic>\n 26.2%,\n W<\/jats:italic>\n \u2009=\u20093156.5,\n p<\/jats:italic>\n \u2009=\u20090.003 and median\u2009=\u200938.3%\n vs.<\/jats:italic>\n 20.5%,\n W<\/jats:italic>\n \u2009=\u2009641.0,\n p<\/jats:italic>\n \u2009=\u20090.005, respectively). Corresponding time savings were also significantly higher for Axilla\u2010BC (median\u2009=\u20096.0\n vs.<\/jats:italic>\n 4.2\u2009h,\n W<\/jats:italic>\n \u2009=\u20093089.5,\n p<\/jats:italic>\n \u2009=\u20090.008 and median\u2009=\u20099.6\n vs.<\/jats:italic>\n 5.1\u2009h,\n W<\/jats:italic>\n \u2009=\u2009639.0,\n p<\/jats:italic>\n \u2009=\u20090.005). In Axilla\u2010BC, 20% of the 70 trials missed a total of 18 finally included articles, but only one PET\u2010utility trial missed one article.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n In two large oncology\u2010SRs, Covidence's AI\u2010assisted Stage\u2010I screening function demonstrated meaningful potential to reduce reviewer workload and screening time. Nonetheless, Covidence's performance varied notably between reviews, suggesting a possible association between the results and the characteristics of the SR, including topic, methodological complexity, relevance rate, and inclusion rate. Careful human oversight remains essential to ensure that no potentially finally included studies are missed in evidence synthesis, particularly in nuanced or methodologically complex reviews.<\/jats:p>\n <\/jats:sec>","DOI":"10.1002\/cai2.70077","type":"journal-article","created":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T04:55:46Z","timestamp":1790657746000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Evaluating the Performance of Covidence's Artificial Intelligence\u2010Based Screening Function in Cancer Systematic Reviews"],"prefix":"10.1002","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7650-1720","authenticated-orcid":false,"given":"Ashirbani","family":"Saha","sequence":"first","affiliation":[{"name":"Department of Oncology McMaster University Hamilton Canada"},{"name":"Escarpment Cancer Research Institute (ECRI) McMaster University and Hamilton Health Sciences Hamilton Canada"},{"name":"CentRE for dAta science and digiTal hEalth (CREATE) Hamilton Health Sciences Hamilton Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1210-3758","authenticated-orcid":false,"given":"Xiaomei","family":"Yao","sequence":"additional","affiliation":[{"name":"Department of Oncology McMaster University Hamilton Canada"},{"name":"Department of Health Research Methods, Evidence, and Impact McMaster University Hamilton Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sharan","family":"Saravanan","sequence":"additional","affiliation":[{"name":"Department of Health Research Methods, Evidence, and Impact McMaster University Hamilton Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aditya","family":"Misra","sequence":"additional","affiliation":[{"name":"Faculty of Health Sciences McMaster University Hamilton Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ashley","family":"Low","sequence":"additional","affiliation":[{"name":"Department of Health Research Methods, Evidence, and Impact McMaster University Hamilton Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samya","family":"Ali","sequence":"additional","affiliation":[{"name":"Faculty of Health Sciences McMaster University Hamilton Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mariam","family":"Abdelmalek","sequence":"additional","affiliation":[{"name":"Faculty of Health Sciences McMaster University Hamilton Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shakil","family":"Ahmed","sequence":"additional","affiliation":[{"name":"Department of Health Research Methods, Evidence, and Impact McMaster University Hamilton Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sussman","family":"Jonathan","sequence":"additional","affiliation":[{"name":"Department of Oncology McMaster University Hamilton Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,9,28]]},"reference":[{"key":"e_1_2_14_2_1","unstructured":"\u201cEvidence Synthesis | Royal Society.\u201d accessed March 2 2026 https:\/\/royalsociety.org\/news-resources\/projects\/evidence-synthesis\/."},{"key":"e_1_2_14_3_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2016-012545"},{"key":"e_1_2_14_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/cai2.70021"},{"key":"e_1_2_14_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2022.02.015"},{"key":"e_1_2_14_6_1","doi-asserted-by":"publisher","DOI":"10.1111\/jep.70100"},{"key":"e_1_2_14_7_1","unstructured":"\u201cPosition Statement on Artificial Intelligence (AI) use in Evidence Synthesis across Cochrane the Campbell Collaboration JBI and the Collaboration for Environmental Evidence 2025 | Cochrane Library.\u201d accessed March 2 2026 https:\/\/link.springer.com\/article\/10.1186\/s13750-025-00374-5."},{"key":"e_1_2_14_8_1","doi-asserted-by":"publisher","DOI":"10.1002\/cesm.70046"},{"key":"e_1_2_14_9_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-020-0897-3"},{"key":"e_1_2_14_10_1","unstructured":"\u201cMachine learning Product Update: December 2022.\u201d accessed March 2 2026 https:\/\/www.covidence.org\/blog\/release-notes-december-2022-machine-learning\/."},{"key":"e_1_2_14_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.imed.2025.12.008"},{"key":"e_1_2_14_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.canep.2023.102511"},{"key":"e_1_2_14_13_1","unstructured":"\u201cProgram in Evidence\u2010Based Care | Cancer Care Ontario.\u201d accessed March 2 2026 https:\/\/www.cancercareontario.ca\/en\/cancer-care-ontario\/programs\/data-research\/evidence-based-care."},{"key":"e_1_2_14_14_1","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.21.00934"},{"key":"e_1_2_14_15_1","unstructured":"F. G.Baldassarre and The Program in Evidence\u2010Based Care Disease Site Group Reviewers. \u201cEvidence from Primary Studies and Systematic Reviews and Recommendations from Clinical Practice Guidelines January to June 2024 \u201d May 2025. accessed March 19 2026 https:\/\/www.cancercareontario.ca\/en\/guidelines-advice\/types-of-cancer\/78226."},{"key":"e_1_2_14_16_1","doi-asserted-by":"publisher","DOI":"10.1101\/2021.09.30.462652"},{"key":"e_1_2_14_17_1","unstructured":"Posit Team \u201cRStudio: Integrated Development Environment for R \u201d 2025 Posit Software PBC Boston MA. Accessed March 19 2026 http:\/\/www.posit.co\/."},{"key":"e_1_2_14_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/jrsm.1354"},{"key":"e_1_2_14_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jamda.2024.105040"},{"key":"e_1_2_14_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.imed.2025.12.006"}],"container-title":["Cancer Innovation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cai2.70077","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/cai2.70077","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cai2.70077","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T11:38:16Z","timestamp":1791977896000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cai2.70077"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,28]]},"references-count":19,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,10]]}},"alternative-id":["10.1002\/cai2.70077"],"URL":"https:\/\/doi.org\/10.1002\/cai2.70077","archive":["Portico"],"relation":{},"ISSN":["2770-9191","2770-9183"],"issn-type":[{"value":"2770-9191","type":"print"},{"value":"2770-9183","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,28]]},"assertion":[{"value":"2026-04-09","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-08-03","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70077"}}