{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T16:55:58Z","timestamp":1776358558727,"version":"3.51.2"},"update-to":[{"DOI":"10.12688\/hrbopenres.14129.1","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T00:00:00Z","timestamp":1776297600000}},{"DOI":"10.12688\/hrbopenres.14129.2","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T00:00:00Z","timestamp":1776297600000}}],"reference-count":41,"publisher":"F1000 Research Ltd","license":[{"start":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T00:00:00Z","timestamp":1776297600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010414","name":"Health Research Board","doi-asserted-by":"publisher","award":["ESI-2021-001"],"award-info":[{"award-number":["ESI-2021-001"]}],"id":[{"id":"10.13039\/100010414","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["hrbopenresearch.org"],"crossmark-restriction":false},"short-container-title":["HRB Open Res"],"abstract":"<ns7:p>\n                    Background Systematic reviews can be resource-intensive and require timely completion, yet experienced reviewers may have limited availability, necessitating the inclusion of novice screeners. This Study Within A Review (SWAR) will pilot and examine the feasibility of study methods to explore whether training and level of experience within the screening pair affects the reliability of decisions made by novice screeners during study selection. Methods A 2(training: task-specific, minimal guidance) x 2(experience level of screening partner, \u2018Reviewer 1\u2019: moderate experience, minimal experience) pilot randomised trial will be conducted within a host systematic review in the topic area of depression and psychosocial functioning. Participants (\n                    <ns7:italic>N<\/ns7:italic>\n                    \u00a0=\u00a012), consisting of higher education students with no prior experience in evidence synthesis, will be randomised to one of the four conditions to complete a standardised study selection task at title\/abstract level (\n                    <ns7:italic>k<\/ns7:italic>\n                    \u00a0=\u00a0219 records) on Covidence screening software, blindly and independently. Total participation time is estimated at 5\u00a0hours. Screening decisions made by participants will be assessed for reliability against the consensus-based decisions by two reviewers with content and methodological expertise (expert standard), through calculation of chance-corrected Cohen\u2019s kappa, prevalence-adjusted bias-adjusted kappa (PABAK), and percentage of agreement, then compared across the conditions. Secondary outcomes will include reliability within the screening pair (participant and allocated screening partner), validity of screening decisions (false positives, false negatives, sensitivity, specificity), feasibility measures, including time taken to complete the study selection task and success of blinding, as well as acceptability. Conclusions Findings will be used to inform the design of subsequent trial work to determine the efficacy of training and screener pairing for study selection. Ultimately, these insights will help to build capacity among novice screeners to engage with evidence synthesis and work alongside experienced review teams. Registration Northern Ireland Hub for Trials Methodology Research SWAR Registry: SWAR 38.\n                  <\/ns7:p>","DOI":"10.12688\/hrbopenres.14129.3","type":"journal-article","created":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T15:59:08Z","timestamp":1776355148000},"page":"54","update-policy":"https:\/\/doi.org\/10.12688\/hrbopenres.crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Training and Experience in Study Selection (TESS): study protocol for a pilot randomised trial within a systematic review"],"prefix":"10.12688","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9230-6776","authenticated-orcid":false,"given":"Elayne","family":"Ahern","sequence":"first","affiliation":[{"name":"Department of Psychology Castletroy Limerick V94 T9PX University of Limerick"},{"name":"Health Research Institute Castletroy Limerick V94 T9PX University of Limerick"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-8630-8066","authenticated-orcid":false,"given":"Temitayo","family":"Adedeji","sequence":"additional","affiliation":[{"name":"Department of Psychology Castletroy Limerick V94 T9PX University of Limerick"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0300-5910","authenticated-orcid":false,"given":"Aoife","family":"Whiston","sequence":"additional","affiliation":[{"name":"Department of Psychology Castletroy Limerick V94 T9PX University of Limerick"},{"name":"Health Research Institute Castletroy Limerick V94 T9PX University of Limerick"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6659-2606","authenticated-orcid":false,"given":"Sarah","family":"Dillon","sequence":"additional","affiliation":[{"name":"Health Research Institute Castletroy Limerick V94 T9PX University of Limerick"},{"name":"School of Allied Health Castletroy Limerick V94 T9PX University of Limerick"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0216-643X","authenticated-orcid":false,"given":"Fiona","family":"Lynn","sequence":"additional","affiliation":[{"name":"School of Nursing and Midwifery Belfast BT9 7BL Queen's University Belfast"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"2560","published-online":{"date-parts":[[2026,4,16]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"1-11","DOI":"10.1177\/2158244020939530","article-title":"Assessing quality in systematic literature reviews: a study of novice rater training.","volume":"10","author":"S Acosta","year":"2020","journal-title":"Sage Open."},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.17605\/OSF.IO\/WKHQJ","article-title":"Training and Experience in Study Selection - The TESS study. [Open Science Framework Project].","author":"E Ahern","year":"2025"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"e012545","DOI":"10.1136\/bmjopen-2016-012545","article-title":"Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry.","volume":"7","author":"R Borah","year":"2017","journal-title":"BMJ Open."},{"key":"ref4","doi-asserted-by":"publisher","first-page":"423-429","DOI":"10.1016\/0895-4356(93)90018-v","article-title":"Bias, prevalence and kappa.","volume":"46","author":"T Byrt","year":"1993","journal-title":"J Clin Epidemiol."},{"key":"ref5","article-title":"boot: bootstrap R (S-Plus) functions. R package version 1.3\u201331.","author":"A Canty","year":"2024"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"200-207","DOI":"10.7326\/0003-4819-158-3-201302050-00583","article-title":"SPIRIT 2013 statement: defining standard protocol items for clinical trials.","volume":"158","author":"A Chan","year":"2013","journal-title":"Ann Intern Med."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"37-46","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales.","volume":"20","author":"J Cohen","year":"1960","journal-title":"Educ Psychol Meas."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"494-500","DOI":"10.1079\/phn2005877","article-title":"An assessment of inter-rater agreement of the literature filtering process in the development of evidence-based dietary guidelines.","volume":"9","author":"M Cooper","year":"2006","journal-title":"Public Health Nutr."},{"key":"ref9","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1186\/s13643-017-0441-7","article-title":"Effect of standardized training on the reliability of the Cochrane risk of bias assessment tool: a prospective study.","volume":"6","author":"B Costa","year":"2017","journal-title":"Syst Rev."},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511802843","author":"A Davison","year":"1997","journal-title":"Bootstrap methods and their application."},{"key":"ref11","doi-asserted-by":"publisher","first-page":"328-332","DOI":"10.1111\/jebm.12505","article-title":"Study Within A Review (SWAR).","volume":"15","author":"D Devane","year":"2022","journal-title":"J Evid Based Med."},{"key":"ref12","doi-asserted-by":"publisher","first-page":"1635-1640","DOI":"10.1002\/sim.1190","article-title":"Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records.","volume":"21","author":"P Edwards","year":"2002","journal-title":"Stat Med."},{"key":"ref13","doi-asserted-by":"publisher","first-page":"i5239","DOI":"10.1136\/bmj.i5239","article-title":"CONSORT 2010 statement: extension to randomised pilot and feasibility trials.","volume":"355","author":"S Eldridge","year":"2016","journal-title":"BMJ."},{"key":"ref14","doi-asserted-by":"publisher","first-page":"e22","DOI":"10.1371\/journal.pctr.0020022","article-title":"Development and evaluation of a pedagogical tool to improve understanding of a quality checklist: a randomised controlled trial.","volume":"2","author":"L Fourcade","year":"2007","journal-title":"PLoS Clin Trials."},{"key":"ref15","doi-asserted-by":"publisher","first-page":"13-22","DOI":"10.1016\/j.jclinepi.2020.10.007","article-title":"Cochrane rapid reviews methods group offers evidence-informed guidance to conduct rapid reviews.","volume":"130","author":"C Garritty","year":"2021","journal-title":"J Clin Epidemiol."},{"key":"ref16","doi-asserted-by":"publisher","first-page":"20-28","DOI":"10.1016\/j.jclinepi.2020.01.005","article-title":"Single-reviewer abstract screening missed 13 percent of relevant studies: a crowd-based, randomized controlled trial.","volume":"121","author":"G Gartlehner","year":"2020","journal-title":"J Clin Epidemiol."},{"key":"ref17","doi-asserted-by":"publisher","first-page":"179-183","DOI":"10.1007\/s40362-014-0023-1","article-title":"The policy role of systematic reviews: past, present and future.","volume":"2","author":"N Haddaway","year":"2014","journal-title":"Springer Sci Rev."},{"key":"ref18","doi-asserted-by":"publisher","first-page":"1-11","DOI":"10.1016\/j.jclinepi.2021.05.022","article-title":"Nearly 80 systematic reviews were published each day: observational study on trends in epidemiology and reporting over the years 2000\u20132019.","volume":"138","author":"F Hoffmann","year":"2021","journal-title":"J Clin Epidemiol."},{"key":"ref19","doi-asserted-by":"publisher","first-page":"289-298","DOI":"10.1016\/j.jclinepi.2009.04.007","article-title":"Systematic review data extraction: cross-sectional study showed that experience did not increase accuracy.","volume":"63","author":"J Horton","year":"2010","journal-title":"J Clin Epidemiol."},{"key":"ref21","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1186\/1471-2288-14-41","article-title":"The statistical interpretation of pilot trials: should significance thresholds be reconsidered?","volume":"14","author":"E Lee","year":"2014","journal-title":"BMC Med Res Methodol."},{"key":"ref23","doi-asserted-by":"publisher","first-page":"626-629","DOI":"10.1016\/j.jpsychires.2010.10.008","article-title":"The role and interpretation of pilot studies in clinical research.","volume":"45","author":"A Leon","year":"2011","journal-title":"J Psychiatr Res."},{"key":"ref24","doi-asserted-by":"publisher","first-page":"89-95","DOI":"10.2147\/CLEP.S43118 3604876","article-title":"Titles versus titles and abstracts for initial screening of articles for systematic reviews.","volume":"5","author":"F Mateen","year":"2013","journal-title":"Clin Epidemiol."},{"key":"ref25","author":"M McDonagh","year":"2013","journal-title":"Methods Guide for Comparative Effectiveness Reviews."},{"key":"ref26","doi-asserted-by":"publisher","first-page":"1-5","DOI":"10.1207\/s15326985ep2001_1","article-title":"Methodological quality as a component of meta-analysis.","volume":"20","author":"J McGuire","year":"1985","journal-title":"Educ Psychol."},{"key":"ref27","doi-asserted-by":"publisher","first-page":"276-282","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: the kappa statistic.","volume":"22","author":"M McHugh","year":"2012","journal-title":"Biochem Med (Zagreb)."},{"key":"ref28","doi-asserted-by":"publisher","first-page":"515-519","DOI":"10.1093\/eurjcn\/zvab041","article-title":"Rapid reviews: the pros and cons of an accelerated review process.","volume":"20","author":"P Moons","year":"2021","journal-title":"Eur J Cardiovasc Nurs."},{"key":"ref29","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1186\/2046-4053-3-121","article-title":"Title and Abstract Screening and Evaluation in Systematic Reviews (TASER): a pilot randomised controlled trial of title and abstract screening by medical students.","volume":"3","author":"L Ng","year":"2014","journal-title":"Syst Rev."},{"key":"ref30","doi-asserted-by":"publisher","first-page":"130-139","DOI":"10.1016\/j.jclinepi.2021.01.006","article-title":"An evaluation of Cochrane Crowd found that crowdsourcing produced accurate results in identifying randomized trials.","volume":"133","author":"A Noel-Storr","year":"2021","journal-title":"J Clin Epidemiol."},{"key":"ref31","doi-asserted-by":"publisher","first-page":"418-423","DOI":"10.1136\/bmjebm-2022-112185","article-title":"Rapid reviews methods series: guidance on team considerations, study selection, data extraction and risk of bias assessment.","volume":"28","author":"B Nussbaumer-Streit","year":"2023","journal-title":"BMJ Evid Based Med."},{"key":"ref32","doi-asserted-by":"publisher","first-page":"e001368","DOI":"10.1136\/bmjopen-2012-001368","article-title":"Inter-rater and test\u2013retest reliability of quality assessments by novice student raters using the Jadad and Newcastle\u2013Ottawa Scales.","volume":"2","author":"M Oremus","year":"2012","journal-title":"BMJ Open."},{"key":"ref34","doi-asserted-by":"publisher","first-page":"330-342","DOI":"10.1002\/jrsm.1354 6771536","article-title":"Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses.","volume":"10","author":"J Polanin","year":"2019","journal-title":"Res Synth Methods"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"121-135","DOI":"10.1016\/j.jclinepi.2018.10.003","article-title":"Few studies exist examining methods for selecting studies, abstracting data, and appraising quality in a systematic review.","volume":"106","author":"R Robson","year":"2019","journal-title":"J Clin Epidemiol."},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.32614\/CRAN.package.kappaSize","article-title":"kappaSize: sample size estimation functions for studies of interobserver agreement. R package version 1.2.","author":"M Rotondi","year":"2018"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"747-770","DOI":"10.1146\/annurev-psych-010418-102803","article-title":"How to do a systematic review: a best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses.","volume":"70","author":"A Siddaway","year":"2019","journal-title":"Annu Rev Psychol."},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.32614\/CRAN.package.epiR","article-title":"epiR: tools for the analysis of epidemiological data. R package version 2.0.89.","author":"M Stevenson","year":"2025"},{"key":"ref39","doi-asserted-by":"publisher","first-page":"539-545","DOI":"10.1002\/jrsm.1369","article-title":"The value of a second reviewer for study selection in systematic reviews.","volume":"10","author":"C Stoll","year":"2019","journal-title":"Res Synth Methods."},{"key":"ref40","doi-asserted-by":"publisher","first-page":"b3128","DOI":"10.1136\/bmj.b3128","article-title":"Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study.","volume":"339","author":"B Tendal","year":"2009","journal-title":"BMJ."},{"key":"ref41","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1186\/s13643-023-02374-3","article-title":"Title-plus-abstract versus title-only first-level screening approach: a case study using a systematic review of dietary patterns and sarcopenia risk to compare screening performance.","volume":"12","author":"L Teo","year":"2023","journal-title":"Syst Rev."},{"key":"ref42","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1186\/s40814-019-0423-8","article-title":"A guide to the reporting of protocols of pilot and feasibility trials.","volume":"5","author":"L Thabane","year":"2019","journal-title":"Pilot Feasibility Stud."},{"key":"ref43","article-title":"Covidence systematic review software."},{"key":"ref44","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1186\/s12874-019-0782-0","article-title":"Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review.","volume":"19","author":"S Waffenschmidt","year":"2019","journal-title":"BMC Med Res Methodol."}],"container-title":["HRB Open Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/hrbopenresearch.org\/articles\/8-54\/v3\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/hrbopenresearch.org\/articles\/8-54\/v3\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/hrbopenresearch.org\/articles\/8-54\/v3\/iparadigms","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T16:09:08Z","timestamp":1776355748000},"score":1,"resource":{"primary":{"URL":"https:\/\/hrbopenresearch.org\/articles\/8-54\/v3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,16]]},"references-count":41,"URL":"https:\/\/doi.org\/10.12688\/hrbopenres.14129.3","relation":{"has-version":[{"id-type":"doi","id":"10.12688\/hrbopenres.14129.2","asserted-by":"subject"},{"id-type":"doi","id":"10.12688\/hrbopenres.14129.1","asserted-by":"subject"}]},"ISSN":["2515-4826"],"issn-type":[{"value":"2515-4826","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,16]]},"assertion":[{"value":"Indexed","URL":"https:\/\/hrbopenresearch.org\/articles\/8-54\/v3#article-reports","order":0,"name":"referee-status","label":"Referee status","group":{"name":"current-referee-status","label":"Current Referee Status"}},{"value":"10.21956\/hrbopenres.15528.r50011, Declan Devane, University of Galway, Galway, 30 Sep 2025, version 1, 1 approved","URL":"https:\/\/hrbopenresearch.org\/articles\/8-54\/v1#referee-response-50011","order":0,"name":"referee-response-50011","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"<b>Elayne Ahern<\/b>; \n<i>Posted: 17 Dec 2025<\/i>; Thank you kindly for the time and consideration given to reviewing our protocol. For ease, we have provided the peer-review comments, formatted in bold, with our response then presented immediately thereafter, point-by-point.  \n<b>This protocol addresses a practical and under\u2011studied step of evidence synthesis: how to train and pair novice screeners so that study selection remains reliable and efficient. The rationale is well supported by prior work on dual screening, training effects, and kappa\u2011based reliability. Strengths include a clear factorial design, preregistration, transparency about materials, and attention to feasibility and acceptability. Overall, the protocol is clear and useful, but several aspects would benefit from clarification or modest redesign to strengthen internal validity and the interpretability of results.<\/b> Thank you for positively receiving our work. The recommendations are much appreciated, with the validity and clarity of our protocol strengthened following these suggested revisions.  \n<b>Your paper already says this is a pilot that will inform a larger trial. To make that unmistakable, add a brief sentence near the start of Methods and again in Analysis that the goal is learning and planning. Then list two or three simple progression rules, for example completion rate, time to finish, and acceptability thresholds, based on the feasibility data you already collect on pages 7 to 9.<\/b> Thank you for this suggestion. We have added a statement at the beginning of the Methods and again throughout the Analysis section to clarify that the primary purpose of this SWAR is to learn from pilot study findings and plan for a future definitive trial. In addition, we have now added a \u2018Progression criteria\u2019 subsection to the Methods, where we have specified criteria to guide decisions about progression and refinement of the full-scale trial based on feasibility and acceptability outcomes, including completion rate, time required of participants, and perceived usefulness of training and screener pairings when undertaking the study selection task.  \n<b>The current aim reads like a test of which arm is best. A small rewording might fix that e.g., \u201cThis pilot will assess feasibility and acceptability to plan a definitive trial\u201d and align outcomes accordingly.<\/b> Thank you for this helpful suggestion. We have revised the aim to more clearly reflect the pilot purpose of assessing feasibility and acceptability to inform a future definitive trial. We have also updated the objectives and language so that reliability outcomes are framed as exploratory estimates intended to guide the design of the future trial, rather than as formal comparative tests.  \n<b>Readers will appreciate a brief expansion of how the 219 records will be chosen.<\/b> We have expanded the Methods section to clarify the process used to select the 219 records. Specifically, we now explain that the sample was purposefully drawn to reflect the breadth of eligibility criteria in the host review, including a proportion of records previously screened as \u201cinclude\u201d (\n<i>k<\/i> = 44) and a diverse set of \u201cexclude\u201d records (\n<i>k<\/i> = 175) to represent the full range of exclusion reasons. This ensures that participants encounter a variety of screening decisions and the need to apply various exclusion rationales, supporting a meaningful assessment of how well the training conditions prepare novice screeners to apply the eligibility criteria.  \n<b>Because Covidence shows agreement or conflict as people work, learning during the task is very plausible. Suggest that the record order be randomised for each participant if possible, and report performance by time or by item deciles to show any learning pattern.<\/b> Thank you for this helpful guidance. We explored options for randomising record order and analysing learning patterns. However, as confirmed by Covidence Support, Covidence software does not currently support randomisation of the screening order. Records are displayed by default using the \u201cmost relevant\u201d ordering, and this is how the study selection task was presented to participants. The \u201cmost relevant\u201d ordering is underpinned by Covidence\u2019s internal active learning algorithm, although users can switch to alternative static orderings (e.g., Title or Author) if they choose. Further, the Covidence platform does not provide time-stamped screening data (only the date the record was screened is available), which prevents analyses by time or by item deciles. We recognise that not being able to explore learning during study selection is a limitation and unfortunately cannot action this for our pilot SWAR given the constraints of the platform.  \n<b>With only three participants per cell, the averages of kappa by arm will be wide regardless. You might consider keeping those and adding a complementary item-level mixed model that treats each screened record as one observation and includes effects for training, partner type, and their interaction, with random effects for participant and item.<\/b> Thank you for this recommended addition to our analysis plan. We agree that with three participants per condition, arm-level kappa estimates will be imprecise and should be interpreted cautiously. Consistent with the pilot and exploratory aims of this SWAR, we have added an item-level mixed-effects analysis to explore patterns in screening decisions while accounting for clustering by participant and by item (i.e. individual study records). We clarify that this analysis is exploratory and is intended to inform the design and analytical approach of a subsequent definitive trial, rather than to support confirmatory analyses.  \n<b>In Analysis on page 9 you plan to treat confidence intervals that cross zero as evidence of equivalence. That can be misread. Suggest you avoid equivalence language or set a small, prespecified equivalence margin for kappa differences and say any tests are exploratory.<\/b> We agree. Any reference to evidence of equivalence has now been removed and we further underscore that any analyses are exploratory.  \n<b>Kappa can be sensitive to prevalence. Keep kappa and percent agreement, then add one robust agreement index such as PABAK. This may make the results easier to read if the include rate is low or high.<\/b> Thank you for this suggestion. We have revised the protocol to include reporting of the prevalence-adjusted and bias-adjusted kappa (PABAK), alongside Cohen\u2019s kappa and percent agreement, to support interpretation of reliability given the low number of \u2018includes\u2019 during the study selection task.  \n<b>You define the partner with 4 to 6 years as moderate experience on page 6, but label the arm \u201cexperienced\u201d. Maybe revised labelling.<\/b> This was an oversight on our part and we thank you for drawing this to our attention. We have now updated our figure labels to reflect \u2018moderately experienced\u2019 and likewise updated this throughout the manuscript.  \n<b>Because one person represents each partner type, partner identity is confounded with partner type. Flag this as a pilot limitation and say that the full trial will include multiple partners per level.<\/b> Thank you for prompting consideration of this. We have now included a note on this limitation in our conclusion section with a view to addressing it in the future trial, where feasible.  \n<b>The Procedure seems to ask participants to vote YES when unsure. Suggest adding a sentence that explains the consequence of this rule for sensitivity and specificity, so readers know you expect higher sensitivity and possibly lower specificity by design.<\/b> Many thanks for this suggestion. We acknowledge the importance of this caveat and have added a note to our methods to indicate the impact and trade-off between sensitivity and specificity by reducing decision options to just two, \u2018YES\u2019 or \u2018NO\u2019.  \n<b>Time is self-reported in your feasibility outcomes. If Covidence logs can provide start or finish times, say that you will also capture those as an objective check on efficiency.<\/b> We have queried the availability of such logs with Covidence Support and unfortunately such data is not available. Our self-reported measure was included as a means to overcome this and have access to data on time efficiency. We nonetheless acknowledge that an objective measure would strengthen the validity and reliability of this outcome.","URL":"https:\/\/hrbopenresearch.org\/articles\/8-54\/v3#referee-comment-3858","order":1,"name":"referee-comment-3858","label":"Referee Comment","group":{"name":"article-reports","label":"Article Reports"}},{"value":"10.21956\/hrbopenres.15764.r53773, Heather MacDonald, MacOdrum Library, Carleton University, Ottawa, On, 12 Mar 2026, version 2, indexed","URL":"https:\/\/hrbopenresearch.org\/articles\/8-54\/v2#referee-response-53773","order":2,"name":"referee-response-53773","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"<b>Elayne Ahern<\/b>; \n<i>Posted: 31 Mar 2026<\/i>; Thank you kindly for the time and attention given to reviewing our protocol. The feedback is very much appreciated and has helped to enhance the quality of our work. For ease of reference, we have listed the peer-review comments in bold, followed immediately by our responses, presented point-by-point.  \n<b>This protocol is well designed to address the study questions posed. There is currently a lack of evidence on novice screener training, and this protocol helps fill this gap. The changes from the first round of revisions to this protocol increases its rigour and clarity.<\/b> Thank you for positively receiving our work and for noting the improved rigour and clarity as a result of the revisions already incorporated.  \n<b>I only have one comment about the open-ended question about recommendations for improvement. There is no analysis mentioned for this. What do you hope to gain from including this question? With such a small sample size a thematic analysis is not appropriate. Will you highlight any quotes? Consider adding a sentence to address this.<\/b> Thank you for this helpful suggestion as we acknowledge that we lacked sufficient detail on this in our previous manuscript version. We have now clarified the handling of the open-ended responses about recommendations, which we intend to use for the purposes of refining the intervention or study design in subsequent work. Given the small sample size and the expectation that responses will be brief, rather than in-depth, we do not plan to conduct formal qualitative analysis. Instead, the responses will be summarised descriptively, with illustrative quotes included where they help to highlight aspects of participants\u2019 experiences and suggested improvements for future trial work.","URL":"https:\/\/hrbopenresearch.org\/articles\/8-54\/v3#referee-comment-3899","order":3,"name":"referee-comment-3899","label":"Referee Comment","group":{"name":"article-reports","label":"Article Reports"}},{"value":"This study was supported by the Health Research Board (Ireland) and the HSC Public Health Agency [ESI-2021-001] through Evidence Synthesis Ireland\/Cochrane Ireland. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.","order":4,"name":"grant-information","label":"Grant Information"},{"value":"This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.","order":0,"name":"copyright-info","label":"Copyright"}]}}