{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T08:49:08Z","timestamp":1791622148558,"version":"4.3.4"},"update-to":[{"DOI":"10.12688\/hrbopenres.14338.1","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T00:00:00Z","timestamp":1791590400000}}],"reference-count":32,"publisher":"F1000 Research Ltd","license":[{"start":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T00:00:00Z","timestamp":1791590400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Health Research Board SPHeRE-2022-1","id":[{"id":"https:\/\/ror.org\/003hb2249","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":["hrbopenresearch.org"],"crossmark-restriction":false},"short-container-title":["HRB Open Res"],"abstract":"<ns3:p>Background Knowledge translation (KT) interventions support the use of research evidence in health policymaking but vary substantially in form, complexity and evaluation. There is no comprehensive synthesis of how KT interventions involving health policy decision makers are evaluated across intervention types and policy contexts. Objectives This scoping review aims to map and describe how KT interventions involving health policy decision makers are evaluated. It will identify evaluation approaches, designs and methods; describe theories, frameworks, programme logics and rationales informing evaluation choices; examine how evaluation characteristics vary by intervention type, complexity, intended outcomes, policy context and evaluation use; and identify gaps in evaluative coverage. Methods This review will be conducted in accordance with established scoping review guidance and reported using PRISMA-ScR. Eligibility will be structured using the Population\u2013Concept\u2013Context framework. Studies published in English from 2000 onwards that empirically evaluate a KT intervention involving or directed towards health policy decision makers will be eligible. Evaluation characteristics will be examined using descriptive numerical mapping, Lemire\u2019s Evaluation Metro Map as an organising and communication tool, and structured qualitative content analysis. The Mixed Methods Appraisal Tool will provide supplementary methodological context. The review will characterise evaluation practice rather than estimate intervention effectiveness or rank studies. Dissemination Findings will be published in an open-access peer-reviewed journal and disseminated through conference presentations and plain-language summaries for academic and policy audiences.<\/ns3:p>","DOI":"10.12688\/hrbopenres.14338.2","type":"journal-article","created":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T08:19:06Z","timestamp":1791620346000},"page":"27","update-policy":"https:\/\/doi.org\/10.12688\/hrbopenres.crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["How knowledge translation interventions involving health policy decision makers are evaluated: a scoping review protocol."],"prefix":"10.12688","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-4328-533X","authenticated-orcid":false,"given":"Amelia","family":"Phillips","sequence":"first","affiliation":[{"name":"School of Nursing and Midwifery Galway County Galway H91 TK33 University of Galway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Medeina","family":"Macenaite","sequence":"additional","affiliation":[{"name":"School of Health Sciences Galway County Galway H91 TK33 University of Galway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Molly","family":"Byrne","sequence":"additional","affiliation":[{"name":"School of Psychology Galway County Galway H91 TK33 University of Galway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7239-7241","authenticated-orcid":false,"given":"Shaun","family":"Treweek","sequence":"additional","affiliation":[{"name":"School of Medicine, Medical Sciences and Nutrition Aberdeen Scotland AB24 3FX University of Aberdeen"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elaine","family":"Toomey","sequence":"additional","affiliation":[{"name":"School of Nursing and Midwifery Galway County Galway H91 TK33 University of Galway"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"2560","published-online":{"date-parts":[[2026,10,10]]},"reference":[{"key":"ref1","first-page":"1-9","article-title":"Defining health policy makers and their use of evidence.","volume":"13","author":"A Haynes","year":"2015","journal-title":"Health Res Policy Syst."},{"key":"ref2","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1186\/s12961-023-01066-7","article-title":"Disseminating health research to public health policy-makers and practitioners.","volume":"21","author":"S McCrabb","year":"2023","journal-title":"Health Res Policy Syst."},{"key":"ref5","doi-asserted-by":"publisher","first-page":"37-45","DOI":"10.1002\/chp.49","article-title":"Research, public policymaking, and knowledge translation processes: Canadian efforts to build bridges.","volume":"26","author":"J Lavis","year":"2006","journal-title":"J. Contin. Educ. Heal. Prof."},{"key":"ref6","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1186\/s12961-022-00884-5","article-title":"Development of a STandard reporting guideline for Evidence briefs for Policy (STEP): Context and study protocol.","volume":"20","author":"X Yu","year":"2022","journal-title":"Health Res Policy Syst."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"n2061","DOI":"10.1136\/bmj.n2061","article-title":"A new framework for developing and evaluating complex interventions: Update of Medical Research Council guidance.","volume":"374","author":"K Skivington","year":"2021","journal-title":"BMJ."},{"key":"ref9","doi-asserted-by":"publisher","first-page":"266-274","DOI":"10.1080\/13668250903093125","article-title":"The Kirkpatrick model: A useful tool for evaluating training outcomes.","volume":"34","author":"A Smidt","year":"2009","journal-title":"J. Intellect. Develop. Disabil."},{"key":"ref11","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1186\/s12961-019-0511-5","article-title":"Strengthening capacity to use research evidence in health sector policy-making: Experience from Kenya and Malawi.","volume":"17","author":"R Oronje","year":"2019","journal-title":"Health Res Policy Syst."},{"key":"ref12","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1186\/s13012-016-0472-9","article-title":"Designing a rapid response program to support evidence-informed decision-making in the Americas region.","volume":"11","author":"M Haby","year":"2016","journal-title":"Implement. Sci."},{"key":"ref13","doi-asserted-by":"publisher","first-page":"122-131","DOI":"10.12927\/hcpol.2014.23808","article-title":"Deliberative dialogues as a strategy for system-level knowledge translation and exchange.","volume":"9","author":"J Boyko","year":"2014","journal-title":"Health Policy."},{"key":"ref14","doi-asserted-by":"publisher","first-page":"1287","DOI":"10.1186\/1471-2458-14-1287","article-title":"Evaluating deliberative dialogues focussed on healthy public policy.","volume":"14","author":"J Lavis","year":"2014","journal-title":"BMC Public Health."},{"key":"ref15","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1186\/s12961-020-00612-x","article-title":"Evaluation of the performance and achievements of the WHO Evidence-informed Policy Network (EVIPNet) Europe.","volume":"18","author":"L Lester","year":"2020","journal-title":"Health Res Policy Syst."},{"key":"ref16","doi-asserted-by":"publisher","first-page":"267-279","DOI":"10.1332\/174426409X463811","article-title":"Knowledge brokering: The missing link in the evidence to action chain?","volume":"5","author":"V Ward","year":"2009","journal-title":"Evid Policy."},{"key":"ref17","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1186\/s13012-015-0351-9","article-title":"Exploring the function and effectiveness of knowledge brokers as facilitators of knowledge translation in health-related settings: A systematic review and thematic analysis.","volume":"10","author":"C Bornbaum","year":"2015","journal-title":"Implement Sci."},{"key":"ref18","doi-asserted-by":"publisher","first-page":"236-240","DOI":"10.1377\/hlthaff.19.3.236","article-title":"Using linkage and exchange to move research into policy at a Canadian foundation.","volume":"19","author":"J Lomas","year":"2000","journal-title":"Health Aff (Millwood)."},{"key":"ref19","doi-asserted-by":"publisher","first-page":"270-280","DOI":"10.1177\/1356389012451663","article-title":"Contribution analysis: Coming of age?","volume":"18","author":"J Mayne","year":"2012","journal-title":"Evaluation."},{"key":"ref20","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1186\/s12961-015-0018-7","article-title":"Applying knowledge translation tools to inform policy: The case of mental health in Lebanon.","volume":"13","author":"F Yehia","year":"2015","journal-title":"Health Res Policy Syst."},{"key":"ref21","doi-asserted-by":"crossref","first-page":"1-12","DOI":"10.1186\/s12889-017-4254-3","article-title":"Translational research for obesity prevention in communities (TROPIC): A knowledge exchange project in Fiji.","volume":"17","author":"P Kremer","year":"2017","journal-title":"BMC Public Health."},{"key":"ref22","first-page":"345-357","article-title":"Knowledge management systems for health policy: A scoping review.","volume":"127","author":"F El-Jardali","year":"2023","journal-title":"Health Policy."},{"key":"ref23","doi-asserted-by":"publisher","first-page":"751","DOI":"10.1186\/1471-2458-12-751","article-title":"The effectiveness of knowledge translation strategies used in public health: A systematic review.","volume":"12","author":"R LaRocca","year":"2012","journal-title":"BMC Public Health."},{"key":"ref24","doi-asserted-by":"publisher","first-page":"266","DOI":"10.1186\/s13643-019-1161-y","article-title":"Systematic review of the use of process evaluations in knowledge translation research.","volume":"8","author":"S Scott","year":"2019","journal-title":"Syst Rev."},{"key":"ref25","doi-asserted-by":"publisher","first-page":"a1655","DOI":"10.1136\/bmj.a1655","article-title":"Developing and evaluating complex interventions: The new Medical Research Council guidance.","volume":"337","author":"P Craig","year":"2008","journal-title":"BMJ."},{"key":"ref26","doi-asserted-by":"publisher","first-page":"h1258","DOI":"10.1136\/bmj.h1258","article-title":"Process evaluation of complex interventions: Medical Research Council guidance.","volume":"350","author":"G Moore","year":"2015","journal-title":"BMJ."},{"key":"ref27","doi-asserted-by":"publisher","first-page":"35-40","DOI":"10.56645\/jmde.v20i48.1063","article-title":"The Evaluation Metro Map.","volume":"20","author":"S Lemire","year":"2024","journal-title":"J MultiDiscipl Eval."},{"key":"ref28","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1186\/s12874-018-0611-x","article-title":"Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach.","volume":"18","author":"Z Munn","year":"2018","journal-title":"BMC Med Res Methodol."},{"key":"ref29","doi-asserted-by":"publisher","first-page":"520-532","DOI":"10.11124\/JBIES-22-00123","article-title":"Recommendations for the extraction, analysis, and presentation of results in scoping reviews.","volume":"21","author":"D Pollock","year":"2023","journal-title":"JBI Evid Synth."},{"key":"ref30","doi-asserted-by":"crossref","first-page":"19-32","DOI":"10.1080\/1364557032000119616","article-title":"Scoping studies: Towards a methodological framework.","volume":"8","author":"H Arksey","year":"2005","journal-title":"Int. J. Soc. Res. Methodol."},{"key":"ref31","doi-asserted-by":"publisher","first-page":"467-473","DOI":"10.7326\/M18-0850","article-title":"PRISMA extension for Scoping Reviews (PRISMA-ScR): Checklist and explanation.","volume":"169","author":"A Tricco","year":"2018","journal-title":"Ann Int Med."},{"key":"ref32","doi-asserted-by":"publisher","first-page":"40-46","DOI":"10.1016\/j.jclinepi.2016.01.021","article-title":"PRESS Peer Review of Electronic Search Strategies: 2015 guideline statement.","volume":"75","author":"J McGowan","year":"2016","journal-title":"J Clin Epidemiol."},{"key":"ref33","doi-asserted-by":"publisher","first-page":"210","DOI":"10.1186\/s13643-016-0384-4","article-title":"Rayyan\u2014a web and mobile app for systematic reviews.","volume":"5","author":"M Ouzzani","year":"2016","journal-title":"Syst Rev."},{"key":"ref34","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1186\/s41073-016-0007-6","article-title":"Sex and Gender Equity in Research: rationale for the SAGER guidelines and recommended use.","volume":"1","author":"S Heidari","year":"2016","journal-title":"Res Integr Peer Rev."},{"key":"ref35","doi-asserted-by":"publisher","first-page":"107-115","DOI":"10.1111\/j.1365-2648.2007.04569.x","article-title":"The qualitative content analysis process.","volume":"62","author":"S Elo","year":"2008","journal-title":"J Adv Nursing."},{"key":"ref36","doi-asserted-by":"publisher","first-page":"459-467","DOI":"10.1111\/jep.12884","article-title":"Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT).","volume":"24","author":"Q Hong","year":"2018","journal-title":"J Eval Clin Pract."}],"container-title":["HRB Open Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/hrbopenresearch.org\/articles\/9-27\/v2\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/hrbopenresearch.org\/articles\/9-27\/v2\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/hrbopenresearch.org\/articles\/9-27\/v2\/iparadigms","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T08:19:08Z","timestamp":1791620348000},"score":1,"resource":{"primary":{"URL":"https:\/\/hrbopenresearch.org\/articles\/9-27\/v2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,10]]},"references-count":32,"URL":"https:\/\/doi.org\/10.12688\/hrbopenres.14338.2","relation":{"has-version":[{"id-type":"doi","id":"10.12688\/hrbopenres.14338.1","asserted-by":"subject"}]},"ISSN":["2515-4826"],"issn-type":[{"value":"2515-4826","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,10]]},"assertion":[{"value":"Indexed","URL":"https:\/\/hrbopenresearch.org\/articles\/9-27\/v2#article-reports","order":0,"name":"referee-status","label":"Referee status","group":{"name":"current-referee-status","label":"Current Referee Status"}},{"value":"10.21956\/hrbopenres.15790.r54476, Ian Graham, Ottawa Hospital Research Institute, Ottawa, Ontario, 05 May 2026, version 1, indexed","URL":"https:\/\/hrbopenresearch.org\/articles\/9-27\/v1#referee-response-54476","order":0,"name":"referee-response-54476","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"<b>Amelia Phillips<\/b>; \n<i>Posted: 15 Jul 2026<\/i>; We sincerely thank Dr. Graham for his thoughtful and constructive review. His comments were particularly helpful in strengthening the conceptual clarity of the protocol, especially in relation to the distinction between evaluation approach and evaluation design, the consistent definition of the population of interest, and the use of the Evaluation Metro Map. We have given careful consideration to each suggestion.  \n<b>General comments<\/b> \n<b>Comment:<\/b> \n<i>This paper presents a scoping review protocol examining how KT interventions directed at health policymakers are evaluated. The review is timely and will highlight the current evaluation approaches, designs and methods used in evaluating KT interventions involving policymakers. The proposed methodology is appropriate and rigorous.<\/i> \n<b>Response:<\/b> We thank the reviewer for this positive assessment and for recognising the relevance and methodological rigour of the proposed review.  \n<b>Organisation of Table 1<\/b> \n<b>Comment:<\/b> \n<i>Table 1 provides an excellent conceptual overview, although the rationale for the ordering of the rows is unclear. Starting with the push, pull and exchange strategies and then grouping the related KT interventions may help readers appreciate similarities between the strategies and interventions.<\/i> \n<b>Response:<\/b> Thank you, and we do agree. Table 1 has been reorganised so that the principal KT orientations of push, pull and exchange provide the primary structure. Interventions associated with each orientation are now grouped together. Where an intervention may involve more than one orientation, this is acknowledged rather than assigning it unambiguously to a single category. This revision is intended to make the conceptual organisation clearer while recognising that KT interventions may combine multiple strategies.  \n<b>Use of the Evaluation Metro Map<\/b> \n<b>Comment:<\/b> \n<i>Few studies may explicitly label the evaluation approach taken, but where this information is reported or can be inferred, the Lemire Evaluation Metro Map, or another taxonomy, may be informative. The theory line may also provide insight into the epistemological perspectives of the evaluations.<\/i> \n<b>Response:<\/b> We agree and have adopted Lemire\u2019s Evaluation Metro Map as an organising and communication tool within the review. The Metro Map will be used to describe relevant dimensions of the evaluations consistently, including their intended use, theoretical orientation and design characteristics. An additional extraction will be undertaken using the language of the Metro Map across all included studies. This will allow the review to communicate more clearly what each evaluation appears to be trying to accomplish, how it is conceptually informed and how it is operationalised. We recognise that many studies may not explicitly identify a named evaluation approach or theoretical orientation. We will therefore distinguish between information that is explicitly reported by study authors and information that can be cautiously interpreted from the study description. We will avoid assigning a named approach where there is insufficient evidence to support that classification. The Metro Map will not be used as a quality-assessment or ranking tool. It will instead provide a shared language through which the different dimensions of an evaluation can be described without reducing the evaluation to its study design alone. A citation to Lemire\u2019s Evaluation Metro Map has been added, and its role in the extraction and analysis has been explained in the Methods section. Involvement of health policy decision makers in the review team  \n<b>Comment:<\/b> \n<i>It is unclear whether health policymakers are included in the study team.<\/i> \n<b>Response:<\/b> No health policy decision makers are formal members of the review team. This will now be stated explicitly in the manuscript. There will be health policymakers included at the end of this study to assist with dissemination and translation of research findings. In all subsequent studies for the PhD project there will be consistent health policymaker inclusion. It would have been preferable to have policymaker input from the conception of this review, however there was time needed to build networks with the relevant individuals and that overlapped with the review timeline.  \n<b>Definition and consistency of terminology<\/b> \n<b>Comment:<\/b> \n<i>Policymakers should be defined earlier in the manuscript. It is also unclear how \u201chealth policymakers\u201d differ from \u201cpolicy-level decision makers\u201d.<\/i> \n<b>Response:<\/b> We agree that the previous terminology introduced unnecessary ambiguity. The manuscript has been revised to use the term health policy decision makers consistently and exclusively when referring to the population of interest. A definition has been added to the Background section before the term is used in the eligibility criteria. For the purposes of this review, health policy decision makers are defined as individuals who have a formal or substantive role in developing, influencing, implementing, commissioning or advising on decisions relating to health policy, health services or health systems. This may include decision makers based within government departments, public health agencies, health service organisations, regulatory bodies and other organisations with responsibility for health policy decisions. Individuals working outside the health sector will be included only where their role involves identifiable decisions that directly affect health policy, population health or health-system organisation. The term \u201cpolicy-level decision makers\u201d has been removed to avoid suggesting that it represents a separate population.  \n<b>Sex- and gender-related data<\/b> \n<b>Comment:<\/b> \n<i>The review should consider collecting and reporting sex- and gender-related data. Even if these characteristics are rarely reported, their absence may be an important finding. The SAGER reporting guideline should also be considered.<\/i> \n<b>Response:<\/b> We agree. The data-charting framework has been amended to collect information on the sex and\/or gender of evaluation participants where this is reported in the included studies. We will record the terminology used by the original authors and will not treat sex and gender as interchangeable or infer either characteristic where it has not been reported. Where studies provide data that allow findings to be examined by sex or gender, this will also be recorded. However, we anticipate that the extent of reporting may be limited. The review will therefore report both the information provided and the extent to which sex and gender are absent from evaluation reporting. The implications of limited reporting for understanding whether KT interventions may be experienced differently by health policy decision makers of different sexes or genders will be discussed cautiously. The SAGER guidelines have been added as a reporting reference and will inform how these data and their limitations are presented.  \n<b>English-language restriction<\/b> \n<b>Comment:<\/b> \n<i>Given the availability of reasonably good AI translation technologies, a rationale should be provided for including only studies published in English.<\/i> \n<b>Response:<\/b> We agree that developments in machine-assisted and AI-supported translation have increased the potential accessibility of studies published in other languages. However, the English-language restriction will be retained as a feasibility constraint and will be justified more clearly in the manuscript. Although AI-generated translation may support initial comprehension, translations of technical methodological descriptions, theoretical concepts and context-specific policy terminology would still require verification to ensure that the extracted information was accurate. This would require access to reviewers or translators with appropriate language proficiency and subject knowledge. The review includes detailed extraction relating to evaluation theory, rationale, design, methods, outcomes and policy context. Inaccuracies in the translation of these concepts could materially affect study classification and interpretation. Verifying translations across potentially multiple languages would therefore introduce substantial additional work that cannot be accommodated within the timeframe and resources of the PhD. The manuscript will acknowledge that the restriction may exclude relevant evidence and may limit the geographical breadth of the review.  \n<b>PRESS review and search dates<\/b> \n<b>Comment:<\/b> \n<i>The reason the search strategy was not PRESS reviewed is unclear. The search period should also be stated in the search strategy section.<\/i> \n<b>Response:<\/b> We agree that this requires further explanation. The original search strategy was not prospectively peer reviewed using the PRESS framework because of the timing and resource constraints associated with conducting the review within the PhD programme. A retrospective PRESS review of the search strategy is now being undertaken by an information specialist at the University of Galway. So far, the PRESS review has recommended using Ovid MEDLINE ALL rather than PubMed for any revised search, presenting the strategy line by line with hit counts, combining database-specific subject headings with free-text terms, and reviewing spelling variants, proximity operators and evaluation-related terminology. It also prompted consideration of overlap between ScienceDirect and Scopus. The manuscript will state clearly that the PRESS assessment was retrospective rather than implying that it occurred before the searches were conducted. Any issues identified through the retrospective review, and any resulting amendments or implications for the review, will be documented transparently. The Methods section will also be revised to state explicitly that the search covers literature published from 2000 to the date on which each database search was conducted. The final search date for each database will be reported.  \n<b>Distinction between evaluation approach and evaluation design<\/b> \n<b>Comment:<\/b> \n<i>Evaluation approach and evaluation design are distinct concepts and should be defined and not used as synonyms.<\/i> \n<b>Response:<\/b> We agree and have revised the manuscript to distinguish these concepts consistently. An evaluation approach refers to the broader orientation or set of principles that shapes how an evaluation is understood and undertaken. Examples may include realist, utilisation-focused, developmental or equity-focused evaluation. An evaluation design refers to the empirical structure used to generate and analyse evidence, such as a case study, randomised controlled trial, interrupted time-series study, before-and-after study or qualitative interview study. The previous manuscript did not always communicate this distinction sufficiently clearly. The relevant sections have therefore been revised to ensure that evaluation approach, design, methods and intended use are not treated as interchangeable concepts. Adoption of the Evaluation Metro Map will support this distinction by providing language through which the different dimensions of an evaluation can be described separately. The review will not infer that a study\u2019s design, such as a case study or randomised trial, constitutes its broader evaluation approach. Where an approach is not named or cannot be identified with reasonable confidence, this will be reported as not stated or unclear rather than being derived solely from the design.","URL":"https:\/\/hrbopenresearch.org\/articles\/9-27\/v2#referee-comment-3948","order":1,"name":"referee-comment-3948","label":"Referee Comment","group":{"name":"article-reports","label":"Article Reports"}},{"value":"10.21956\/hrbopenres.15790.r54469, Jacqui Cameron, University of Wollongong, Wollongong, 20 May 2026, version 1, indexed","URL":"https:\/\/hrbopenresearch.org\/articles\/9-27\/v1#referee-response-54469","order":2,"name":"referee-response-54469","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"<b>Amelia Phillips<\/b>; \n<i>Posted: 15 Jul 2026<\/i>; We sincerely thank Dr Cameron for her careful review and encouraging assessment of the protocol. Her comments helped us improve the clarity and transparency of the methods, particularly the rationale for using PCC, the presentation of the data-charting framework, the description of the analysis, and the justification for retaining a scoping review design. We have revised the manuscript accordingly.  \n<b>General comments<\/b> \n<b>Comment:<\/b> \n<i>The manuscript is well written, important and within the scope of the journal. The suggested revisions are minor.<\/i> \n<b>Response:<\/b> We thank the reviewer for the positive assessment and for the constructive recommendations provided.  \n<b>Abstract wording<\/b> \n<b>Comment:<\/b> \n<i>Change the first sentence of the Methods section of the abstract to \u201cThis review will be conducted\u2026\u201d.<\/i> \n<b><i>Response:<\/i><\/b> This sentence has been amended as suggested.  \n<b>Position and analytical role of Table 1<\/b> \n<b>Comment:<\/b> \n<i>The inclusion of Table 1 in the Background is uncertain. It may provide a valuable analytical lens.<\/i> \n<b>Response:<\/b> We agree that the purpose of Table 1 required clarification. The table will be retained as a conceptual overview of the forms of KT intervention potentially relevant to the review. As recommended by Reviewer 1, it has been reorganised around push, pull and exchange orientations. The table will also inform the descriptive categorisation of the included interventions during data charting and analysis. However, it will not be used as a rigid classification system, as interventions may incorporate more than one KT orientation. Its function is distinct from that of the Evaluation Metro Map. Table 1 will support the description of the KT interventions, whereas the Metro Map will support communication of the different dimensions of their evaluation. This distinction has been made explicit in the manuscript.  \n<b>Rationale for using PCC<\/b> \n<b>Comment:<\/b> \n<i>Please explain why the Population, Concept and Context framework was selected.<\/i> \n<b>Response:<\/b> A rationale for using the PCC framework has been added. PCC was selected because the purpose of the review is to map and characterise a broad and heterogeneous body of evidence rather than evaluate the effectiveness of a narrowly defined intervention. The framework corresponds directly to the review question and eligibility criteria:  Population: health policy decision makers; Concept: the evaluation of KT interventions involving or directed towards health policy decision makers; and Context: health policy, health service and health-system decision-making settings.  Unlike frameworks structured primarily around intervention comparisons and outcomes, PCC allows the review to include diverse forms of KT intervention, evaluation design, methodological approach and policy context. This is consistent with the exploratory and mapping purposes of a scoping review.  \n<b>Presentation of the data-charting framework<\/b> \n<b>Comment:<\/b> \n<i>The data-charting list is extensive and may be clearer if presented in a table.<\/i> \n<b>Response:<\/b> We agree. The data-charting items have been reorganised into a table, grouped under relevant domains. These include study characteristics, characteristics of the health policy decision-maker population, KT intervention characteristics, policy context, evaluation use and theoretical orientation, evaluation approach, evaluation design, data-collection and analytical methods, outcomes, sex- and gender-related reporting, methodological limitations and author reflections. This presentation is intended to improve readability and make the planned extraction process more readily reproducible.  \n<b>Description of the analysis<\/b> \n<b>Comment:<\/b> \n<i>The proposed analysis beyond data charting is vague. The terms \u201csynthesised descriptively\u201d and \u201csynthesised narratively\u201d require further explanation and methodological support.<\/i> \n<b>Response:<\/b> We agree and have expanded the Analysis section substantially. The revised section distinguishes among three related stages of analysis. First, descriptive numerical summaries will be used to map study characteristics, KT intervention types, populations, contexts, evaluation designs, data-collection methods, outcomes and other categorical variables. Counts will be interpreted as characteristics of the included evidence base rather than estimates of prevalence beyond that evidence base. Second, evaluation characteristics will be mapped using the language of the Evaluation Metro Map. This will support consistent consideration of the intended use of the evaluation, its theoretical orientation and its design and methodological characteristics. Explicitly reported information will be distinguished from cautious reviewer interpretation. Third, a structured narrative analysis will examine patterns across the included studies, including relationships among intervention aims, intervention complexity, intended outcomes, policy contexts and evaluation choices. The purpose will be to identify recurring patterns, areas of variation and gaps in reporting or evaluative coverage. The analysis will not be used to determine intervention effectiveness or to rank evaluation designs. The phrases \u201csynthesised descriptively\u201d and \u201csynthesised narratively\u201d have been replaced with more specific descriptions of the analytical procedures. Supporting methodological references have also been added.  \n<b>Scope of the review and suitability of a scoping review<\/b> \n<b>Comment:<\/b> \n<i>The scope appears large and there are several research questions. A systematic review may be more appropriate if several completed studies are expected.<\/i> \n<b>Response:<\/b> We considered this concern carefully but have retained the scoping review design. The primary purpose of the review is to identify and characterise the range of evaluation approaches, designs and methods used in relation to KT interventions involving health policy decision makers. It is not intended to estimate intervention effectiveness, compare a narrowly specified set of interventions or produce a pooled estimate of effect. The expected evidence base is heterogeneous in relation to the types of KT intervention, policy setting, intended use, outcomes, evaluation design and data-collection methods. A scoping review is therefore more appropriate for mapping how evaluation has been undertaken and identifying areas in which evidence or methodological reporting is limited. The review questions have been reviewed to ensure that they operate as related subquestions under one overarching mapping objective rather than as separate reviews. The objectives and analysis have also been revised to clarify their relationship and to ensure that the proposed work remains feasible within the review\u2019s scope. The existence of completed evaluation studies does not in itself require a systematic review of effectiveness. A subsequent systematic review could be warranted if this scoping review identifies a sufficiently coherent group of interventions, outcomes and designs capable of supporting a narrower effectiveness or methodological question.  \n<b>Consistency of tense<\/b> \n<b>Comment:<\/b> \n<i>The manuscript uses a mixture of past, present and future tense and should be revised for consistency.<\/i> \n<b>Response:<\/b> The manuscript has been reviewed for consistency of tense. Procedures that are proposed within the protocol are now described in the future tense. Actions that had already occurred before revision, including development of the original search and its retrospective PRESS assessment, are described in the past or present perfect tense where necessary for transparency.  \n<b>Missing references<\/b> \n<b>Comment:<\/b> \n<i>Several references are missing, including references for PRISMA-ScR and Rayyan.<\/i> \n<b>Response:<\/b> We agree and have conducted a full review of the manuscript\u2019s citations. References have been added or corrected for PRISMA-ScR, Rayyan, the PCC framework, PRESS, the Evaluation Metro Map, the SAGER guidelines and the methodological guidance informing the analysis. Citations relating to the review\u2019s conceptual background and data-charting procedures have also been checked for completeness and accuracy.","URL":"https:\/\/hrbopenresearch.org\/articles\/9-27\/v2#referee-comment-3949","order":3,"name":"referee-comment-3949","label":"Referee Comment","group":{"name":"article-reports","label":"Article Reports"}},{"value":"10.21956\/hrbopenres.15790.r54471, Lupin Battersby, Alison J. Moore, Simon Fraser University, Burnaby, 09 Oct 2026, version 1, indexed","URL":"https:\/\/hrbopenresearch.org\/articles\/9-27\/v1#referee-response-54471","order":4,"name":"referee-response-54471","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"This research was funded by the Health Research Board SPHeRE\/2022\/1. The funder had no role in the design of the study or preparation of this manuscript. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.","order":5,"name":"grant-information","label":"Grant Information"},{"value":"This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.","order":0,"name":"copyright-info","label":"Copyright"}]}}