{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T02:00:41Z","timestamp":1772244041807,"version":"3.50.1"},"update-to":[{"DOI":"10.3310\/nihropenres.13236.1","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,2,23]],"date-time":"2022-02-23T00:00:00Z","timestamp":1645574400000}}],"reference-count":88,"publisher":"National Institute for Health and Care Research","license":[{"start":{"date-parts":[[2022,2,23]],"date-time":"2022-02-23T00:00:00Z","timestamp":1645574400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.nationalarchives.gov.uk\/doc\/open-government-licence\/version\/3\/"}],"funder":[{"DOI":"10.13039\/501100000272","name":"National Institute for Health Research","doi-asserted-by":"publisher","award":["NIHR300099"],"award-info":[{"award-number":["NIHR300099"]}],"id":[{"id":"10.13039\/501100000272","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["openresearch.nihr.ac.uk"],"crossmark-restriction":false},"short-container-title":["NIHR Open Res"],"abstract":"<ns7:p>\n                    <ns7:bold>Background:<\/ns7:bold>\n                    A new nationally commissioned NHS England Genomic Medicine Service (GMS) was recently established to deliver genomic testing with equity of access for patients affected by rare diseases and cancer. The overarching aim of this research is to evaluate the implementation of the GMS during its early years, identify barriers and enablers to successful implementation, and provide recommendations for practice. The focus will be on the use of genomic testing for paediatric rare diseases.\n                  <\/ns7:p>\n                  <ns7:p>\n                    <ns7:bold>Methods:<\/ns7:bold>\n                    This will be a four-year mixed-methods research programme using clinic observations, interviews and surveys. Study 1 consists of qualitative interviews with designers\/implementers of the GMS in Year 1 of the research programme, along with documentary analysis to understand the intended outcomes for the Service. These will be revisited in Year 4 to compare intended outcomes with what happened in practice, and to identify barriers and facilitators that were encountered along the way. Study 2 consists of clinic observations (pre-test counselling and results disclosure) to examine the interaction between health professionals and parents, along with follow-up interviews with both after each observation. Study 3 consists of a longitudinal survey with parents at two timepoints (time of testing and 12 months post-results) along with follow-up interviews, to examine parent-reported experiences and outcomes. Study 4 consists of qualitative interviews and a cross-sectional survey with medical specialists to identify preparedness, facilitators and challenges to mainstreaming genomic testing. The use of theory-based and pre-specified constructs will help generalise the findings and enable integration across the various sub-studies. \u00a0\n                  <\/ns7:p>\n                  <ns7:p>\n                    <ns7:bold>Dissemination:<\/ns7:bold>\n                    We will disseminate our results to policymakers as findings emerge, so any suggested changes to service provision can be considered in a timely manner. A workshop with key stakeholders will be held in Year 4 to develop and agree a set of recommendations for practice.\n                  <\/ns7:p>","DOI":"10.3310\/nihropenres.13236.2","type":"journal-article","created":{"date-parts":[[2022,2,23]],"date-time":"2022-02-23T11:27:50Z","timestamp":1645615670000},"page":"23","update-policy":"https:\/\/doi.org\/10.3310\/nihropenres.crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Mixed-methods evaluation of the NHS Genomic Medicine Service for paediatric rare diseases: study protocol"],"prefix":"10.3310","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7169-1521","authenticated-orcid":false,"given":"Celine","family":"Lewis","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2528-0638","authenticated-orcid":false,"given":"James","family":"Buchanan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1200-9286","authenticated-orcid":false,"given":"Angus","family":"Clarke","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Emma","family":"Clement","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bettina","family":"Friedrich","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jillian","family":"Hastings-Ward","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3900-1425","authenticated-orcid":false,"given":"Melissa","family":"Hill","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5714-3905","authenticated-orcid":false,"given":"Ruth","family":"Horn","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anneke M.","family":"Lucassen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4191-0663","authenticated-orcid":false,"given":"Chris","family":"Patch","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandra","family":"Pickard","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lauren","family":"Roberts","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Saskia C.","family":"Sanderson","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sarah L.","family":"Wynn","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7859-1646","authenticated-orcid":false,"given":"Cecilia","family":"Vindrola-Padros","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9855-2043","authenticated-orcid":false,"given":"Monica","family":"Lakhanpaul","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"6221","published-online":{"date-parts":[[2022,2,23]]},"reference":[{"key":"ref-1","article-title":"National Genomic Test Directory","year":"2019"},{"key":"ref-2","article-title":"The 100,000 Genomes Project Protocol.","year":"2017"},{"key":"ref-3","article-title":"Genome UK: the future of healthcare","year":"2020"},{"key":"ref-4","article-title":"Everything you need to know about the NHS genomic medicine service","author":"J Robinson","year":"2020"},{"key":"ref-5","article-title":"Master's in Genomic Medicine","year":"2020"},{"key":"ref-6","article-title":"Genomics toolkit."},{"key":"ref-7","article-title":"Genomics resources"},{"key":"ref-8","article-title":"Whole genome sequencing for rare disease: Information for patients and family members.","year":"2020"},{"key":"ref-9","article-title":"Whole genome sequencing for a rare disorder: easy read.","year":"2020"},{"key":"ref-10","article-title":"Record of Discussion Regarding Genomic Testing."},{"key":"ref-11","article-title":"Taking part"},{"key":"ref-12","doi-asserted-by":"publisher","first-page":"e029699","DOI":"10.1136\/bmjopen-2019-029699","article-title":"Delivering genome sequencing in clinical practice: an interview study with healthcare professionals involved in the 100 000 Genomes Project.","volume":"9","author":"S Sanderson","year":"2019","journal-title":"BMJ Open."},{"key":"ref-13","doi-asserted-by":"publisher","first-page":"874-884","DOI":"10.1038\/s41431-020-0575-2","article-title":"Parents' motivations, concerns and understanding of genome sequencing: a qualitative interview study.","volume":"28","author":"C Lewis","year":"2020","journal-title":"Eur J Hum Genet."},{"key":"ref-14","doi-asserted-by":"publisher","first-page":"1083-1091","DOI":"10.1038\/s41436-018-0310-3","article-title":"Opening the \"black box\" of informed consent appointments for genome sequencing: a multisite observational study.","volume":"21","author":"S Sanderson","year":"2019","journal-title":"Genet Med."},{"key":"ref-15","doi-asserted-by":"publisher","first-page":"335-341","DOI":"10.1016\/j.ejmg.2018.11.024","article-title":"Fostering trust in healthcare: Participants' experiences, views, and concerns about the 100,000 genomes project.","volume":"62","author":"S Dheensa","year":"2019","journal-title":"Eur J Med Genet."},{"key":"ref-16","doi-asserted-by":"publisher","first-page":"320-328","DOI":"10.1038\/gim.2017.157","article-title":"\"Not pathogenic until proven otherwise\": perspectives of UK clinical genomics professionals toward secondary findings in context of a Genomic Medicine Multidisciplinary Team and the 100,000 Genomes Project.","volume":"20","author":"E Ormondroyd","year":"2018","journal-title":"Genet Med."},{"key":"ref-17","doi-asserted-by":"publisher","first-page":"732-741","DOI":"10.1038\/s41431-019-0570-7","article-title":"Exploring broad consent in the context of the 100,000 Genomes Project: a mixed methods study.","volume":"28","author":"L Ballard","year":"2020","journal-title":"Eur J Hum Genet."},{"key":"ref-18","article-title":"Decision-making, attitudes and understanding amongst patients and relatives invited to undergo genome sequencing in the 100,000 Genomes Project: a multi-site survey study.","author":"S Sanderson","journal-title":"Genetics In Medicine."},{"key":"ref-19","doi-asserted-by":"publisher","first-page":"89-99","DOI":"10.1377\/hlthaff.23.3.89","article-title":"How does the quality of care compare in five countries?","volume":"23","author":"P Hussey","year":"2004","journal-title":"Health Aff (Millwood)."},{"key":"ref-20","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1186\/1748-5908-4-50","article-title":"Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science.","volume":"4","author":"L Damschroder","year":"2009","journal-title":"Implement Sci."},{"key":"ref-21","doi-asserted-by":"publisher","first-page":"40-6","DOI":"10.1136\/qhc.12.1.40","article-title":"Process evaluation on quality improvement interventions.","volume":"12","author":"M Hulscher","year":"2003","journal-title":"Qual Saf Health Care."},{"key":"ref-22","doi-asserted-by":"publisher","first-page":"S1-S8","DOI":"10.1111\/j.1525-1497.2006.00355.x","article-title":"The role of formative evaluation in implementation research and the QUERI experience.","volume":"21 Suppl 2","author":"C Stetler","year":"2006","journal-title":"J Gen Intern Med."},{"key":"ref-23","article-title":"Rare Diseases: Understanding this Public Health Priority","year":"2005"},{"key":"ref-24","doi-asserted-by":"publisher","first-page":"e0172532","DOI":"10.1371\/journal.pone.0172532","article-title":"Diagnostic needs for rare diseases and shared prediagnostic phenomena: Results of a German-wide expert Delphi survey.","volume":"12","author":"S Bl\u00f6\u00df","year":"2017","journal-title":"PLoS One."},{"key":"ref-25","doi-asserted-by":"publisher","first-page":"1055-1063","DOI":"10.1038\/gim.2017.1","article-title":"A clinical utility study of exome sequencing versus conventional genetic testing in pediatric neurology.","volume":"19","author":"L Vissers","year":"2017","journal-title":"Genet Med."},{"key":"ref-26","doi-asserted-by":"publisher","first-page":"253-268","DOI":"10.1038\/nrg.2017.116","article-title":"Paediatric genomics: diagnosing rare disease in children.","volume":"19","author":"C Wright","year":"2018","journal-title":"Nat Rev Genet."},{"key":"ref-27","doi-asserted-by":"publisher","first-page":"105-107","DOI":"10.1136\/archdischild-2016-311029","article-title":"The 100 000 Genomes Project: What it means for paediatrics.","volume":"102","author":"B Griffin","year":"2017","journal-title":"Arch Dis Child Educ Pract Ed."},{"key":"ref-28","doi-asserted-by":"publisher","first-page":"1496-502","DOI":"10.1002\/ajmg.a.36525","article-title":"Parents' experiences of receiving their child's genetic diagnosis: a qualitative study to inform clinical genetics practice.","volume":"164A","author":"S Ashtiani","year":"2014","journal-title":"Am J Med Genet A."},{"key":"ref-29","doi-asserted-by":"publisher","first-page":"578-588","DOI":"10.1002\/ajmg.a.38613","article-title":"Gratitude, protective buffering, and cognitive dissonance: How families respond to pediatric whole exome sequencing in the absence of actionable results.","volume":"176","author":"A Werner-Lin","year":"2018","journal-title":"Am J Med Genet A."},{"key":"ref-30","doi-asserted-by":"publisher","first-page":"792-805","DOI":"10.1007\/s10897-016-0052-9","article-title":"Pediatric Whole Exome Sequencing: an Assessment of Parents' Perceived and Actual Understanding.","volume":"26","author":"L Tolusso","year":"2017","journal-title":"J Genet Couns."},{"key":"ref-31","doi-asserted-by":"publisher","first-page":"2635-46","DOI":"10.1002\/ajmg.a.37256","article-title":"Experiences with obtaining informed consent for genomic sequencing.","volume":"167A","author":"B Bernhardt","year":"2015","journal-title":"Am J Med Genet A."},{"key":"ref-32","doi-asserted-by":"publisher","first-page":"62-72","DOI":"10.1007\/s10897-015-9842-8","article-title":"\"Not Tied Up Neatly with a Bow\": Professionals' Challenging Cases in Informed Consent for Genomic Sequencing.","volume":"25","author":"A Tomlinson","year":"2016","journal-title":"J Genet Couns."},{"key":"ref-33","doi-asserted-by":"publisher","first-page":"1403-1416","DOI":"10.1007\/s00439-021-02331-x","article-title":"The clinical utility of exome and genome sequencing across clinical indications: a systematic review.","volume":"140","author":"S Shickh","year":"2021","journal-title":"Hum Genet."},{"key":"ref-34","doi-asserted-by":"publisher","first-page":"3-16","DOI":"10.1038\/s41436-018-0024-6","article-title":"Clinical Application of Genome and Exome Sequencing as a Diagnostic Tool for Pediatric Patients: a Scoping Review of the Literature.","volume":"21","author":"H Smith","year":"2019","journal-title":"Genet Med."},{"key":"ref-35","doi-asserted-by":"publisher","first-page":"721-728","DOI":"10.1136\/jmedgenet-2018-105396","article-title":"Rapid Paediatric Sequencing (RaPS): comprehensive real-life workflow for rapid diagnosis of critically ill children.","volume":"55","author":"L Mestek-Boukhibar","year":"2018","journal-title":"J Med Genet."},{"key":"ref-36","doi-asserted-by":"publisher","first-page":"1305-14","DOI":"10.1016\/S0140-6736(14)61705-0","article-title":"Genetic diagnosis of developmental disorders in the DDD study: a scalable analysis of genome-wide research data.","volume":"385","author":"C Wright","year":"2015","journal-title":"Lancet."},{"key":"ref-37","doi-asserted-by":"publisher","first-page":"1976-1985","DOI":"10.1038\/s41436-020-0912-4","article-title":"Parental experiences of ultrarapid genomic testing for their critically unwell infants and children.","volume":"22","author":"G Brett","year":"2020","journal-title":"Genet Med."},{"key":"ref-38","doi-asserted-by":"publisher","first-page":"2781-2790","DOI":"10.1038\/s41436-019-0565-3","article-title":"Psychological outcomes related to exome and genome sequencing result disclosure: a meta-analysis of seven Clinical Sequencing Exploratory Research (CSER) Consortium studies.","volume":"21","author":"J Robinson","year":"2019","journal-title":"Genet Med."},{"key":"ref-39","doi-asserted-by":"publisher","first-page":"1019-31","DOI":"10.1007\/s10897-016-9933-1","article-title":"Not the End of the Odyssey: Parental Perceptions of Whole Exome Sequencing (WES) in Pediatric Undiagnosed Disorders.","volume":"25","author":"A Rosell","year":"2016","journal-title":"J Genet Couns."},{"key":"ref-40","doi-asserted-by":"publisher","first-page":"1207-1214","DOI":"10.1007\/s10897-016-9958-5","article-title":"Understanding the Psychosocial Effects of WES Test Results on Parents of Children with Rare Diseases.","volume":"25","author":"L Krabbenborg","year":"2016","journal-title":"J Genet Couns."},{"key":"ref-41","doi-asserted-by":"publisher","first-page":"273-277","DOI":"10.7861\/clinmedicine.19-4-273","article-title":"The new genomic medicine service and implications for patients.","volume":"19","author":"J Barwell","year":"2019","journal-title":"Clin Med (Lond)."},{"key":"ref-42","doi-asserted-by":"publisher","first-page":"187-196","DOI":"10.1007\/s10897-017-0134-3","article-title":"Physician Experiences and Understanding of Genomic Sequencing in Oncology.","volume":"27","author":"C Weipert","year":"2018","journal-title":"J Genet Couns."},{"key":"ref-43","doi-asserted-by":"publisher","first-page":"747-753","DOI":"10.1188\/17.CJON.747-753","article-title":"Genomics Education: Knowledge of Nurses Across the Profession and Integration Into Practice.","volume":"21","author":"L Aiello","year":"2017","journal-title":"Clin J Oncol Nurs."},{"key":"ref-44","doi-asserted-by":"publisher","first-page":"151","DOI":"10.3389\/fgene.2020.00151","article-title":"Preparing Medical Specialists for Genomic Medicine: Continuing Education Should Include Opportunities for Experiential Learning.","volume":"11","author":"B McClaren","year":"2020","journal-title":"Front Genet."},{"key":"ref-45","doi-asserted-by":"publisher","first-page":"156-63","DOI":"10.1007\/s10897-013-9624-0","article-title":"Use of genetic tests among neurologists and psychiatrists: knowledge, attitudes, behaviors, and needs for training.","volume":"23","author":"M Salm","year":"2014","journal-title":"J Genet Couns."},{"key":"ref-46","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1186\/s12920-018-0360-z","article-title":"Clinical providers' experiences with returning results from genomic sequencing: an interview study.","volume":"11","author":"J Wynn","year":"2018","journal-title":"BMC Med Genomics."},{"key":"ref-47","doi-asserted-by":"publisher","first-page":"65-76","DOI":"10.1007\/s10488-010-0319-7","article-title":"Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda.","volume":"38","author":"E Proctor","year":"2011","journal-title":"Adm Policy Ment Health."},{"key":"ref-48","doi-asserted-by":"publisher","first-page":"45-59","DOI":"10.1177\/1468794106058870","article-title":"Triangulation and integration: processes, claims and implications.","volume":"6","author":"J Moran-Ellis","year":"2006","journal-title":"Qual Res."},{"key":"ref-49","article-title":"Research design. Qualitative, quantitative, and mixed methods approaches.","author":"J Creswell","year":"2003"},{"key":"ref-50","doi-asserted-by":"publisher","DOI":"10.4135\/9781506326139.n534","article-title":"Pragmatic Paradigm.","author":"K Weaver","year":"2018"},{"key":"ref-51","doi-asserted-by":"publisher","first-page":"655-663","DOI":"10.1038\/gim.2017.144","article-title":"Developing a common framework for evaluating the implementation of genomic medicine interventions in clinical care: the IGNITE Network's Common Measures Working Group.","volume":"20","author":"L Orlando","year":"2018","journal-title":"Genet Med."},{"key":"ref-52","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1038\/s41525-020-00168-3","article-title":"Learning from scaling up ultra-rapid genomic testing for critically ill children to a national level.","volume":"6","author":"S Best","year":"2021","journal-title":"NPJ Genom Med."},{"key":"ref-53","doi-asserted-by":"publisher","first-page":"743-747","DOI":"10.1038\/s41436-018-0080-y","article-title":"Opportunities to implement a sustainable genomic medicine program: lessons learned from the IGNITE Network.","volume":"21","author":"K Levy","year":"2019","journal-title":"Genet Med."},{"key":"ref-54","doi-asserted-by":"publisher","first-page":"1534-1540","DOI":"10.1038\/s41436-018-0378-9","article-title":"Qualitative study of system-level factors related to genomic implementation.","volume":"21","author":"A Zebrowski","year":"2019","journal-title":"Genet Med."},{"key":"ref-55","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1186\/s13012-017-0550-7","article-title":"Using the Consolidated Framework for Implementation Research (CFIR) to produce actionable findings: a rapid-cycle evaluation approach to improving implementation.","volume":"12","author":"R Keith","year":"2017","journal-title":"Implement Sci."},{"key":"ref-56","article-title":"Achieving behaviour change: A guide for local government and partners.","author":"R West","year":"2019"},{"key":"ref-57","article-title":"Centre for Engagement and Dissemination - Recognition payments for public contributors","year":"2016"},{"key":"ref-58","article-title":"Genomics in Mainstream Medicine Working Group","year":"2017"},{"key":"ref-59","article-title":"Logic Model Development Guide.","year":"2004"},{"key":"ref-60","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1186\/1471-2288-13-117","article-title":"Using the framework method for the analysis of qualitative data in multi-disciplinary health research.","volume":"13","author":"N Gale","year":"2013","journal-title":"BMC Med Res Methodol."},{"key":"ref-61","article-title":"Nvivo","year":"2020"},{"key":"ref-62","doi-asserted-by":"publisher","first-page":"369-374","DOI":"10.1111\/j.1442-2018.2010.00541.x","article-title":"Recruitment of hard-to-reach population subgroups via adaptations of the snowball sampling strategy.","volume":"12","author":"G Sadler","year":"2010","journal-title":"Nurs Health Sci."},{"key":"ref-63","article-title":"Observations on observation.","author":"M Angrosino","year":"2012","journal-title":"Collecting and Interpreting Qualitative Materials."},{"key":"ref-64","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1186\/s12913-018-2938-8","article-title":"Qualitative approaches to use of the RE-AIM framework: rationale and methods.","volume":"18","author":"J Holtrop","year":"2018","journal-title":"BMC Health Serv Res."},{"key":"ref-65","doi-asserted-by":"crossref","DOI":"10.4135\/9781849209274","article-title":"Interactionism.","author":"P Atkinson","year":"2003"},{"key":"ref-66","doi-asserted-by":"publisher","first-page":"1277-88","DOI":"10.1177\/1049732305276687","article-title":"Three approaches to qualitative content analysis.","volume":"15","author":"H Hsieh","year":"2005","journal-title":"Qual Health Res."},{"key":"ref-67","doi-asserted-by":"publisher","first-page":"77-101","DOI":"10.1191\/1478088706qp063oa","article-title":"Using thematic analysis in psychology.","volume":"3","author":"V Braun","year":"2006","journal-title":"Qual Res Psychol."},{"key":"ref-68","article-title":"Annual Report on the Rare Diseases and Conditions Research Activities of the National Institutes of Health FY 2000","year":"2005"},{"key":"ref-69","article-title":"User Manual \u2013 Decision Regret Scale Ottawa: Ottawa Hospital Research Institute","author":"A O'Connor","year":"1996"},{"key":"ref-70","doi-asserted-by":"publisher","first-page":"1039-1048","DOI":"10.1111\/cge.13200","article-title":"Diagnostic exome sequencing in children: A survey of parental understanding, experience and psychological impact.","volume":"93","author":"J Wynn","year":"2018","journal-title":"Clin Genet."},{"key":"ref-71","doi-asserted-by":"publisher","first-page":"1966-1972","DOI":"10.1016\/j.pec.2018.07.011","article-title":"Development of the Knowledge of Genome Sequencing (KOGS) questionnaire.","volume":"101","author":"S Sanderson","year":"2018","journal-title":"Patient Educ Couns."},{"key":"ref-72","doi-asserted-by":"publisher","first-page":"99-108","DOI":"10.1046\/j.1369-6513.2001.00140.x","article-title":"A measure of informed choice.","volume":"4","author":"T Marteau","year":"2001","journal-title":"Health Expect."},{"key":"ref-73","article-title":"User Manual \u2013 Decisional Conflict Scale.","author":"A O'Connor","year":"1993"},{"key":"ref-74","doi-asserted-by":"publisher","first-page":"1092-7","DOI":"10.1001\/archinte.166.10.1092","article-title":"A brief measure for assessing generalized anxiety disorder: the GAD-7.","volume":"166","author":"R Spitzer","year":"2006","journal-title":"Arch Intern Med."},{"key":"ref-75","doi-asserted-by":"publisher","first-page":"324-334","DOI":"10.1016\/j.ejmg.2018.11.015","article-title":"Developing a short-form of the Genetic Counselling Outcome Scale: The Genomics Outcome Scale.","volume":"62","author":"P Grant","year":"2019","journal-title":"Eur J Med Genet."},{"key":"ref-76","doi-asserted-by":"publisher","first-page":"875-86","DOI":"10.1007\/s11136-010-9648-y","article-title":"Development of the EQ-5D-Y: a child-friendly version of the EQ-5D.","volume":"19","author":"N Wille","year":"2010","journal-title":"Qual Life Res."},{"key":"ref-77","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1186\/1477-7525-2-55","article-title":"The PedsQL Family Impact Module: preliminary reliability and validity.","volume":"2","author":"J Varni","year":"2004","journal-title":"Health Qual Life Outcomes."},{"key":"ref-78","doi-asserted-by":"publisher","first-page":"477-490","DOI":"10.1007\/s10897-018-0286-9","article-title":"The Feelings About genomiC Testing Results (FACToR) Questionnaire: Development and Preliminary Validation.","volume":"28","author":"M Li","year":"2019","journal-title":"J Genet Couns."},{"key":"ref-79","doi-asserted-by":"publisher","first-page":"191-9","DOI":"10.1002\/ajmg.c.30214","article-title":"Development, testing, and validation of a patient satisfaction questionnaire for use in the clinical genetics setting.","volume":"151c","author":"B Zellerino","year":"2009","journal-title":"Am J Med Genet C Semin Med Genet."},{"key":"ref-80","doi-asserted-by":"publisher","first-page":"105-17","DOI":"10.1016\/j.janxdis.2006.03.014","article-title":"Fearing the unknown: a short version of the Intolerance of Uncertainty Scale.","volume":"21","author":"R Carleton","year":"2007","journal-title":"J Anxiety Disord."},{"key":"ref-81","doi-asserted-by":"publisher","first-page":"194-200","DOI":"10.1080\/10705500802222972","article-title":"The brief resilience scale: assessing the ability to bounce back.","volume":"15","author":"B Smith","year":"2008","journal-title":"Int J Behav Med."},{"key":"ref-82","doi-asserted-by":"publisher","first-page":"255-274","DOI":"10.3102\/01623737011003255","article-title":"Toward a Conceptual Framework for Mixed-Method Evaluation Designs.","volume":"11","author":"J Greene","year":"1989","journal-title":"Educ Eval Policy Anal."},{"key":"ref-83","article-title":"IBM SPSS Statistics for Windows, Version 27.0.","author":"I Corp","year":"2020"},{"key":"ref-84","doi-asserted-by":"publisher","DOI":"10.1037\/qup0000196","article-title":"Conceptual and Design Thinking for Thematic Analysis.","author":"V Braun","year":"2021","journal-title":"Qual Psychol."},{"key":"ref-85","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1186\/s13012-017-0635-3","article-title":"Psychometric assessment of three newly developed implementation outcome measures.","volume":"12","author":"B Weiner","year":"2017","journal-title":"Implement Sci."},{"key":"ref-86","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1186\/1748-5908-9-45","article-title":"The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership.","volume":"9","author":"G Aarons","year":"2014","journal-title":"Implement Sci."},{"key":"ref-87","doi-asserted-by":"publisher","first-page":"481-505","DOI":"10.1177\/0021886307303654","article-title":"Organizational change recipients' beliefs scale: Development of an assessment instrument.","volume":"43","author":"A Armenakis","year":"2007","journal-title":"J Appl Behav Sci."},{"key":"ref-88","article-title":"UK policy framework for health and social care research","year":"2017"}],"container-title":["NIHR Open Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/1-23\/v2\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/1-23\/v2\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/1-23\/v2\/iparadigms","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,15]],"date-time":"2023-08-15T11:50:47Z","timestamp":1692100247000},"score":1,"resource":{"primary":{"URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/1-23\/v2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,23]]},"references-count":88,"URL":"https:\/\/doi.org\/10.3310\/nihropenres.13236.2","relation":{"has-review":[{"id-type":"doi","id":"10.3310\/nihropenres.14382.r28509","asserted-by":"subject"},{"id-type":"doi","id":"10.3310\/nihropenres.14382.r28507","asserted-by":"subject"},{"id-type":"doi","id":"10.3310\/nihropenres.14382.r28507","asserted-by":"object"},{"id-type":"doi","id":"10.3310\/nihropenres.14382.r28509","asserted-by":"object"}]},"ISSN":["2633-4402"],"issn-type":[{"value":"2633-4402","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,23]]},"assertion":[{"value":"Indexed","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/1-23\/v2#article-reports","order":0,"name":"referee-status","label":"Referee status","group":{"name":"current-referee-status","label":"Current Referee Status"}},{"value":"10.3310\/nihropenres.14382.r28507, Julian Barwell, Maurice Dungey, Leicester Royal Infirmary, University Hospitals of Leicester NHS Trust, Leicester, UK, 28 Feb 2022, version 2, indexed","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/1-23\/v2#referee-response-28507","order":0,"name":"referee-response-28507","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"10.3310\/nihropenres.14382.r28509, Zornitza Stark, Australian Genomics, Melbourne, Australia, 24 Feb 2022, version 2, indexed","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/1-23\/v2#referee-response-28509","order":1,"name":"referee-response-28509","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"10.3310\/nihropenres.14352.r28439, Nina Sperber, Duke Department of Population Health Sciences, School of Medicine, Duke University, Durham, NC, USA, 07 Dec 2021, version 1, indexed","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/1-23\/v1#referee-response-28439","order":2,"name":"referee-response-28439","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"<b>Celine Lewis<\/b>; \n<i>Posted: 14 Feb 2022<\/i>; \n<b>We thank the reviewer for their thoughtful and constructive comments on the study protocol. We hope that we have addressed these points to your satisfaction.<\/b>  The authors have presented a mixed-methods protocol to evaluate implementation of genomic medicine with a focus on paediatric rare diseases. This project includes four parts which will each address different levels of implementation. The first part focuses on a \n<u>program level<\/u>, aiming to develop a logic model of the NHS GMS to evaluate concordance between intended and actual outcomes and identify barriers to outcomes by examining documents and interviewing key stakeholders involved in delivering services (including program designers, implementers, and providers). The second part focuses on an \n<u>inter-personal level<\/u> by observing providers and families (parents and child patients) conducting genomic testing in clinical settings, using a structured guide used in the 100,000 Genomes Project, to denote observations about appointment structure and provider\/patient communication, shedding light on processes and practices in naturalistic environments. The third part will target an \n<u>individual level<\/u>, focusing on parents\u2019 perspectives about genomic testing with a longitudinal survey and interview data to measure decisional regret, comparing parents who receive a diagnostic result with those who do not. The fourth part will also target an individual level to obtain perspectives of different non-genetic medical specialists about different aspects of implementation including readiness, process, and outcomes using a cross-sectional survey developed with validated measures (acceptability, feasibility, implementation leadership support, and organisational change expectations) and from prior thematic qualitative analysis. Findings from all parts will be integrated to identify overarching findings and recommendations and discussed with key stakeholders.  This paper clearly describes rationale for and objectives of the project. The authors describe a need to understand effects of the relatively newly established Genomic Medicine Service. While some prior work has studied what health care professionals need to integrate genomic medicine into routine care, such as specialized training and access to genomics specialists, gaps exists in our understanding of effective processes and procedures. The authors rationalize their focus on paediatric rare diseases by describing that children make up 50-75% of rare disease cases; they describe a benefit of genomic testing in reducing the \u201cdiagnostic odyssey\u201d and need for more information on effects for parents. They outline four objectives commensurate with each of the four parts of the overall design.  The study design is appropriate for the research questions. Generally, they seek to delineate the process of GMS implementation and resultant outcomes, considering different contextual locations and perspectives. Their overall study design of four parts that include multiple types of methods (e.g., document review, individual interviews, participant observations) and mixing of methods (e.g., integrating qualitative and quantitative data about parent-reported experiences or using qualitative data to create a provider survey) will lead them to this end. They have organized the Detailed Study Plan section by research method or technique. For example, they describe the first part (Study 1) as \u201cImplementation interviews with key stakeholders\u201d. While the methods seem appropriate for the questions of each study part, the protocol may be strengthened by leading with objective rather than method. For example, the subheading for Study 1 could be \u201cSpecify Programme Theory of the GMS\u201d (or something like that). The methods used to achieve that objective would then be described. This change would emphasize the multiple levels of analysis to capture different facets of program implementation and better follow the way it is outlined in the Background and Aims section.  \n<b>Thank you for this overview of the study. As suggested, we have changed the subheadings to lead with objective rather than method.<\/b>  Details of the analysis are clearly described; however, some questions remain:  There could be more specificity about how CFIR will be used: it is very large and typically requires discussion ahead of time to prioritize constructs for specific studies, especially for qualitative interviews.  \n<b>We agree that there is a need to prioritise domains and constructs and these will be identified with input from the advisory team, study aims and the literature. We are planning to use the five overarching domains as main codes to analyse our qualitative data as well as allow themes to emerge inductively. We have now specified this in the manuscript.<\/b>  More detail is also needed about interviews. How will interview questions be developed? There is a CFIR Guide available for free online. Will those questions be used to ask about CFIR constructs? How long will the interviews last and will they be conducted remotely or in-person? The length of interviews matter because it seems like they could be very long if using CFIR, Proctor, literature, etc. I think more detail in this section would be helpful, including stating if key constructs and outcomes will be selected, rather than asking about all of them.  \n<b>We will identify which of the CFIR constructs are most aligned to the aims of the study and interests of the advisory team. The online CFIR guide will be used to help think about the structure of the questions, for those domains\/constructs of interest. They will be used to develop a priori codes for the analysis with subcodes emerging inductively in an iterative process. We are allowing 1 hour for the interviews (Study 1) and these will be conducted remotely using Office Teams. We have provided this additional detail in the manuscript. <\/b>  One minor question is whether the sample size of parents in Study 2 for the qualitative interviews will be sufficient to stratify and compare those who receive a diagnostic result vs. those who do not. This comparison was described in the quan. part but not the qual.  \n<b>I think this point is referring to Study 3 \u2013 we will continue to conduct interviews until saturation is reached.<\/b>  In Study 3, will recruitment site be factored into modelling? The case was made that they offer diversity. It would be helpful to know how site is accounted for, or rationale for not (e.g., maybe education, ethnicity etc. are proxies?).  \n<b>Each survey will be given a unique identifier so that we know which site the responder is from as well as the clinical indication. We are aiming for around 60 completed surveys per site so that comparison across sites can be made. <\/b>\n<b>This detail has been added.<\/b>  In Data Synthesis from All Studies section, please describe what you mean by this sentence: \u201cSummary tables will be developed to identify context-specific barriers and facilitators (or suggested changes) to implementation.\u201d It would be helpful to have a better sense of what the final product might look like. It seems that there was emphasis throughout the paper on using explanatory models and identifying mechanisms, and so the final synthesis might include more about process rather than summary of facilitators and barriers. It may be worthwhile to consider Intervention Mapping or something like that.  \n<b>The summary tables will not only focus on barriers and facilitators, they will also capture the process of implementation, perceptions etc. We will also be mapping the programme theory guiding the intervention at the central and local levels and this will also be included in the summary tables. We have added further details about this in the manuscript.<\/b>  In general, this is a strong paper with a strong rationale and study design. Changes in formatting would help with clarity. Some additional detail would strengthen description of methods and analysis.","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/1-23\/v2#referee-comment-3197","order":3,"name":"referee-comment-3197","label":"Referee Comment","group":{"name":"article-reports","label":"Article Reports"}},{"value":"10.3310\/nihropenres.14352.r28442, Lotte Krabbenborg, Karine Wendrich, Institute for Science in Society, Radboud University, Nijmegen, The Netherlands, 22 Dec 2021, version 1, indexed","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/1-23\/v1#referee-response-28442","order":4,"name":"referee-response-28442","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"<b>Celine Lewis<\/b>; \n<i>Posted: 14 Feb 2022<\/i>; \n<b>We thank both reviewers for their thoughtful and constructive comments on the study protocol. We hope that we have addressed these points to your satisfaction.<\/b>  General: This article presents the study protocol of a mixed-methods study to evaluate the NHS Genomic Medicine Service for paediatric rare diseases. This is a highly relevant study and we like the mixed-methods approach (surveys, interviews, observations) with the inclusion of different stakeholder groups (designers\/implementers, healthcare providers and patients). This way, a variety of important topics and perspectives are covered. The study protocol seems to be very well thought through. However, we would recommend some adaptations to how the protocol is currently presented in the article. According to us, the main points of improvement are a better substantiation of certain decisions and more attention for the broader relevance of this study, as further elaborated in the comments.  Major comments:  At the start and the end of the article, we would like this study to be placed in a broader context than the NHS Genomic Medicine Service alone, in order to show the relevance of this study beyond the NHS GMS. For instance, what could other healthcare programs or the field of implementation science gain from your study? As an example, in the description of study 1, the authors rightly state:  \u201cImplementation science, the systematic study of methods that support the application of research findings and other evidence-based knowledge into policy and practice, is increasingly being seen as playing a critical role health services research47,54. Previous research on new interventions has highlighted that as well as assessing outcomes, it is valuable to look at the process of the intervention as this can shed light on the mechanisms responsible for whether and how successful it is55. This formative work can also enable researchers to suggest ways to answer questions about how interventions might be adapted and respond to change in order to produce positive outcomes56.\u201d  This kind of information should be presented upfront, already in the Introduction, to highlight the importance of the current study. Similarly, in the Discussion section at the end of the article, the authors only talk about the NHS GMS, whereas this would be a good place to discuss how this study can have a broader contribution, beyond the current study case.  \n<b>Thank you for these suggestions. We have moved the section on implementation science into the Intro, and added text around the broader contribution of the study beyond the current study case.<\/b>  The Introduction presents a lot of information. This information is concise and well written, but to us it feels like an overload of information without a clear focus. Related to our previous comment, we suggest the authors to first present some general information about the GM, followed by the relevance and the focus of the current study. Next, the authors could have a \u201cbackground information\u201d section, which discusses the previous research on topics that are relevant for the study.  \n<b>We have made the changes as suggested.<\/b>  As the authors mention, the GMS will be implemented for rare diseases and cancer. Why do the authors only focus on paediatric diseases in this study? We would say that it is highly interesting to compare the implementation of the GMS for (paediatric) rare diseases and cancer. So, why has the decision been made to focus on paediatric rare diseases?  \n<b>Yes, we agree it would be very interesting to compare the implementation of the GMS for rare diseases with cancer. Unfortunately, with our limited funding and resources (1 x PI \u2013 4 years and 1 x research associate \u2013 2 years) it isn\u2019t possible to do this. As this is NIHR Advanced Fellowship, the PI focused on paediatric rare disease as this is her main area of interest and therefore builds on her previous work.<\/b>  Minor comments:  When presenting the research approach and conceptual framework, the authors mention that they use a theory-driven approach. We would like the authors to explain why they have chosen these theories. Why are these the most suitable, given the aim of this study? Moreover, the authors could consider to shortly explain the different domains of The Consolidated Framework for Implementation Research and Proctor\u2019s taxonomy, as now it remains quite abstract.  \n<b>We have stated in the text that we chose the CFIR because it has been used previously to evaluate the implementation of genomic medicine. We have also added that it is suitable to guide development of actionable findings as well as to rapid-cycle evaluations. We have also added in descriptions of the domains.<\/b>  The figures are really nice, providing a clear schematic overview of the study and the different sub-studies. In Figures 1 and 2, we recommend the authors to change the titles of studies 3 and 4. Based on the titles in these figures, it is not clear that study 3 is about parents and study 4 is about healthcare providers. As we see it, both studies are mixed methods with parents and healthcare providers respectively. So, our suggestion is to rename study 3 something like: \u201cmixed methods parental perspective\u201d and study 4: \u201cmixed methods medical specialist perspective\u201d.  \n<b>These have been changed as per Reviewer Nina Sperber\u2019s comments and now include the word parent (study 3) and medical specialists (study 4).<\/b>  Why are the stakeholder interviews and document analysis in study 1 only performed in year 1 and 4 of the study? We can imagine that adding another timepoint in the middle of the study period would allow for gaining more insight into the implementation dynamics.  \n<b>We agree this would be the ideal, however, it is not going to be possible due to resource limitations.<\/b>  Regarding the recruitment of patients and healthcare providers in studies 2, 3 and 4: how will you gain access to these patients and healthcare providers? Will this be done through the principal investigator of each of the seven study sites?  \n<b>The PIs at each recruitment site will support identification of health professionals for study 2 (who will then be tasked with identifying potential patient participants). For study 3, the health professional consenting the patient or the research co-ordinator at each site will identify and send out the survey to potential participants. For study 4, we will adopt a multi-pronged approach to reduce the risk of bias (recruiting through societies, email distribution lists and social media). These recruitment strategies have now been made clearer in the text.&nbsp;<\/b>  We really like the idea of a workshop with key stakeholders at the end of the study to further refine the recommendations for practice. Maybe you can include some additional information about this workshop? What kind of stakeholders will participate, how will they be recruited? Will this workshop also be recorded and analysed for research purposes?  \n<b>We have added further detail around who will be invited to take part in the workshop as well as what will happen during the workshop.<\/b>  In study 3, respondents seem to be exposed to a lot of surveys (nicely summarize in Figure 3), which can place a high burden on the respondents. It is our understanding that the main focus of study 3 is on decisional regret. The authors mention a lot of secondary outcomes. Are all these surveys and secondary outcomes really necessary? You should not just collect as much data as possible, \u201cjust because you can\u201d, but only in case you really need this information.  \n<b>We are currently in the process of piloting the time 1 survey with parents recruited through a support group to ensure the included measures and questions are appropriate and to check the time required to answer the survey. The feedback has been that the survey is clear and answerable and was not considered too long. Many of the survey items were used in our previous study reporting decisions, attitudes and outcomes of patients in the 100,000 Genomes Project [see <\/b>\n<b>https:\/\/www.gimjournal.org\/article\/S1098-3600(21)01124-2\/fulltext#secsectitle0010<\/b>\n<b>] where the response rate was over 50%.<\/b>  \n<b>We will make clear in the Participant Information Sheet the expected time required to complete the survey. All the items in the survey have been identified by our advisory team (including PPI team) as important constructs in our understanding of the experience, impact and outcomes of genomic testing for patients with rare disease. Many have also been used in other studies which will allow us to compare across projects and countries [see for example <\/b>\n<b>https:\/\/pubmed.ncbi.nlm.nih.gov\/31189963\/<\/b>\n<b> ] We have also sought input from our Australian colleagues who are conducting a similar study with patients undergoing genomic testing as to their views on the importance of and experience using the included measures. As a gesture of appreciation for their time, we are also offering all participants a voucher worth &pound;10.<\/b>  The authors state:&nbsp;\u201cPrior to any observation or interview taking place, consent will be sought and recorded, either verbally (if the observation\/interview is taking place virtually) or in written form (if the observation\/interview is taking place face-to-face).\u201d&nbsp;- Our experience is that written consent can also be obtained in the case of a virtual observation\/interview, for instance by sending the consent form to the participant by e-mail. So, we suggest the authors to also obtain written consent for the virtual observations and interviews.  \n<b>We want to make the process of consent as easy as possible for the participant. The HRA does not require written consent for non-clinical trials, and where risk to the participant is minimal. In order to ensure that that consent is valid, each item on the consent form will be read aloud and the participant required to say \u2018yes\u2019 to each item. Verbal consent will be recorded digitally so there is a clear record of the consent process. In addition, it will be recorded electronically and signed\/dated by the researcher and a copy of the electronic record sent to the participant as well as kept in the study file. We have added this additional detail in the manuscript.<\/b>  When addressing these comments, we believe the authors will present a nice article about a highly interesting study protocol. We wish the authors the best of luck with the execution of the study protocol.","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/1-23\/v2#referee-comment-3198","order":5,"name":"referee-comment-3198","label":"Referee Comment","group":{"name":"article-reports","label":"Article Reports"}},{"value":"This research is funded through an NIHR Advanced Fellowship Grant (NIHR300099), awarded to Celine Lewis. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.","order":6,"name":"grant-information","label":"Grant Information"},{"value":"This open access work is licensed under the Open Government Licence v3.0.","order":0,"name":"copyright-info","label":"Copyright"}]}}