{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T18:06:38Z","timestamp":1781114798413,"version":"3.54.1"},"reference-count":33,"publisher":"Public Library of Science (PLoS)","issue":"6","license":[{"start":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T00:00:00Z","timestamp":1781049600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"International Federation for the Surgery of Obesity and Metabolic Disorders"},{"name":"(NIHR) Biomedical Research Centre","award":["BRC-1215-20011"],"award-info":[{"award-number":["BRC-1215-20011"]}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:sec id=\"sec001\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Core outcome sets are an established method for standardising the collection, measurement and reporting of treatment outcomes in effectiveness trials. Using Delphi survey methodology, core sets are developed by prioritising and re-prioritising data items facilitated by provision of feedback of other stakeholders\u2019 responses. It is unknown how best to provide feedback to ensure that it influences the re-prioritisation of items effectively. This study examined whether informing participants of the top-rated items from the previous survey round may influence the re-prioritisation of data items in a subsequent survey round during the development of core data sets.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec002\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This study was nested in the development of a registry core data set. In round two of the Delphi survey, participants were randomised to receive \u2018standard\u2019 or \u2018enhanced\u2019 instructions. \u2018Enhanced\u2019 instructions included summarised data of the top five data items scored by participants in the previous survey round) in addition to standard feedback (the median round 1 score per item). Items scored 7\u20139 by \u226570% of participants in round 2 were considered \u2018prioritised\u2019. Concordant\/discordant items were determined and extent of agreement between groups calculated (kappa statistics).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec003\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Both groups prioritised a larger number of items in round 2 than in round 1 and there was little difference in the percentage of respondents prioritising the \u2018Top 5\u2019 items in round 2 (mean change in prioritisation of Top 5 items for all four core sets combined \u2013 2.3% increase in standard group and 3.2% increase in enhanced group). Overall agreement in data items prioritised by both groups improved in round 2 (discordant items \u2013 11% in round 1 and 4% in round 2).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec id=\"sec004\">\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Providing participants with additional feedback during the process of item prioritisation did not promote prioritisation of items during development of a core set. In the development of health core sets, where often many items are prioritised, further work to determine how to clearly and optimally communicatee feedback in a manner that promotes consensus effectively is required. Specifically, qualitative work with relevant stakeholders, exploring and clarifying the concepts of prioritisation and consensus, is warranted.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1371\/journal.pone.0348136","type":"journal-article","created":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T17:27:07Z","timestamp":1781112427000},"page":"e0348136","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Optimising the Delphi survey method during core set development: The impact of summarised feedback on stakeholders\u2019 prioritisation of core data items and consensus"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4923-3000","authenticated-orcid":true,"given":"Katy A.","family":"Chalmers","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0510-4290","authenticated-orcid":true,"given":"Karen","family":"Coulman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jane M.","family":"Blazeby","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John","family":"Dixon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lilian","family":"Kow","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronald","family":"Liem","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dimitri J.","family":"Pournaras","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9243-2390","authenticated-orcid":true,"given":"Johan","family":"Ottosson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard","family":"Welbourn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wendy","family":"Brown","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kerry N. L.","family":"Avery","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,6,10]]},"reference":[{"key":"pone.0348136.ref001","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1186\/1745-6215-8-39","article-title":"Standardising outcomes for clinical trials and systematic reviews","volume":"8","author":"M Clarke","year":"2007","journal-title":"Trials"},{"key":"pone.0348136.ref002","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.jclinepi.2023.03.011","article-title":"A systematic review of studies reporting the development of core outcome sets for use in routine care","volume":"158","author":"A Kearney","year":"2023","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"pone.0348136.ref003","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1186\/s13063-019-3230-x","article-title":"Core Outcome Set-STAndardised Protocol Items: the COS-STAP Statement","volume":"20","author":"JJ Kirkham","year":"2019","journal-title":"Trials"},{"issue":"3","key":"pone.0348136.ref004","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1186\/s13063-017-1978-4","article-title":"The COMET Handbook: version 1.0","volume":"18","author":"PR Williamson","year":"2017","journal-title":"Trials"},{"issue":"11","key":"pone.0348136.ref005","doi-asserted-by":"crossref","first-page":"2738","DOI":"10.1007\/s11695-016-2186-0","article-title":"A Comparison of Health Professionals\u2019 and Patients\u2019 Views of the Importance of Outcomes of Bariatric Surgery","volume":"26","author":"KD Coulman","year":"2016","journal-title":"Obes Surg"},{"issue":"1","key":"pone.0348136.ref006","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1186\/s13063-016-1479-x","article-title":"Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development","volume":"17","author":"ST Brookes","year":"2016","journal-title":"Trials"},{"key":"pone.0348136.ref007","unstructured":"COMET Initiative. https:\/\/comet-initiative.org\/Studies\/SearchResults. Accessed 2024 January 30."},{"issue":"1","key":"pone.0348136.ref008","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1186\/s13063-023-07442-6","article-title":"Rating versus ranking in a Delphi survey: a randomized controlled trial","volume":"24","author":"C Del Grande","year":"2023","journal-title":"Trials"},{"issue":"5","key":"pone.0348136.ref009","doi-asserted-by":"crossref","first-page":"1463","DOI":"10.1007\/s11695-023-06545-y","article-title":"Development of a Bariatric Surgery Core Data Set for an International Registry","volume":"33","author":"KD Coulman","year":"2023","journal-title":"Obes Surg"},{"issue":"7","key":"pone.0348136.ref010","doi-asserted-by":"crossref","first-page":"3031","DOI":"10.1007\/s11695-021-05359-0","article-title":"National Bariatric Surgery Registries: an International Comparison","volume":"31","author":"EO Akpinar","year":"2021","journal-title":"Obes Surg"},{"key":"pone.0348136.ref011","doi-asserted-by":"crossref","first-page":"103208","DOI":"10.1016\/j.jbi.2019.103208","article-title":"The REDCap consortium: Building an international community of software platform partners","volume":"95","author":"PA Harris","year":"2019","journal-title":"J Biomed Inform"},{"issue":"2","key":"pone.0348136.ref012","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1016\/j.jbi.2008.08.010","article-title":"Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support","volume":"42","author":"PA Harris","year":"2009","journal-title":"J Biomed Inform"},{"issue":"4","key":"pone.0348136.ref013","doi-asserted-by":"crossref","first-page":"700","DOI":"10.1097\/SLA.0000000000002204","article-title":"Development of a Core Outcome Set for Clinical Effectiveness Trials in Esophageal Cancer Resection Surgery","volume":"267","author":"KNL Avery","year":"2018","journal-title":"Ann Surg"},{"issue":"2","key":"pone.0348136.ref014","doi-asserted-by":"crossref","first-page":"238","DOI":"10.1097\/SLA.0000000000004975","article-title":"A Core Outcome Set for Seamless, Standardized Evaluation of Innovative Surgical Procedures and Devices (COHESIVE): A Patient and Professional Stakeholder Consensus Study","volume":"277","author":"KNL Avery","year":"2023","journal-title":"Ann Surg"},{"issue":"11","key":"pone.0348136.ref015","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pmed.1002447","article-title":"Core Outcome Set-STAndards for Development: The COS-STAD recommendations","volume":"14","author":"JJ Kirkham","year":"2017","journal-title":"PLoS Med"},{"issue":"10","key":"pone.0348136.ref016","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pmed.1002148","article-title":"Core Outcome Set-STAndards for Reporting: The COS-STAR Statement","volume":"13","author":"JJ Kirkham","year":"2016","journal-title":"PLoS Med"},{"issue":"11","key":"pone.0348136.ref017","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pmed.1002187","article-title":"A Core Outcome Set for the Benefits and Adverse Events of Bariatric and Metabolic Surgery: The BARIACT Project","volume":"13","author":"KD Coulman","year":"2016","journal-title":"PLoS Med"},{"issue":"5","key":"pone.0348136.ref018","first-page":"360","article-title":"Understanding interobserver agreement: the kappa statistic","volume":"37","author":"AJ Viera","year":"2005","journal-title":"Fam Med"},{"key":"pone.0348136.ref019","volume-title":"Stata Statistical Software: Release 17","author":"StataCorp","year":"2021"},{"issue":"1","key":"pone.0348136.ref020","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1186\/s13063-017-2405-6","article-title":"Impact of question order on prioritisation of outcomes in the development of a core outcome set: a randomised controlled trial","volume":"19","author":"ST Brookes","year":"2018","journal-title":"Trials"},{"issue":"1","key":"pone.0348136.ref021","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1186\/s12874-020-0912-8","article-title":"Comparison of different rating scales for the use in Delphi studies: different scales lead to different consensus and show different test-retest reliability","volume":"20","author":"T Lange","year":"2020","journal-title":"BMC Med Res Methodol"},{"issue":"1","key":"pone.0348136.ref022","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1186\/s12874-020-01197-3","article-title":"Methodology in core outcome set (COS) development: the impact of patient interviews and using a 5-point versus a 9-point Delphi rating scale on core outcome selection in a COS development study","volume":"21","author":"A Remus","year":"2021","journal-title":"BMC Med Res Methodol"},{"issue":"11","key":"pone.0348136.ref023","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2019-032338","article-title":"Participating in core outcome set development via Delphi surveys: qualitative interviews provide pointers to inform guidance","volume":"9","author":"AM Biggane","year":"2019","journal-title":"BMJ Open"},{"key":"pone.0348136.ref024","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.jclinepi.2018.06.007","article-title":"A survey of Delphi panelists after core outcome set development revealed positive feedback and methods to facilitate panel member participation","volume":"102","author":"AE Turnbull","year":"2018","journal-title":"J Clin Epidemiol"},{"key":"pone.0348136.ref025","unstructured":"COMET Initiative. https:\/\/comet-initiative.org\/Resources\/PlainLanguage. Accessed 2024 February 15."},{"issue":"7","key":"pone.0348136.ref026","article-title":"Recruiting and retaining participants in e-Delphi surveys for core outcome set development: Evaluating the COMiT\u2019ID study","volume":"13","author":"DA Hall","year":"2018","journal-title":"PLoS One"},{"key":"pone.0348136.ref027","doi-asserted-by":"crossref","unstructured":"Lang KM, de Waal E, Bereczky T, Harrison K, Geissler J, Baolanos N. Delphi hackathon \u2013 a new approach to develop Core Outcome Sets for blood cancers. In: HARMONY Alliance. 2023;1\u20138.","DOI":"10.21203\/rs.3.rs-3961968\/v1"},{"key":"pone.0348136.ref028","unstructured":"James Lind Alliance. https:\/\/www.jla.nihr.ac.uk\/. Accessed 2025 May 22."},{"issue":"6","key":"pone.0348136.ref029","doi-asserted-by":"crossref","DOI":"10.1016\/S2214-109X(25)00059-2","article-title":"Top ten research priorities in global burns care: findings from the james lind alliance global burns research priority setting partnership","volume":"13","author":"HS Richards","year":"2025","journal-title":"Lancet Glob Health"},{"key":"pone.0348136.ref030","unstructured":"JLA Guidebook. https:\/\/www.jla.nihr.ac.uk\/jla-guidebook 2025."},{"issue":"1","key":"pone.0348136.ref031","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1186\/s13063-023-07285-1","article-title":"How to reach agreement: the impact of different analytical approaches to Delphi process results in core outcomes set development","volume":"24","author":"J Webbe","year":"2023","journal-title":"Trials"},{"issue":"8","key":"pone.0348136.ref032","article-title":"Core outcomes for colorectal cancer surgery: a consensus study","volume":"13","author":"AGK McNair","year":"2016","journal-title":"PLoS Med"},{"issue":"11","key":"pone.0348136.ref033","doi-asserted-by":"crossref","first-page":"1360","DOI":"10.1002\/bjs.9883","article-title":"Development of a core outcome set for research and audit studies in reconstructive breast surgery","volume":"102","author":"BRAVO Steering Group","year":"2015","journal-title":"Br J Surg"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0348136","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T17:27:16Z","timestamp":1781112436000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0348136"}},"subtitle":[],"editor":[{"given":"Sulemana","family":"Bankuoru Egala","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,6,10]]},"references-count":33,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2026,6,10]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0348136","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,10]]}}}