{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,14]],"date-time":"2025-05-14T04:22:42Z","timestamp":1747196562038,"version":"3.40.5"},"reference-count":32,"publisher":"Public Library of Science (PLoS)","issue":"10","license":[{"start":{"date-parts":[[2024,10,24]],"date-time":"2024-10-24T00:00:00Z","timestamp":1729728000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"FCT \u2013 Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia , I.P., within Cintesis, R&D Unit","award":["UIDB\/4255\/2020"],"award-info":[{"award-number":["UIDB\/4255\/2020"]}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:sec id=\"sec001\">\n<jats:title>Objective<\/jats:title>\n<jats:p>To describe a protocol to achieve consensus on valid and important indicators to assess primary health care (PHC) quality regarding all clinical contexts of PHC in European health systems.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec002\">\n<jats:title>Study design<\/jats:title>\n<jats:p>Qualitative study using the Delphi technique to gain consensus among European panels of experts comprising a heterogeneous professional background.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec003\">\n<jats:title>Methods<\/jats:title>\n<jats:p>Potential candidate indicators were extracted and translated according to a set of informative elements (<jats:italic>i<\/jats:italic>.<jats:italic>e<\/jats:italic>., name, description, formula, unit of analysis, and sources). This list was then independently reviewed, and duplicates were removed totaling 1726 indicators. To guarantee a good response rate, indicators were distributed across 57 Delphi panels organized by clinical context. Each panel is a Delphi process, assessing between 23 to 33 indicators. Experts\u2019 opinions on the validity and importance of the extracted indicators will be obtained through two rounds of online questionnaires, using a 9-point Likert scale and free-text boxes. To prevent biased responses, participation will be anonymous to other participants and to the team administrating panels. Consensus will be considered if at least 70% of ratings (\u22657 assuming 10 participants) lie within the 7\u20139 range and less than 15% of ratings (&lt;2 assuming 10 participants) are in the 1\u20133 range. Analysis of results will be streamlined and generalizable across panels using scripts.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec004\">\n<jats:title>Conclusions<\/jats:title>\n<jats:p>This protocol will contribute to improve the quality of PHC in Europe by achieving a consensual and concise list of PHC quality indicators retrieved from the scientific literature that fit current clinical guidelines and populations\u2019 needs in countries from the European region according to the World Health Organization.<\/jats:p>\n<\/jats:sec>","DOI":"10.1371\/journal.pone.0309395","type":"journal-article","created":{"date-parts":[[2024,10,24]],"date-time":"2024-10-24T17:24:11Z","timestamp":1729790651000},"page":"e0309395","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Selection of primary health care quality indicators in Europe: A Delphi study protocol"],"prefix":"10.1371","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3890-7735","authenticated-orcid":true,"given":"Mariana","family":"Lobo","sequence":"first","affiliation":[]},{"given":"Andreia","family":"Pinto","sequence":"additional","affiliation":[]},{"given":"Gl\u00f3ria","family":"Concei\u00e7\u00e3o","sequence":"additional","affiliation":[]},{"given":"Sara","family":"Escadas","sequence":"additional","affiliation":[]},{"given":"Adriane Mesquita","family":"de Medeiros","sequence":"additional","affiliation":[]},{"given":"Bruna","family":"Dias","sequence":"additional","affiliation":[]},{"given":"Marta","family":"Sousa Pinto","sequence":"additional","affiliation":[]},{"given":"Em\u00edlia","family":"Pinto","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8099-3043","authenticated-orcid":true,"given":"Andr\u00e9","family":"Ramalho","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8290-5297","authenticated-orcid":true,"given":"Ant\u00f3nio","family":"Pereira","sequence":"additional","affiliation":[]},{"given":"Manuel","family":"Gon\u00e7alves Pinho","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2018-6573","authenticated-orcid":true,"given":"Pedro","family":"Castro","sequence":"additional","affiliation":[]},{"given":"Vera","family":"Pinheiro","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2362-5527","authenticated-orcid":true,"given":"Paulo","family":"Santos","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4696-1002","authenticated-orcid":true,"given":"Jo\u00e3o Vasco","family":"Santos","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2113-9653","authenticated-orcid":true,"given":"Alberto","family":"Freitas","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2024,10,24]]},"reference":[{"key":"pone.0309395.ref001","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12961-019-0456-8","article-title":"The impact of governance in primary health care delivery: a systems thinking approach with a European panel","volume":"17","author":"AB Espinosa-Gonz\u00e1lez","year":"2019","journal-title":"Health research policy and systems"},{"key":"pone.0309395.ref002","doi-asserted-by":"crossref","first-page":"576","DOI":"10.1007\/s11606-015-3311-9","article-title":"Disentangling the linkage of primary care features to patient outcomes: a review of current literature, data sources, and measurement needs","volume":"30","author":"AS O\u2019Malley","year":"2015","journal-title":"Journal of general internal medicine"},{"key":"pone.0309395.ref003","unstructured":"WHO. A vision for primary health care in the 21st century: towards universal health coverage and the Sustainable Development Goals. World Health Organization, 2018. https:\/\/iris.who.int\/bitstream\/handle\/10665\/328065\/WHO-HIS-SDS-2018.15-eng.pdf"},{"key":"pone.0309395.ref004","unstructured":"OECD. Realising the potential of primary health care. OECD Publishing, 2020."},{"key":"pone.0309395.ref005","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1093\/intqhc\/mzg081","article-title":"Defining and classifying clinical indicators for quality improvement","volume":"15","author":"J Mainz","year":"2003","journal-title":"International journal for quality in health care"},{"key":"pone.0309395.ref006","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1177\/152715440100200312","article-title":"Institute of Medicine report: crossing the quality chasm: a new health care system for the 21st century","volume":"2","author":"A. Wolfe","year":"2001","journal-title":"Policy, Politics, & Nursing Practice"},{"key":"pone.0309395.ref007","unstructured":"Eckhardt H, Smith P, Quentin W. Pay for Quality: using financial incentives to improve quality of care. In: Busse R, Klazinga N, Panteli D, et al., editors. Health Policy Series, No. 53. Copenhagen (Denmark): European Observatory on Health Systems and Policies. Improving healthcare quality in Europe: Characteristics, effectiveness and implementation of different strategies; 2019. pp: 357\u2013396."},{"key":"pone.0309395.ref008","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1093\/intqhc\/mzn008","article-title":"An international review of projects on hospital performance assessment","volume":"20","author":"O Groene","year":"2008","journal-title":"International journal for quality in health care"},{"key":"pone.0309395.ref009","unstructured":"OECD. Policy Brief: Realising the potential of primary health care.. 2019. https:\/\/www.oecd.org\/health\/health-systems\/OECD-Policy-Brief-Primary-Health-Care-May-2019.pdf"},{"key":"pone.0309395.ref010","doi-asserted-by":"crossref","first-page":"e0220888","DOI":"10.1371\/journal.pone.0220888","article-title":"Primary health care quality indicators: an umbrella review","volume":"14","author":"A Ramalho","year":"2019","journal-title":"PloS one"},{"key":"pone.0309395.ref011","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1136\/qshc.7.3.130","article-title":"Indicators of the appropriateness of long-term prescribing in general practice in the United Kingdom: consensus development, face and content validity, feasibility, and reliability","volume":"7","author":"JA Cantrill","year":"1998","journal-title":"BMJ Quality & Safety"},{"key":"pone.0309395.ref012","doi-asserted-by":"crossref","first-page":"e004625","DOI":"10.1136\/bmjopen-2013-004625","article-title":"Development of evidence-based Australian medication-related indicators of potentially preventable hospitalisations: a modified RAND appropriateness method","volume":"4","author":"GE Caughey","year":"2014","journal-title":"BMJ open"},{"key":"pone.0309395.ref013","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1093\/oxfordjournals.intqhc.a002610","article-title":"Preventing drug-related morbidity\u2014determining valid indicators","volume":"14","author":"C Morris","year":"2002","journal-title":"International Journal for Quality in Health Care"},{"key":"pone.0309395.ref014","first-page":"509","article-title":"Quality indicators for the prevention and management of cardiovascular disease in primary care in nine European countries","volume":"15","author":"SM Campbell","year":"2008","journal-title":"European Journal of Preventive Cardiology"},{"key":"pone.0309395.ref015","unstructured":"WONCA. An Introduction to the International Classification of Primary Care\u2014Version 2.http:\/\/www.ph3c.org\/PH3C\/docs\/27\/000098\/0000054.pdf"},{"key":"pone.0309395.ref016","doi-asserted-by":"crossref","first-page":"e20476","DOI":"10.1371\/journal.pone.0020476","article-title":"Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review","volume":"6","author":"R Boulkedid","year":"2011","journal-title":"PloS one"},{"key":"pone.0309395.ref017","doi-asserted-by":"crossref","first-page":"1008","DOI":"10.1046\/j.1365-2648.2000.t01-1-01567.x","article-title":"Research guidelines for the Delphi survey technique","volume":"32","author":"F Hasson","year":"2000","journal-title":"Journal of advanced nursing"},{"key":"pone.0309395.ref018","unstructured":"Helmer O. Analysis of the future: The Delphi method. Rand Corp Santa Monica CA, 1967. https:\/\/mn.gov\/web\/prod\/static\/mnddc\/live\/parallels2\/pdf\/70s\/72\/72-MRA-RRT.pdf#page=11"},{"key":"pone.0309395.ref019","doi-asserted-by":"crossref","first-page":"458","DOI":"10.1287\/mnsc.9.3.458","article-title":"An experimental application of the Delphi method to the use of experts","volume":"9","author":"N Dalkey","year":"1963","journal-title":"Management science"},{"key":"pone.0309395.ref020","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.techfore.2017.01.023","article-title":"Real-Time Delphi in practice\u2014A comparative analysis of existing software-based tools","volume":"118","author":"S Aengenheyster","year":"2017","journal-title":"Technological Forecasting and Social Change"},{"key":"pone.0309395.ref021","doi-asserted-by":"crossref","DOI":"10.1155\/2016\/2516765","article-title":"Quality improvement in critical care: selection and development of quality indicators","volume":"2016","author":"CA Chrusch","year":"2016","journal-title":"Canadian respiratory journal"},{"key":"pone.0309395.ref022","unstructured":"Marshall M, Leatherman S, Mattke S. Selecting indicators for the quality of health promotion, prevention and primary care at the health systems level in OECD countries. 2004."},{"key":"pone.0309395.ref023","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1136\/qhc.11.4.358","article-title":"Research methods used in developing and applying quality indicators in primary care","volume":"11","author":"SM Campbell","year":"2002","journal-title":"Quality and Safety in Health Care"},{"key":"pone.0309395.ref024","doi-asserted-by":"crossref","first-page":"333","DOI":"10.3390\/ijerph13030333","article-title":"Indicators for monitoring water, sanitation, and hygiene: a systematic review of indicator selection methods","volume":"13","author":"S Schwemlein","year":"2016","journal-title":"International journal of environmental research and public health"},{"key":"pone.0309395.ref025","unstructured":"Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR. The RAND\/UCLA appropriateness method user\u2019s manual. Rand Corp Santa Monica CA, 2001. https:\/\/apps.dtic.mil\/sti\/tr\/pdf\/ADA393235.pdf"},{"key":"pone.0309395.ref026","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1745-6215-13-176","article-title":"Developing a guideline for clinical trial protocol content: Delphi consensus survey","volume":"13","author":"JM Tetzlaff","year":"2012","journal-title":"Trials"},{"key":"pone.0309395.ref027","doi-asserted-by":"crossref","first-page":"1681","DOI":"10.1016\/j.techfore.2011.04.006","article-title":"Validating an innovative real-time Delphi approach-A methodological comparison between real-time and conventional Delphi studies","volume":"78","author":"T Gnatzy","year":"2011","journal-title":"Technological Forecasting and Social Change"},{"key":"pone.0309395.ref028","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1191\/1478088706qp063oa","article-title":"Using thematic analysis in psychology","volume":"3","author":"V Braun","year":"2006","journal-title":"Qualitative research in psychology"},{"key":"pone.0309395.ref029","doi-asserted-by":"crossref","first-page":"337","DOI":"10.2105\/AJPH.62.3.337","article-title":"The nominal group as a research instrument for exploratory health studies","volume":"62","author":"AH Van de Ven","year":"1972","journal-title":"American journal of public health"},{"key":"pone.0309395.ref030","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1136\/qshc.2007.023226","article-title":"Developing quality indicators for general practice care for vulnerable elders; transfer from US to The Netherlands","volume":"17","author":"E van der Ploeg","year":"2008","journal-title":"BMJ Quality & Safety"},{"key":"pone.0309395.ref031","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.im.2003.11.002","article-title":"The Delphi method as a research tool: an example, design considerations and applications","volume":"42","author":"C Okoli","year":"2004","journal-title":"Information & management"},{"key":"pone.0309395.ref032","first-page":"e912","article-title":"Internet versus mailed questionnaires: a controlled comparison (2)","volume":"6","author":"P Leece","year":"2004","journal-title":"Journal of medical Internet research"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0309395","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,24]],"date-time":"2024-10-24T17:24:25Z","timestamp":1729790665000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0309395"}},"subtitle":[],"editor":[{"given":"Vivian Aline","family":"Mininel","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2024,10,24]]},"references-count":32,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2024,10,24]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0309395","relation":{},"ISSN":["1932-6203"],"issn-type":[{"type":"electronic","value":"1932-6203"}],"subject":[],"published":{"date-parts":[[2024,10,24]]}}}