{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T18:05:00Z","timestamp":1789668300200,"version":"4.0.1"},"reference-count":55,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T00:00:00Z","timestamp":1789603200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"SANBA Metaprogram"},{"name":"MOSBA project"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>Over recent years, antimicrobial use (AMU) in pig farming has substantially declined across Europe, including France, following regulatory measures and sector-wide initiatives. Maintaining these reductions while preserving pig health and welfare remains a major challenge. Stakeholders have therefore highlighted the need for a structured on-farm health monitoring protocol to support judicious AMU. However, no comprehensive set of pig health indicators has yet been systematically assessed and selected for this purpose. Expert elicitation offers a promising approach to address this gap. This study aimed to develop and compare two participatory methods for mobilizing expert knowledge to identify and characterize relevant pig health indicators while making efficient use of experts\u2019 time. Both methods followed the same six-step framework but differed in their implementation. In each method, experts assessed indicators according to four attributes (feasibility, reliability, specificity, and sensitivity) and subsequently discussed the results to reach agreement on a final set of relevant indicators. Porcine respiratory health was used as a proof-of-concept to compare the performance and outputs of the two methods and select the most appropriate one. Both methods enabled experts to reach agreement and generated well-characterized indicators, supported by experts\u2019 rationales. Although Method B required greater expert involvement, it produced a more comprehensive set of 52 relevant respiratory health indicators and allowed greater flexibility in how experts allocated their time throughout the process. This participatory approach method provides a practical framework that encourages expert involvement and allows the selection and characterization of relevant pig health indicators. The method will be extended to other pig health systems beyond respiratory health.<\/jats:p>","DOI":"10.1371\/journal.pone.0358467","type":"journal-article","created":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T17:22:52Z","timestamp":1789665772000},"page":"e0358467","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Comparative assessment of two participatory approach methods for selecting pig health indicators: A proof-of-concept on respiratory health"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-0474-6387","authenticated-orcid":true,"given":"Th\u00e9o","family":"Dessier","sequence":"first","affiliation":[{"name":"Oniris INRAE, BIOEPAR","place":["Nantes, France"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nikky","family":"Millar","sequence":"additional","affiliation":[{"name":"Oniris INRAE, BIOEPAR","place":["Nantes, France"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Em\u00e9lie","family":"Suazo","sequence":"additional","affiliation":[{"name":"Oniris INRAE, BIOEPAR","place":["Nantes, France"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"C\u00e9line","family":"Tallet","sequence":"additional","affiliation":[{"name":"PEGASE, INRAE, Institut Agro Rennes Angers","place":["Rennes, France"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6739-7319","authenticated-orcid":true,"given":"Catherine","family":"Belloc","sequence":"additional","affiliation":[{"name":"Oniris INRAE, BIOEPAR","place":["Nantes, France"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mily","family":"Leblanc-Maridor","sequence":"additional","affiliation":[{"name":"Oniris INRAE, BIOEPAR","place":["Nantes, France"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,9,17]]},"reference":[{"issue":"2","key":"pone.0358467.ref001","doi-asserted-by":"crossref","first-page":"52","DOI":"10.3390\/antibiotics9020052","article-title":"Antimicrobial resistance and food animals: influence of livestock environment on the emergence and dissemination of antimicrobial resistance","volume":"9","author":"N Vidovic","year":"2020","journal-title":"Antibiotics (Basel)"},{"issue":"1","key":"pone.0358467.ref002","doi-asserted-by":"crossref","DOI":"10.1057\/s41599-018-0152-2","article-title":"Pharming animals: a global history of antibiotics in food production (1935\u20132017)","volume":"4","author":"C Kirchhelle","year":"2018","journal-title":"Palgrave Commun"},{"issue":"18","key":"pone.0358467.ref003","doi-asserted-by":"crossref","first-page":"5649","DOI":"10.1073\/pnas.1503141112","article-title":"Global trends in antimicrobial use in food animals","volume":"112","author":"TP Van Boeckel","year":"2015","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pone.0358467.ref004","doi-asserted-by":"crossref","first-page":"452","DOI":"10.3389\/fvets.2019.00452","article-title":"Raising animals without antibiotics: U.S. producer and veterinarian experiences and opinions","volume":"6","author":"RS Singer","year":"2019","journal-title":"Front Vet Sci"},{"issue":"11","key":"pone.0358467.ref005","article-title":"A participatory approach for building ex ante impact pathways towards a prudent use of antimicrobials in pig and poultry sectors in France","volume":"17","author":"M-J Guenin","year":"2022","journal-title":"PLoS One"},{"issue":"10","key":"pone.0358467.ref006","doi-asserted-by":"crossref","first-page":"1370","DOI":"10.3390\/antibiotics11101370","article-title":"Towards a better use of antimicrobials on farms: insights from a participatory approach in the french pig and poultry sectors","volume":"11","author":"C Ducrot","year":"2022","journal-title":"Antibiotics (Basel)"},{"issue":"11","key":"pone.0358467.ref007","doi-asserted-by":"crossref","first-page":"1817","DOI":"10.1016\/j.jand.2018.09.005","article-title":"Establishing validity and cross-context equivalence of measures and indicators","volume":"119","author":"EA Frongillo","year":"2019","journal-title":"J Acad Nutr Diet"},{"key":"pone.0358467.ref008","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.ecolind.2011.04.034","article-title":"A method to assess the relevance of sustainability indicators: application to the indicator set of the Czech Republic\u2019s Sustainable Development Strategy","volume":"17","author":"T Hak","year":"2012","journal-title":"Ecol Indic"},{"key":"pone.0358467.ref009","doi-asserted-by":"crossref","first-page":"41","DOI":"10.3389\/fvets.2018.00041","article-title":"Connecting different data sources to assess the interconnections between biosecurity, health, welfare, and performance in commercial pig farms in great britain","volume":"5","author":"F Pandolfi","year":"2018","journal-title":"Front Vet Sci"},{"key":"pone.0358467.ref010","volume-title":"Glossary of Key Terms in Evaluation and Results-based Management in Sustainable Development","year":"2023"},{"issue":"6","key":"pone.0358467.ref011","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0020476","article-title":"Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review","volume":"6","author":"R Boulkedid","year":"2011","journal-title":"PLoS One"},{"key":"pone.0358467.ref012","year":"2022"},{"issue":"12","key":"pone.0358467.ref013","doi-asserted-by":"crossref","first-page":"3450","DOI":"10.3390\/ani11123450","article-title":"Welfare of farmed crocodilians: identification of potential animal-based measures using elicitation of expert opinion","volume":"11","author":"L Hewitt","year":"2021","journal-title":"Animals (Basel)"},{"issue":"1","key":"pone.0358467.ref014","doi-asserted-by":"crossref","first-page":"10249","DOI":"10.1038\/s41598-019-45810-y","article-title":"Validation of mouse welfare indicators: a Delphi consultation survey","volume":"9","author":"I Campos-Luna","year":"2019","journal-title":"Sci Rep"},{"issue":"1","key":"pone.0358467.ref015","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0309760","article-title":"An E-Delphi study to facilitate animal welfare assessment in Italian zoos and aquaria","volume":"20","author":"B de Mori","year":"2025","journal-title":"PLoS One"},{"issue":"1","key":"pone.0358467.ref016","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1186\/s12889-018-5463-0","article-title":"Indicators for evaluating European population health: a Delphi selection process","volume":"18","author":"\u00c2 Freitas","year":"2018","journal-title":"BMC Public Health"},{"key":"pone.0358467.ref017","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1017\/S0962728600025641","article-title":"Animal-based measures for the assessment of welfare state of dairy cattle, pigs and laying hens: consensus of expert opinion","volume":"12","author":"HR Whay","year":"2003","journal-title":"Anim Welf"},{"issue":"6","key":"pone.0358467.ref018","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1017\/S1751731110002594","article-title":"Validating indicators of sheep welfare through a consensus of expert opinion","volume":"5","author":"CJ Phythian","year":"2011","journal-title":"Animal"},{"key":"pone.0358467.ref019","volume-title":"Participatory Approaches. Methodological Briefs Impact Evaluation No. 5","author":"IM Guijt","year":"2014"},{"issue":"1","key":"pone.0358467.ref020","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.eiar.2005.06.002","article-title":"Indicators validation for the improvement of environmental and social impact quantitative assessment","volume":"26","author":"V-A Cloquell-Ballester","year":"2006","journal-title":"Environmental Impact Assessment Review"},{"key":"pone.0358467.ref021","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1016\/j.envsci.2019.07.012","article-title":"Cost and value of stakeholders participation: a systematic literature review","volume":"101","author":"M Anggraeni","year":"2019","journal-title":"Environmental Science & Policy"},{"issue":"1","key":"pone.0358467.ref022","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1186\/s40813-023-00342-w","article-title":"The economic impact of endemic respiratory disease in pigs and related interventions - a systematic review","volume":"9","author":"M Boeters","year":"2023","journal-title":"Porcine Health Manag"},{"key":"pone.0358467.ref023","doi-asserted-by":"crossref","DOI":"10.1002\/9781119350927","volume-title":"Diseases of swine","author":"JJ Zimmerman","year":"2019","edition":"11"},{"issue":"2","key":"pone.0358467.ref024","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1111\/j.1365-2648.2006.03716.x","article-title":"Consulting the oracle: ten lessons from using the Delphi technique in nursing research","volume":"53","author":"S Keeney","year":"2006","journal-title":"J Adv Nurs"},{"key":"pone.0358467.ref025","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/j.ijforecast.2010.05.004","article-title":"Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task","volume":"27","author":"A Graefe","year":"2011","journal-title":"Int J Forecast"},{"key":"pone.0358467.ref026","doi-asserted-by":"crossref","first-page":"104991","DOI":"10.1016\/j.prevetmed.2020.104991","article-title":"\u201cCan we agree on that\u201d? Plurality, power and language in participatory research","volume":"180","author":"K Fischer","year":"2020","journal-title":"Prev Vet Med"},{"key":"pone.0358467.ref027","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1177\/1046496405275232","article-title":"Communication medium and member familiarity: the effects on decision time, accuracy, and satisfaction","volume":"36","author":"SJ Adams","year":"2005","journal-title":"Small Group Research"},{"key":"pone.0358467.ref028","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1006\/obhd.1996.0061","article-title":"Group composition and decision making: how member familiarity and information distribution affect process and performance","volume":"67","author":"DH Gruenfeld","year":"1996","journal-title":"Organ Behav Hum Decis Process"},{"key":"pone.0358467.ref029","first-page":"1","article-title":"I know my teammates: the role of Group Member Familiarity in Computer-Supported and face-to-face collaborative learning","author":"S Zhang","year":"2023","journal-title":"Educ Inf Technol (Dordr)"},{"issue":"6","key":"pone.0358467.ref030","doi-asserted-by":"crossref","first-page":"2719","DOI":"10.1007\/s11135-019-00887-5","article-title":"A methodology for cross-national comparative focus group research: illustrations from discussions about political protest","volume":"53","author":"MJ van Bezouw","year":"2019","journal-title":"Qual Quant"},{"issue":"5","key":"pone.0358467.ref031","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1007\/s10726-020-09688-w","article-title":"Comparing structured and unstructured facilitation approaches in consultation workshops: a field experiment","volume":"29","author":"J Honey-Ros\u00e9s","year":"2020","journal-title":"Group Decis Negot"},{"key":"pone.0358467.ref032","first-page":"278","article-title":"Guidance on expert knowledge elicitation in food and feed safety risk assessment","author":"EFSA (European Food Safety Authority)","year":"2014","journal-title":"EFSA Journal"},{"key":"pone.0358467.ref033","doi-asserted-by":"crossref","first-page":"101968","DOI":"10.1016\/j.erss.2021.101968","article-title":"It is best to ask: designing a stakeholder-centric approach to selecting sustainable energy development indicators","volume":"74","author":"I Gunnarsd\u00f3ttir","year":"2021","journal-title":"Energy Research & Social Science"},{"key":"pone.0358467.ref034","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1186\/1476-069X-9-19","article-title":"The use of expert elicitation in environmental health impact assessment: a seven step procedure","volume":"9","author":"AB Knol","year":"2010","journal-title":"Environ Health"},{"key":"pone.0358467.ref035","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1080\/00031305.2018.1518265","article-title":"Expert knowledge elicitation: subjective but scientific","volume":"73","author":"A O\u2019Hagan","year":"2019","journal-title":"Am Stat"},{"issue":"37","key":"pone.0358467.ref036","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3310\/hta25370","article-title":"Developing a reference protocol for structured expert elicitation in health-care decision-making: a mixed-methods study","volume":"25","author":"L Bojke","year":"2021","journal-title":"Health Technol Assess"},{"issue":"3","key":"pone.0358467.ref037","doi-asserted-by":"crossref","DOI":"10.3310\/hta2030","article-title":"Consensus development methods, and their use in clinical guideline development","volume":"2","author":"MK Murphy","year":"1998","journal-title":"Health Technol Assess"},{"issue":"2","key":"pone.0358467.ref038","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1046\/j.1365-2648.1998.00692.x","article-title":"Collecting and analysing qualitative data: issues raised by the focus group","volume":"28","author":"J Sim","year":"1998","journal-title":"J Adv Nurs"},{"issue":"4","key":"pone.0358467.ref039","doi-asserted-by":"crossref","first-page":"116","DOI":"10.5662\/wjm.v11.i4.116","article-title":"Delphi methodology in healthcare research: how to decide its appropriateness","volume":"11","author":"P Nasa","year":"2021","journal-title":"World J Methodol"},{"key":"pone.0358467.ref040","doi-asserted-by":"crossref","first-page":"646458","DOI":"10.3389\/fvets.2021.646458","article-title":"Engaging stakeholders in the design of one health surveillance systems: a participatory approach","volume":"8","author":"M Bordier","year":"2021","journal-title":"Front Vet Sci"},{"issue":"17","key":"pone.0358467.ref041","doi-asserted-by":"crossref","first-page":"1757","DOI":"10.3390\/healthcare12171757","article-title":"Delphi technique on nursing competence studies: a scoping review","volume":"12","author":"L Furtado","year":"2024","journal-title":"Healthcare (Basel)"},{"issue":"2","key":"pone.0358467.ref042","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.evalprogplan.2009.06.006","article-title":"Using the Delphi method to engage stakeholders: a comparison of two studies","volume":"33","author":"MR Geist","year":"2010","journal-title":"Eval Program Plann"},{"key":"pone.0358467.ref043","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1080\/19325037.2012.10599216","article-title":"Advantages and limitations of the e-Delphi technique: implications for health education researchers","volume":"43","author":"H Donohoe","year":"2012","journal-title":"Am J Health Educ"},{"key":"pone.0358467.ref044","doi-asserted-by":"crossref","first-page":"1342","DOI":"10.1177\/0146167299259002","article-title":"Group decision making: the effects of initial preferences and time pressure","volume":"25","author":"JR Kelly","year":"1999","journal-title":"Pers Soc Psychol Bull"},{"key":"pone.0358467.ref045","article-title":"Delphi method: strengths and weaknesses","volume":"16","author":"D Fink-Hafner","year":"2019","journal-title":"Adv Methodol Stat"},{"key":"pone.0358467.ref046","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1016\/S0261-5177(00)00067-4","article-title":"The development of indicators for sustainable tourism: results of a Delphi survey of tourism researchers","volume":"22","author":"G Miller","year":"2001","journal-title":"Tour Manag"},{"key":"pone.0358467.ref047","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511619458","volume-title":"International focus group research: A handbook for the health and social sciences","author":"MM Hennink","year":"2007","edition":"1"},{"key":"pone.0358467.ref048","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1080\/00913367.2016.1252288","article-title":"Online focus groups","volume":"46","author":"DW Stewart","year":"2017","journal-title":"J Advert"},{"key":"pone.0358467.ref049","doi-asserted-by":"crossref","DOI":"10.1177\/16094069221142406","article-title":"A literature review of studies that have compared the use of face-to-face and online focus groups","volume":"21","author":"JE Jones","year":"2022","journal-title":"International Journal of Qualitative Methods"},{"issue":"6","key":"pone.0358467.ref050","doi-asserted-by":"crossref","first-page":"2042533313479197","DOI":"10.1177\/2042533313479197","article-title":"Using mixed methods in health research","volume":"4","author":"S Tariq","year":"2013","journal-title":"JRSM Short Rep"},{"key":"pone.0358467.ref051","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1017\/S1357321716000155","article-title":"Bias, guess and expert judgement in actuarial work","volume":"21","author":"ERW Tredger","year":"2016","journal-title":"Br Actuar J"},{"key":"pone.0358467.ref052","doi-asserted-by":"crossref","first-page":"211","DOI":"10.2147\/JMDH.S104807","article-title":"Information bias in health research: definition, pitfalls, and adjustment methods","volume":"9","author":"A Althubaiti","year":"2016","journal-title":"J Multidiscip Healthc"},{"key":"pone.0358467.ref053","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1016\/j.jesp.2018.02.001","article-title":"The anchoring-bias in groups","volume":"76","author":"TRW De Wilde","year":"2018","journal-title":"J Exp Soc Psychol"},{"issue":"4","key":"pone.0358467.ref054","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1016\/j.prevetmed.2014.12.008","article-title":"Measuring general animal health status: development of an animal health barometer","volume":"118","author":"P Depoorter","year":"2015","journal-title":"Prev Vet Med"},{"issue":"2","key":"pone.0358467.ref055","doi-asserted-by":"crossref","first-page":"167","DOI":"10.3390\/vetsci10020167","article-title":"The importance of the slaughterhouse in surveilling animal and public health: a systematic review","volume":"10","author":"J Garc\u00eda-D\u00edez","year":"2023","journal-title":"Vet Sci"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0358467","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T17:23:02Z","timestamp":1789665782000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0358467"}},"subtitle":[],"editor":[{"given":"Ewa","family":"Tomaszewska","sequence":"first","affiliation":[{"name":"University of Life Sciences in Lublin","place":["POLAND"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,9,17]]},"references-count":55,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9,17]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0358467","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,17]]}}}