{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:27:23Z","timestamp":1760218043359,"version":"3.40.5"},"reference-count":30,"publisher":"Public Library of Science (PLoS)","issue":"10","license":[{"start":{"date-parts":[[2022,10,7]],"date-time":"2022-10-07T00:00:00Z","timestamp":1665100800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/publicdomain\/zero\/1.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:sec id=\"sec001\"><jats:title>Background<\/jats:title><jats:p>Use of a standardized verbal autopsy (VA) questionnaire, such as the World Health Organization (WHO) instrument, can improve the consistency and reliability of the data it collects. Systematically revising a questionnaire, however, requires evidence about the performance of its questions. The purpose of this investigation was to use a mixed methods approach to evaluate the performance of questions related to 14 previously reported issues in the 2016 version of the WHO questionnaire, where there were concerns of potential confusion, redundancy, or inability of the respondent to answer the question. The results from this mixed methods analysis are discussed across common themes that may have contributed to the underperformance of questions and have been compiled to inform decisions around the revision of the current VA instrument.<\/jats:p><\/jats:sec><jats:sec id=\"sec002\"><jats:title>Methods<\/jats:title><jats:p>Quantitative analysis of 19,150 VAs for neonates, children, and adults from five project teams implementing VAs predominately in Sub-Saharan Africa included frequency distributions and cross-tabulations to evaluate response patterns among related questions. The association of respondent characteristics and response patterns was evaluated using prevalence ratios. Qualitative analysis included results from cognitive interviewing, an approach that provides a detailed understanding of the meanings and processes that respondents use to answer interview questions. Cognitive interviews were conducted among 149 participants in Morocco and Zambia. Findings from the qualitative and quantitative analyses were triangulated to identify common themes.<\/jats:p><\/jats:sec><jats:sec id=\"sec003\"><jats:title>Results<\/jats:title><jats:p>Four broad themes contributing to the underperformance or redundancy within the instrument were identified: question sequence, overlap within the question series, questions outside the frame of reference of the respondent, and questions needing clarification. The series of questions associated with one of the 14 identified issues (the series of questions on injuries) related to question sequence; seven (tobacco use, sores, breast swelling, abdominal problem, vomiting, vaccination, and baby size) demonstrated similar response patterns among questions within each series capturing overlapping information. Respondent characteristics, including relationship to the deceased and whether or not the respondent lived with the deceased, were associated with differing frequencies of non-substantive responses in three question series (female health related issues, tobacco use, and baby size). An inconsistent understanding of related constructs was observed between questions related to sores\/ulcers, birth weight\/baby size, and diagnosis of dementia\/presence of mental confusion. An incorrect association of the intended construct with that which was interpreted by the respondent was observed in the medical diagnosis question series.<\/jats:p><\/jats:sec><jats:sec id=\"sec004\"><jats:title>Conclusions<\/jats:title><jats:p>In this mixed methods analysis, we identified series of questions which could be shortened through elimination of redundancy, series of questions requiring clarification due to unclear constructs, and the impact of respondent characteristics on the quality of responses. These changes can lead to a better understanding of the question constructs by the respondents, increase the acceptance of the tool, and improve the overall accuracy of the VA instrument.<\/jats:p><\/jats:sec>","DOI":"10.1371\/journal.pone.0274304","type":"journal-article","created":{"date-parts":[[2022,10,7]],"date-time":"2022-10-07T17:29:24Z","timestamp":1665163764000},"page":"e0274304","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["Mixed-methods analysis of select issues reported in the 2016 World Health Organization verbal autopsy questionnaire"],"prefix":"10.1371","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4917-1412","authenticated-orcid":true,"given":"Erin","family":"Nichols","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kristen","family":"Pettrone","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Brent","family":"Vickers","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hermon","family":"Gebrehiwet","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Clarissa","family":"Surek-Clark","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jordana","family":"Leitao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Agbessi","family":"Amouzou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dianna M.","family":"Blau","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Debbie","family":"Bradshaw","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"El Marnissi","family":"Abdelilah","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pamela","family":"Groenewald","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Brian","family":"Munkombwe","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chomba","family":"Mwango","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"F. Sam","family":"Notzon","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Steve","family":"Biko Odhiambo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul","family":"Scanlon","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2022,10,7]]},"reference":[{"key":"pone.0274304.ref001","unstructured":"World Health Organization. Verbal autopsy standards: ascertaining and attributing causes of death. The 2016 WHO verbal autopsy instrument https:\/\/www.who.int\/standards\/classifications\/other-classifications\/verbal-autopsy-standards-ascertaining-and-attributing-causes-of-death-tool."},{"key":"pone.0274304.ref002","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1186\/1741-7015-12-22","article-title":"Comparison of physician-certified verbal autopsy with computer-coded verbal autopsy for cause of death assignment in hospitalized patients in low- and middle-income countries: systematic review","volume":"12","author":"J. Leitao","year":"2014","journal-title":"BMC Med"},{"issue":"2","key":"pone.0274304.ref003","doi-asserted-by":"crossref","first-page":"e000639","DOI":"10.1136\/bmjgh-2017-000639","article-title":"Verbal autopsy in health policy and systems: a literature review","volume":"3","author":"L.M. Thomas","year":"2018","journal-title":"BMJ Glob Health"},{"key":"pone.0274304.ref004","doi-asserted-by":"crossref","first-page":"21518","DOI":"10.3402\/gha.v6i0.21518","article-title":"Revising the WHO verbal autopsy instrument to facilitate routine cause-of-death monitoring","volume":"6","author":"J. Leitao","year":"2013","journal-title":"Global health action"},{"issue":"1","key":"pone.0274304.ref005","doi-asserted-by":"crossref","first-page":"e1002486","DOI":"10.1371\/journal.pmed.1002486","article-title":"The WHO 2016 verbal autopsy instrument: An international standard suitable for automated analysis by InterVA, InSilicoVA, and Tariff 2.0","volume":"15","author":"E.K. Nichols","year":"2018","journal-title":"PLoS medicine"},{"key":"pone.0274304.ref006","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1186\/1741-7015-12-18","article-title":"Prospects for automated diagnosis of verbal autopsies","volume":"12","author":"M. Garenne","year":"2014","journal-title":"BMC Med"},{"key":"pone.0274304.ref007","doi-asserted-by":"crossref","DOI":"10.1002\/0471654728","volume-title":"Methods for Testing and Evaluating Survey Questionnaires","author":"S. Presser","year":"2004"},{"key":"pone.0274304.ref008","volume-title":"Survey Research Methods","author":"E. Babbie","year":"1990","edition":"2"},{"key":"pone.0274304.ref009","doi-asserted-by":"crossref","DOI":"10.4135\/9781412983655","volume-title":"Cognitive Interviewing: A Tool for Improving Questionnaire Design","author":"G. Willis","year":"2005"},{"key":"pone.0274304.ref010","volume-title":"Survey Methodology","author":"R.M. Groves","year":"2009","edition":"2"},{"key":"pone.0274304.ref011","unstructured":"World Health Organization. Review of the WHO Verbal Autopsy (VA) Instruments Meeting Report. 2011; https:\/\/www.who.int\/healthinfo\/statistics\/WHO_VerbalAutopsy_MeetingReport_Dec2012.pdf?ua=1."},{"key":"pone.0274304.ref012","first-page":"423","volume-title":"Innovations in Mixed Methods Evaluations","author":"L.A. Palinkas","year":"2019"},{"issue":"1","key":"pone.0274304.ref013","first-page":"29","volume":"35","author":"P. Pluye","year":"2014","journal-title":"Combining the Power of Stories and the Power of Numbers: Mixed Methods Research and Mixed Studies Reviews"},{"issue":"3","key":"pone.0274304.ref014","doi-asserted-by":"crossref","first-page":"255","DOI":"10.3102\/01623737011003255","article-title":"Toward a Conceptual Framework for Mixed-Method Evaluation Designs","volume":"11","author":"J.C. Greene","year":"1989","journal-title":"Educational Evaluation and Policy Analysis"},{"volume-title":"Cognitive Interviewing in Mixed Research","year":"2014","author":"I. Benitez Baena","key":"pone.0274304.ref015"},{"issue":"4","key":"pone.0274304.ref016","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1177\/1525822X11416564","article-title":"Comparing Cognitive Interviewing and Psychometric Methods to Evaluate a Racial\/Ethnic Discrimination Scale","volume":"23","author":"B.B. Reeve","year":"2011","journal-title":"Field methods"},{"key":"pone.0274304.ref017","doi-asserted-by":"crossref","unstructured":"Using Targeted Embedded Probes to Quantify Cognitive Interviewing Findings, in Advances in Questionnaire Design, Development, Evaluation and Testing. 2020. p. 427\u2013449.","DOI":"10.1002\/9781119263685.ch17"},{"issue":"2","key":"pone.0274304.ref018","first-page":"1","article-title":"Combining Quantitative Experimental Data with Web Probing: The Case of Individual Solutions for the Division of Labor Between Both Genders","volume":"14","author":"M.M. Braun","journal-title":"Methods, Data, Analyses"},{"key":"pone.0274304.ref019","unstructured":"World Health Organization Open Data Kit Issues Identified from End User Feedback. 2019; https:\/\/github.com\/SwissTPH\/WHO_VA_2016."},{"key":"pone.0274304.ref020","unstructured":"CHAMPS: Childhood Health and Mortality Prevention Surveillance https:\/\/champshealth.org\/."},{"key":"pone.0274304.ref021","unstructured":"Countrywide Mortality Surveillance for Action (COMSA). https:\/\/comsamozambique.org\/."},{"volume-title":"South African National Cause-of-Death Validation Project: Methodology and Description of a National Sample of Verbal Autopsies","year":"2020","author":"J.J. Bradshaw D","key":"pone.0274304.ref022"},{"key":"pone.0274304.ref023","doi-asserted-by":"crossref","DOI":"10.1002\/9781118838860","volume-title":"Cognitive Interviewing Methodology","author":"K.W. Miller","year":"2014"},{"key":"pone.0274304.ref024","unstructured":"Centers for Disease Control and Prevention. Qnotes. https:\/\/www.cdc.gov\/nchs\/ccqder\/products\/qnotes.htm."},{"key":"pone.0274304.ref025","unstructured":"Scanlon, P. Cognitive Evaluation Of The Who Verbal Autopsy Questionnaire In Lusaka, Zambia And Rabat, Morocco. 2019 [cited 2021 Jan 11]; https:\/\/wwwn.cdc.gov\/QBank\/Report.aspx?1205."},{"volume-title":"Analysis of the Cognitive Interview in Questionnaire Design","year":"2015","author":"G. Willis","key":"pone.0274304.ref026"},{"key":"pone.0274304.ref027","doi-asserted-by":"crossref","first-page":"e3785","DOI":"10.7717\/peerj.3785","article-title":"Factors influencing healthcare provider respondent fatigue answering a globally administered in-app survey","volume":"5","author":"V.N. O\u2019Reilly-Shah","year":"2017","journal-title":"PeerJ"},{"key":"pone.0274304.ref028","doi-asserted-by":"crossref","first-page":"743","DOI":"10.4135\/9781412963947","volume-title":"Encyclopedia of survey research methods","author":"P.J. Lavrakas","year":"2008"},{"key":"pone.0274304.ref029","doi-asserted-by":"crossref","first-page":"e000961","DOI":"10.1136\/bmjpo-2020-000961","article-title":"Added value of an open narrative in verbal autopsies: a mixed-methods evaluation from Malawi","volume":"5","author":"P. Loh","year":"2021","journal-title":"BMJ Paediatrics Open"},{"issue":"1","key":"pone.0274304.ref030","doi-asserted-by":"crossref","first-page":"1597452","DOI":"10.1080\/16549716.2019.1597452","article-title":"A systematic review of vital events tracking by community health agents","volume":"12","author":"E. K. Nichols","year":"2019","journal-title":"Glob Health Action"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0274304","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,28]],"date-time":"2023-11-28T16:16:36Z","timestamp":1701188196000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0274304"}},"subtitle":[],"editor":[{"given":"Ritesh G.","family":"Menezes","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2022,10,7]]},"references-count":30,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,10,7]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0274304","relation":{},"ISSN":["1932-6203"],"issn-type":[{"type":"electronic","value":"1932-6203"}],"subject":[],"published":{"date-parts":[[2022,10,7]]}}}