{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T18:23:30Z","timestamp":1787250210597,"version":"build-2736575974"},"reference-count":65,"publisher":"Public Library of Science (PLoS)","issue":"8","license":[{"start":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T00:00:00Z","timestamp":1787184000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100016670","name":"Region \u00d6sterg\u00f6tland","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100016670","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005416","name":"Research Council of Norway","doi-asserted-by":"crossref","award":["315444"],"award-info":[{"award-number":["315444"]}],"id":[{"id":"10.13039\/501100005416","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>This multi-stage study explores the novel application of the strategy mapping methodology, incorporated into a software tool, to conduct participatory evaluations of crisis management efforts. The methodology was selected because it has shown promise in other fields of research, aligns with participatory evaluation principles, and addresses the need for more impactful evaluations of crisis management efforts. The evaluation was conducted through a four-step workshop series with 15 healthcare professionals, which culminated in ten participant-developed recommendations. The recommendations targeted central risks as determined throughout the workshop series using the strategy mapping software. Approximately one year after the workshop series, semi-structured interviews with ten of the workshop participants were conducted to assess the perceived use and value of the recommendations and the evaluation process. The findings indicate that the evaluation was impactful, suggesting that the strategy mapping methodology is a viable approach for conducting participatory evaluations of crisis management efforts. Insights are also provided regarding how future participatory evaluations of crisis management efforts can be improved. These insights include the potential need for increased involvement of the evaluator in producing recommendations to secure their clarity and actionability. Furthermore, it is argued that aspects of the strategy mapping methodology perceived as valuable, such as participants rating recommendations, should be incorporated into future participatory evaluations of crisis management efforts even if the complete strategy mapping methodology is not applicable or available. By applying findings from this study, the impact of evaluations of crisis management efforts can be increased, thereby contributing to improved preparedness for future crises.<\/jats:p>","DOI":"10.1371\/journal.pone.0356419","type":"journal-article","created":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T17:24:47Z","timestamp":1787246687000},"page":"e0356419","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Utilizing strategy mapping software for participatory evaluation of crisis management efforts: A case study"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1960-4261","authenticated-orcid":true,"given":"Anton","family":"Bj\u00f6rnqvist","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carl-Oscar","family":"Jonson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Erik","family":"Prytz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bj\u00f6rn J. E.","family":"Johansson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2771-2705","authenticated-orcid":true,"given":"Peter","family":"Berggren","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,8,20]]},"reference":[{"issue":"3","key":"pone.0356419.ref001","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1111\/j.1468-5973.2009.00575.x","article-title":"Disasters, lessons learned, and fantasy documents","volume":"17","author":"TA Birkland","year":"2009","journal-title":"Journal of Contingencies and Crisis Management"},{"key":"pone.0356419.ref002","doi-asserted-by":"crossref","first-page":"105226","DOI":"10.1016\/j.ijdrr.2025.105226","article-title":"Assessing the impact of evaluations of crisis management efforts","volume":"118","author":"A Bj\u00f6rnqvist","year":"2025","journal-title":"International Journal of Disaster Risk Reduction"},{"key":"pone.0356419.ref003","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1186\/1748-5908-4-50","article-title":"Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science","volume":"4","author":"LJ Damschroder","year":"2009","journal-title":"Implement Sci"},{"issue":"3","key":"pone.0356419.ref004","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1177\/1098214009341660","article-title":"Research on evaluation use: A review of the empirical literature from 1986 to 2005","volume":"30","author":"K Johnson","year":"2009","journal-title":"American Journal of Evaluation"},{"key":"pone.0356419.ref005","volume-title":"Diffusion of innovations","author":"EM Rogers","year":"2003","edition":"5"},{"issue":"3","key":"pone.0356419.ref006","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1177\/10982140241234841","article-title":"Mapping evaluation use: A scoping review of extant literature (2005\u20132022)","volume":"45","author":"M Searle","year":"2024","journal-title":"American Journal of Evaluation"},{"key":"pone.0356419.ref007","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1016\/j.ijdrr.2016.09.001","article-title":"Scoping the field of disaster exercise evaluation - A literature overview and analysis","volume":"19","author":"RJJ Beerens","year":"2016","journal-title":"International Journal of Disaster Risk Reduction"},{"key":"pone.0356419.ref008","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.ijdrr.2013.05.001","article-title":"Learning effectiveness of discussion-based crisis management exercises","volume":"5","author":"J Borell","year":"2013","journal-title":"International Journal of Disaster Risk Reduction"},{"issue":"5","key":"pone.0356419.ref009","doi-asserted-by":"crossref","first-page":"64","DOI":"10.5055\/jem.2008.0039","article-title":"Learning from failures in emergency response: Two empirical studies","volume":"6","author":"SWA Dekker","year":"2008","journal-title":"JEM"},{"key":"pone.0356419.ref010","article-title":"Learning from simulating mass casualty events: A systematic search and a comprehensive qualitative review","volume":"17","author":"K Ledbury","year":"2022","journal-title":"Disaster Med Public Health Prep"},{"issue":"8","key":"pone.0356419.ref011","doi-asserted-by":"crossref","first-page":"2949","DOI":"10.3390\/ijerph9082949","article-title":"Use of after action reports (AARs) to promote organizational and systems learning in emergency preparedness","volume":"9","author":"E Savoia","year":"2012","journal-title":"International Journal of Environmental Research and Public Health"},{"issue":"4","key":"pone.0356419.ref012","doi-asserted-by":"crossref","DOI":"10.1111\/1468-5973.70095","article-title":"A Systematic approach to exercises: The importance of focusing on learning needs, evaluation and follow\u2010up","volume":"33","author":"LL Evenseth","year":"2025","journal-title":"Contingencies & Crisis Mgmt"},{"issue":"2","key":"pone.0356419.ref013","doi-asserted-by":"crossref","first-page":"107","DOI":"10.5055\/ajdm.2018.0292","article-title":"A scoping review of evaluation methods for health emergency preparedness exercises","volume":"13","author":"E Skryabina","year":"2018","journal-title":"Am J Disaster Med"},{"issue":"1","key":"pone.0356419.ref014","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1504\/IJEM.2021.10055770","article-title":"Quantitative performance evaluations for crisis response management: A scoping study","volume":"1","author":"N Baroutsi","year":"2021","journal-title":"IJEM"},{"issue":"9","key":"pone.0356419.ref015","doi-asserted-by":"crossref","first-page":"9712","DOI":"10.3390\/ijerph110909712","article-title":"A conceptual framework to measure systems\u2019 performance during emergency preparedness exercises","volume":"11","author":"E Savoia","year":"2014","journal-title":"Int J Environ Res Public Health"},{"issue":"4","key":"pone.0356419.ref016","doi-asserted-by":"crossref","first-page":"397","DOI":"10.3102\/01623737014004397","article-title":"The case for participatory evaluation","volume":"14","author":"JB Cousins","year":"1992","journal-title":"Educational Evaluation and Policy Analysis"},{"issue":"80","key":"pone.0356419.ref017","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1002\/ev.1115","article-title":"The history of participatory evaluation and current debates in the field","volume":"1998","author":"S Brisolara","year":"1998","journal-title":"New Drctns Evaluation"},{"issue":"16","key":"pone.0356419.ref018","doi-asserted-by":"crossref","first-page":"32","DOI":"10.56645\/jmde.v7i16.288","article-title":"Forms and functions of participatory evaluation in international development: A review of the empirical and theoretical literature","volume":"7","author":"AE Cullen","year":"2011","journal-title":"J Multidiscip Eval"},{"key":"pone.0356419.ref019","volume-title":"Visual Strategy: Strategy Mapping for Public and Nonprofit Organizations","author":"JM Bryson","year":"2014"},{"key":"pone.0356419.ref020","author":"JM Bryson","year":"2023"},{"key":"pone.0356419.ref021","volume-title":"Strategic planning for public and nonprofit organizations: a guide to strengthening and sustaining organizational achievement","author":"JM Bryson","year":"2017","edition":"5"},{"issue":"3","key":"pone.0356419.ref022","doi-asserted-by":"crossref","first-page":"615","DOI":"10.1016\/S0377-2217(03)00061-4","article-title":"Cognitive mapping expert views for policy analysis in the public sector","volume":"152","author":"C Eden","year":"2004","journal-title":"European Journal of Operational Research"},{"issue":"4","key":"pone.0356419.ref023","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1177\/10982140251355140","article-title":"Theory-practice connections in collaborative approaches to evaluation: A systematic review of practice","volume":"46","author":"JB Cousins","year":"2025","journal-title":"American Journal of Evaluation"},{"issue":"80","key":"pone.0356419.ref024","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1002\/ev.1116","article-title":"Evaluating for a change: Reflections on participatory methodology","volume":"1998","author":"B Burke","year":"1998","journal-title":"New Drctns Evaluation"},{"issue":"2","key":"pone.0356419.ref025","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1177\/1098214013478142","article-title":"The case for participatory evaluation in an era of accountability","volume":"34","author":"JA Chouinard","year":"2013","journal-title":"American Journal of Evaluation"},{"issue":"2","key":"pone.0356419.ref026","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1016\/j.ejor.2024.01.024","article-title":"Managing grand challenges: Extending the scope of problem structuring methods and behavioural operational research","volume":"319","author":"F Ackermann","year":"2024","journal-title":"European Journal of Operational Research"},{"key":"pone.0356419.ref027","article-title":"Accounting for systemic risk in analysis of crisis scenarios","author":"JJ Gonzalez","year":"2024","journal-title":"ISCRAM"},{"key":"pone.0356419.ref028","article-title":"The core ingredients of ripple effects mapping","volume-title":"A field guide to ripple effects mapping","author":"S Chazdon","year":"2017"},{"issue":"10389","key":"pone.0356419.ref029","doi-asserted-by":"crossref","first-page":"1642","DOI":"10.1016\/S0140-6736(23)01003-6","article-title":"With the COVID-19 PHEIC over, what next?","volume":"401","author":"J Zarocostas","year":"2023","journal-title":"The Lancet"},{"key":"pone.0356419.ref030","article-title":"Instrumental case study","volume-title":"Encyclopedia of case study research","author":"G. Grandy","year":"2010"},{"key":"pone.0356419.ref031","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1007\/978-94-010-0309-4_16","article-title":"Utilization effects of participatory evaluation","author":"J Bradley Cousins","year":"2003","journal-title":"International Handbook of Educational Evaluation. Springer Netherlands"},{"issue":"80","key":"pone.0356419.ref032","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1002\/ev.1114","article-title":"Framing participatory evaluation","volume":"1998","author":"JB Cousins","year":"1998","journal-title":"New Drctns Evaluation"},{"issue":"1","key":"pone.0356419.ref033","doi-asserted-by":"crossref","first-page":"19","DOI":"10.56645\/jmde.v1i1.144","article-title":"Unpacking the participatory process","volume":"1","author":"L Weaver","year":"2004","journal-title":"J Multidiscip Eval"},{"key":"pone.0356419.ref034","author":"A Manzano","year":"2024"},{"issue":"1","key":"pone.0356419.ref035","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0191-491X(04)90001-6","article-title":"Internal participatory evaluation as an organizational learning system: A longitudinal case study","volume":"30","author":"TT Robinson","year":"2004","journal-title":"Studies in Educational Evaluation"},{"issue":"2","key":"pone.0356419.ref036","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1177\/0193841X8801200201","article-title":"Stakeholder participation and utilization in program evaluation","volume":"12","author":"JG Greene","year":"1988","journal-title":"Eval Rev"},{"key":"pone.0356419.ref037","volume-title":"Evaluation theory, models, and applications","author":"DL Stufflebeam","year":"2014","edition":"2"},{"issue":"3","key":"pone.0356419.ref038","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1177\/1098214009340580","article-title":"Toward accurate measurement of participation: Rethinking the conceptualization and operationalization of participatory evaluation","volume":"30","author":"PM Daigneault","year":"2009","journal-title":"American Journal of Evaluation"},{"key":"pone.0356419.ref039","first-page":"1","article-title":"Causal mapping: An historical overview","author":"VK Narayanan","year":"2005","journal-title":"Causal Mapping for Research in Information Technology. Hershey, PA, USA: IGI Global"},{"issue":"3","key":"pone.0356419.ref040","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1111\/j.1467-6486.1992.tb00667.x","article-title":"The analysis of cause mapS","volume":"29","author":"C Eden","year":"1992","journal-title":"J Management Studies"},{"key":"pone.0356419.ref041","first-page":"46","article-title":"What Have We Learned from Almost 30 Years of Research on Causal Mapping? Methodological Lessons and Choices for the Information Systems and Information Technology Communities","author":"GP Hodgkinson","year":"2005","journal-title":"Causal Mapping for Research in Information Technology. IGI Global"},{"key":"pone.0356419.ref042","doi-asserted-by":"crossref","first-page":"100161","DOI":"10.1016\/j.plas.2024.100161","article-title":"Overlooked and underused? The benefits and challenges of using causal mapping for project studies","volume":"5","author":"F Ackermann","year":"2024","journal-title":"Project Leadership and Society"},{"issue":"3","key":"pone.0356419.ref043","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1016\/S0377-2217(03)00056-0","article-title":"Problem structuring methods in action","volume":"152","author":"J Mingers","year":"2004","journal-title":"European Journal of Operational Research"},{"key":"pone.0356419.ref044","first-page":"139","article-title":"Strategic options development and analysis","author":"F Ackermann","year":"2020","journal-title":"Systems Approaches to Making Change: A Practical Guide. Springer London"},{"key":"pone.0356419.ref045","article-title":"Using action-oriented strategy mapping to identify strategic issues and formulate effective strategies","volume-title":"Strategic planning for public and nonprofit organizations: a guide to strengthening and sustaining organizational achievement","author":"JM Bryson","year":"2011","edition":"4"},{"issue":"6","key":"pone.0356419.ref046","doi-asserted-by":"crossref","first-page":"912","DOI":"10.1111\/puar.12608","article-title":"Discovering collaborative advantage: The contributions of goal categories and visual strategy mapping","volume":"76","author":"JM Bryson","year":"2016","journal-title":"Public Administration Review"},{"key":"pone.0356419.ref047","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1007\/978-90-481-9097-3_17","article-title":"The role of group decision support systems: Negotiating safe energy","author":"F Ackermann","year":"2010","journal-title":"Advances in Group Decision and Negotiation. Springer Netherlands"},{"key":"pone.0356419.ref048","volume-title":"Strategyfidnder: The Basics","author":"C Eden","year":"2024"},{"issue":"4","key":"pone.0356419.ref049","doi-asserted-by":"crossref","first-page":"721","DOI":"10.1108\/IJPPM-08-2017-0192","article-title":"Revisiting strategy mapping for performance management: a realist synthesis","volume":"68","author":"R Armstrong","year":"2019","journal-title":"Int J Product Perform Manag"},{"key":"pone.0356419.ref050","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.ijpe.2019.05.020","article-title":"A quantitative methodology to enhance a strategy map","volume":"219","author":"JA Moraga","year":"2020","journal-title":"International Journal of Production Economics"},{"key":"pone.0356419.ref051","doi-asserted-by":"crossref","first-page":"104189","DOI":"10.1016\/j.ijdrr.2023.104189","article-title":"Using multi-stakeholder causal mapping to explore priorities for infrastructure resilience to flooding","volume":"101","author":"M Pregnolato","year":"2024","journal-title":"International Journal of Disaster Risk Reduction"},{"issue":"1","key":"pone.0356419.ref052","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1177\/13563890231196601","article-title":"Causal mapping for evaluators","volume":"30","author":"S Powell","year":"2023","journal-title":"Evaluation"},{"key":"pone.0356419.ref053","year":"2013"},{"issue":"9","key":"pone.0356419.ref054","doi-asserted-by":"crossref","first-page":"1277","DOI":"10.1177\/1049732305276687","article-title":"Three approaches to qualitative content analysis","volume":"15","author":"H-F Hsieh","year":"2005","journal-title":"Qual Health Res"},{"issue":"3","key":"pone.0356419.ref055","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1177\/1098214017717015","article-title":"Definitions of evaluation use and misuse, evaluation influence, and factors affecting use","volume":"38","author":"MC Alkin","year":"2017","journal-title":"American Journal of Evaluation"},{"key":"pone.0356419.ref056","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1186\/s13012-015-0242-0","article-title":"Making sense of implementation theories, models and frameworks","volume":"10","author":"P Nilsen","year":"2015","journal-title":"Implement Sci"},{"issue":"2","key":"pone.0356419.ref057","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1007\/s10488-010-0319-7","article-title":"Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda","volume":"38","author":"E Proctor","year":"2011","journal-title":"Adm Policy Ment Health"},{"issue":"4","key":"pone.0356419.ref058","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1002\/hbe2.195","article-title":"The theory of planned behavior: Frequently asked questions","volume":"2","author":"I Ajzen","year":"2020","journal-title":"Human Behav and Emerg Tech"},{"issue":"2","key":"pone.0356419.ref059","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1080\/14697011003795651","article-title":"Linking change drivers and the organizational change process: A review and synthesis","volume":"10","author":"KS Whelan-Berry","year":"2010","journal-title":"Journal of Change Management"},{"key":"pone.0356419.ref060","article-title":"Participatory evaluation: Case studies of international full scale exercises","author":"H Bruns","year":"2025","journal-title":"ISCRAM"},{"issue":"4","key":"pone.0356419.ref061","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1177\/1098214016665164","article-title":"The historical development of evaluation use","volume":"37","author":"MC Alkin","year":"2016","journal-title":"American Journal of Evaluation"},{"issue":"1","key":"pone.0356419.ref062","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1037\/tps0000290","article-title":"Unfixate your creative mind: Forgetting fixation and its applications","volume":"8","author":"Z Beda","year":"2022","journal-title":"Translational Issues in Psychological Science"},{"issue":"3","key":"pone.0356419.ref063","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1177\/1356389009105884","article-title":"Critical connections between participatory evaluation, organizational learning and intentional change in pluralistic organizations","volume":"15","author":"JC Su\u00e1rez-Herrera","year":"2009","journal-title":"Evaluation"},{"issue":"4","key":"pone.0356419.ref064","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1002\/sdr.444","article-title":"Using a group decision support system to add value to group model building","volume":"26","author":"F Ackermann","year":"2010","journal-title":"System Dynamics Review"},{"issue":"5","key":"pone.0356419.ref065","doi-asserted-by":"crossref","first-page":"783","DOI":"10.1177\/1049732319889354","article-title":"\u201cEverything is perfect, and we have no problems\u201d: Detecting and limiting social desirability bias in qualitative research","volume":"30","author":"N Bergen","year":"2020","journal-title":"Qual Health Res"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0356419","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T17:24:56Z","timestamp":1787246696000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0356419"}},"subtitle":[],"editor":[{"given":"Laura Hannah","family":"Kelly","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,8,20]]},"references-count":65,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2026,8,20]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0356419","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,20]]}}}