{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T01:35:44Z","timestamp":1790300144336,"version":"4.1.0"},"reference-count":32,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2024,8,31]],"date-time":"2024-08-31T00:00:00Z","timestamp":1725062400000},"content-version":"vor","delay-in-days":61,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NIHR Yorkshire and Humber Patient Safety Translational Research Centre","award":["PSTRC-2016-006"],"award-info":[{"award-number":["PSTRC-2016-006"]}]},{"name":"National Institute for Health and Care Research Yorkshire and Humber ARC","award":["NIHR 200166"],"award-info":[{"award-number":["NIHR 200166"]}]},{"name":"NIHR Yorkshire and Humber Patient Safety Translational Research Centre","award":["PSTRC-2016-006"],"award-info":[{"award-number":["PSTRC-2016-006"]}]},{"name":"National Institute for Health and Care Research Yorkshire and Humber ARC","award":["NIHR 200166"],"award-info":[{"award-number":["NIHR 200166"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,9,24]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>There have been growing concerns about the well-being of staff in inpatient mental health settings, with studies suggesting that they have higher burnout and greater work-related stress levels than staff in other healthcare sectors. When addressing staff well-being, psychological safety can be a useful concept. However, there is no measure of psychological safety that is suitable for use in inpatient mental health settings. Edmondson (1999) is the most commonly used measure of psychological safety, but it was designed for use in general physical healthcare settings. As inpatient mental health settings are unique environments, transferability of knowledge from physical to mental healthcare settings cannot be assumed. We sought to develop questionnaire items that capture psychological safety among healthcare staff working in acute inpatient mental healthcare settings. We used the nominal group technique, a consensus method involving rounds of discussion, idea generation, and item rating\/ranking to identify priorities. Twenty-eight stakeholders participated, including 4 who had lived experience of mental health problems, 11 academics and 18 healthcare professionals (8 participants identified with more than 1 category). The study involved a workshop with three parts: (i) an overview of current research and limitations of the Edmondson (1999) measure as outlined above, (ii) discussion on what items should be retained from the Edmondson (1999) measure, and (iii) discussion on what items should be added to the Edmondson (1999) measure. Twenty-one items were generated and retained to capture psychological safety in inpatient mental health settings. These measure professionals\u2019 sense of being valued by their team and organization, feeling supported at work, feeling physically safe and protected from physical harm, and knowing they can raise concerns about risk and safety. This is the first study to generate questionnaire items suitable for measuring staff psychological safety in mental health settings. These have been generated via a consensus method to ensure stakeholders\u2019 views are reflected. Further research is needed to evaluate factor structure, internal reliability, and convergent validity.<\/jats:p>","DOI":"10.1093\/intqhc\/mzae086","type":"journal-article","created":{"date-parts":[[2024,8,31]],"date-time":"2024-08-31T16:02:20Z","timestamp":1725120140000},"source":"Crossref","is-referenced-by-count":1,"title":["How can we measure psychological safety in mental healthcare staff? Developing questionnaire items using a nominal groups technique"],"prefix":"10.1093","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3911-637X","authenticated-orcid":false,"given":"Katharina Sophie","family":"Vogt","sequence":"first","affiliation":[{"name":"Bradford Institute for Health Research, Bradford Royal Infirmary , Temple Bank House, Duckworth Lane, Bradford BD9 6RJ, United Kingdom"},{"name":"School of Psychology, University of Leeds , Lifton Place, Leeds LS2 9JT, United Kingdom"},{"name":"Department of Psychology, Institute of Population Health, University of Liverpool , Brownlow Street, Liverpool L69 3GB, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John","family":"Baker","sequence":"additional","affiliation":[{"name":"School of Healthcare, University of Leeds , Clarendon Way, Leeds LS2 9JT, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rebecca","family":"Coleman","sequence":"additional","affiliation":[{"name":"College of Medical, Veterinary and Life Sciences, University of Glasgow, School of Health and Wellbeing , Clarice Pears Building, Glasgow G12 8TB, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Kendal","sequence":"additional","affiliation":[{"name":"School of Healthcare, University of Leeds , Clarendon Way, Leeds LS2 9JT, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bethany","family":"Griffin","sequence":"additional","affiliation":[{"name":"Bradford Institute for Health Research, Bradford Royal Infirmary , Temple Bank House, Duckworth Lane, Bradford BD9 6RJ, United Kingdom"},{"name":"School of Psychology, University of Leeds , Lifton Place, Leeds LS2 9JT, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taha","family":"Anjum","sequence":"additional","affiliation":[{"name":"South West Yorkshire Partnership NHS Foundation Trust , Ouchthorpe Lane, Wakefield WF1 3SP, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kirsty Louise","family":"Ashley","sequence":"additional","affiliation":[{"name":"Independent expert , United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bethany Lauren","family":"Archer","sequence":"additional","affiliation":[{"name":"Department of Psychology, Institute of Population Health, University of Liverpool , Brownlow Street, Liverpool L69 3GB, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Katherine","family":"Berry","sequence":"additional","affiliation":[{"name":"Division of Psychology and Mental Health, University of Manchester , Oxford Road, Manchester, Manchester M13 9PL, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robyn","family":"Feldman","sequence":"additional","affiliation":[{"name":"Independent expert , United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stephanie","family":"Gray","sequence":"additional","affiliation":[{"name":"Independent expert , United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sally Jane","family":"Giles","sequence":"additional","affiliation":[{"name":"NIHR Greater Manchester Patient Safety Research Collaboration, NIHR School for Primary Care Research, Division of Population Health, Health Services Research and Primary Care, University of Manchester , Oxford Road, Manchester M13 9PL, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin James","family":"Helliwell","sequence":"additional","affiliation":[{"name":"Pennine Care NHS Foundation Trust , Humphrey House, Angouleme Way, Bury BL9 0EQ, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chelsea","family":"Hill","sequence":"additional","affiliation":[{"name":"Independent expert , United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aimee Elisha","family":"Hogan","sequence":"additional","affiliation":[{"name":"Clinical Psychology, Health Innovation Campus, Lancaster University , Sir John Fisher Drive, Bailrigg, Lancaster LA1 4YW, United Kingdom"},{"name":"Secure Forensic Inpatient Service, Lancashire and South Cumbria NHS Foundation Trust , Sceptre Point, Sceptre Way Walton Summit, Preston PR5 6AW, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Magdalena","family":"Iwanow","sequence":"additional","affiliation":[{"name":"Mersey Care NHS Foundation Trust, Kings Business Park , Trust Offices\/V7 Buildings, Prescot L34 1PJ, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Timon Anton Arie","family":"Jansen","sequence":"additional","affiliation":[{"name":"Sheffield Health and Social Care NHS Foundation Trust, Centre Court , Atlas Way, Sheffield S4 7QQ, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zach","family":"Johnson","sequence":"additional","affiliation":[{"name":"Independent expert , United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James A","family":"Kelly","sequence":"additional","affiliation":[{"name":"Clinical Psychology, Health Innovation Campus, Lancaster University , Sir John Fisher Drive, Bailrigg, Lancaster LA1 4YW, United Kingdom"},{"name":"Bolton Community Mental Health, Greater Manchester Mental Health NHS Foundation Trust , Bury New Road, Manchester M25 3BL, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joshua","family":"Law","sequence":"additional","affiliation":[{"name":"Department of Psychology, Institute of Population Health, University of Liverpool , Brownlow Street, Liverpool L69 3GB, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emily","family":"Mizen","sequence":"additional","affiliation":[{"name":"Norwich Medical School, University of East Anglia , Norwich Research Park, Norwich, Norfolk NR4 7TJ, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Owenvbiugie Omorefe","family":"Obasohan","sequence":"additional","affiliation":[{"name":"Tees, Esk and Wear Valley NHS Foundation Trust, West Park Hospital , Edward Pease Way, Darlington, Durham DL2 2TS, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maria","family":"Panagioti","sequence":"additional","affiliation":[{"name":"NIHR Greater Manchester Patient Safety Research Collaboration, NIHR School for Primary Care Research, Division of Population Health, Health Services Research and Primary Care, University of Manchester , Oxford Road, Manchester M13 9PL, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ffion","family":"Smith-Wilkes","sequence":"additional","affiliation":[{"name":"Independent expert , United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Steeg","sequence":"additional","affiliation":[{"name":"NIHR Greater Manchester Patient Safety Research Collaboration, NIHR School for Primary Care Research, Division of Population Health, Health Services Research and Primary Care, University of Manchester , Oxford Road, Manchester M13 9PL, United Kingdom"},{"name":"Division of Psychology and Mental Health, University of Manchester , Oxford Road, Manchester, Manchester M13 9PL, United Kingdom"},{"name":"Manchester Academic Health Science Centre, University of Manchester , CityLabs 1.0, Nelson Street, Manchester M13 9NQ, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher D J","family":"Taylor","sequence":"additional","affiliation":[{"name":"Division of Psychology and Mental Health, University of Manchester , Oxford Road, Manchester, Manchester M13 9PL, United Kingdom"},{"name":"Pennine Care NHS Foundation Trust , Humphrey House, Angouleme Way, Bury BL9 0EQ, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Natasha","family":"Tyler","sequence":"additional","affiliation":[{"name":"NIHR Greater Manchester Patient Safety Research Collaboration, NIHR School for Primary Care Research, Division of Population Health, Health Services Research and Primary Care, University of Manchester , Oxford Road, Manchester M13 9PL, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sophie","family":"Wade","sequence":"additional","affiliation":[{"name":"Counselling & Mental Health Service, Palatine Centre, Durham University , Stockton Road, Durham DH1 3LE, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0431-013X","authenticated-orcid":false,"given":"Judith","family":"Johnson","sequence":"additional","affiliation":[{"name":"Bradford Institute for Health Research, Bradford Royal Infirmary , Temple Bank House, Duckworth Lane, Bradford BD9 6RJ, United Kingdom"},{"name":"School of Psychology, University of Leeds , Lifton Place, Leeds LS2 9JT, United Kingdom"},{"name":"School of Public Health and Community Medicine, University of New South Wales , High Street, Sydney NSW 2052, Australia"},{"name":"Division of Nursing, Midwifery and Social Work, School of Health Sciences, University of Manchester , Jean McFarlane Building, Oxford Road, Manchester M13 9PY, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,8,31]]},"reference":[{"key":"2024092415373458000_R1","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1111\/inm.12416","article-title":"Mental healthcare staff well-being and burnout: a narrative review of trends, causes, implications, and recommendations for future interventions","volume":"27","author":"Johnson","year":"2018","journal-title":"Int J Ment Health Nurs"},{"key":"2024092415373458000_R2","doi-asserted-by":"publisher","DOI":"10.3390\/medicina55090553","article-title":"Influence of burnout on patient safety: systematic review and meta-analysis","volume":"55","author":"Garcia","year":"2019","journal-title":"Medicina (Kaunas)"},{"key":"2024092415373458000_R3","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0159015","article-title":"Healthcare staff wellbeing, burnout, and patient safety: a systematic review","volume":"11","author":"Hall","year":"2016","journal-title":"PLoS One"},{"key":"2024092415373458000_R4","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0206233","article-title":"What causes medication administration errors in a mental health hospital? A qualitative study with nursing staff","volume":"13","author":"Keers","year":"2018","journal-title":"PLoS One"},{"key":"2024092415373458000_R5","doi-asserted-by":"publisher","first-page":"350","DOI":"10.2307\/2666999","article-title":"Psychological safety and learning behavior in work teams","volume":"44","author":"Edmondson","year":"1999","journal-title":"Adm Sci Q"},{"key":"2024092415373458000_R6","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6963-14-61","article-title":"Speaking up for patient safety by hospital-based health care professionals: a literature review","volume":"14","author":"Okuyama","year":"2014","journal-title":"BMC Health Serv Res"},{"key":"2024092415373458000_R7","doi-asserted-by":"publisher","first-page":"e599","DOI":"10.1097\/PTS.0000000000000415","article-title":"Development and psychometric evaluation of the speaking up about patient safety questionnaire","volume":"17","author":"Richard","year":"2021","journal-title":"J Patient Saf"},{"key":"2024092415373458000_R8","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-021-06740-6","article-title":"The presence and potential impact of psychological safety in the healthcare setting: an evidence synthesis","volume":"21","author":"Grailey","year":"2021","journal-title":"BMC Health Serv Res"},{"key":"2024092415373458000_R9","doi-asserted-by":"publisher","DOI":"10.1186\/s13033-021-00439-1","article-title":"Enhancing psychological safety in mental health services","volume":"15","author":"Hunt","year":"2021","journal-title":"Int J Ment Health Syst"},{"key":"2024092415373458000_R10","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1097\/SLA.0000000000003019","article-title":"Burnout in surgery viewed through the lens of psychological safety","volume":"269","author":"Swendiman","year":"2019","journal-title":"Ann Surg"},{"key":"2024092415373458000_R11","doi-asserted-by":"publisher","first-page":"549","DOI":"10.1177\/070674371005500903","article-title":"Challenges and opportunities in measuring the quality of mental health care","volume":"55","author":"Kilbourne","year":"2010","journal-title":"Can J Psychiatry"},{"key":"2024092415373458000_R12","doi-asserted-by":"publisher","first-page":"e781","DOI":"10.1111\/hsc.12806","article-title":"\u201cKeeping Control\u201d: a user-led exploratory study of mental health service user experiences of targeted violence and abuse in the context of adult safeguarding in England","volume":"27","author":"Carr","year":"2019","journal-title":"Health Soc Care Community"},{"key":"2024092415373458000_R13","doi-asserted-by":"publisher","DOI":"10.4102\/sajpsychiatry.v28i0.1700","article-title":"Experience of violence, coping and support for nurses working in acute psychiatric wards","volume":"28","author":"Bekelepi","year":"2022","journal-title":"S Afr J Psychiatr"},{"key":"2024092415373458000_R14","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1016\/j.shaw.2017.01.004","article-title":"Workplace violence toward mental healthcare workers employed in psychiatric wards","volume":"8","author":"d\u2019Ettorre","year":"2017","journal-title":"Saf Health Work"},{"key":"2024092415373458000_R15","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1111\/inm.12530","article-title":"The physical and mental health of acute psychiatric ward staff, and its relationship to experience of physical violence","volume":"28","author":"Renwick","year":"2019","journal-title":"Int J Ment Health Nurs"},{"key":"2024092415373458000_R16","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0258346","article-title":"Aggression on the psychiatric ward: prevalence and risk factors. A systematic review of the literature","volume":"16","author":"Weltens","year":"2021","journal-title":"PLoS One"},{"key":"2024092415373458000_R17","doi-asserted-by":"publisher","DOI":"10.1111\/jpm.13101","article-title":"I think the first priority is physically safe first, before you can actually get psychologically safe: staff perspectives on psychological safety in inpatient mental health settings","author":"Vogt","year":"2024","journal-title":"J Psychiatr Ment Health Nurs"},{"key":"2024092415373458000_R18","doi-asserted-by":"publisher","first-page":"140","DOI":"10.1016\/j.jclinepi.2021.08.008","article-title":"Methodological options of the nominal group technique for survey item elicitation in health research: a scoping review","volume":"139","author":"Harb","year":"2021","journal-title":"J Clin Epidemiol"},{"key":"2024092415373458000_R19","doi-asserted-by":"publisher","first-page":"655","DOI":"10.1007\/s11096-016-0257-x","article-title":"How to use the nominal group and Delphi techniques","volume":"38","author":"McMillan","year":"2016","journal-title":"Int J Clin Pharm"},{"key":"2024092415373458000_R20","article-title":"The Nominal Group Technique - a practical guide for facilitators","volume-title":"Written for the ELESIG SMall Grants Scheme. Version 1.02011","author":"Varga-Atkins","year":"2011"},{"key":"2024092415373458000_R21","article-title":"About us: Meet the Padlet Squad online: padlet.com","author":"Padlet","year":"2023"},{"key":"2024092415373458000_R22","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1332\/174426421X16420955772641","article-title":"What is co-production? Conceptualising and understanding co-production of knowledge and policy across different theoretical perspectives","volume":"19","author":"Bandola-Gill","year":"2023","journal-title":"Evid Policy"},{"key":"2024092415373458000_R23","author":"Qualtrics"},{"key":"2024092415373458000_R24","doi-asserted-by":"publisher","first-page":"316","DOI":"10.1080\/09638230802053359","article-title":"A conceptual model of the aims and functions of acute inpatient psychiatry","volume":"18","author":"Bowers","year":"2009","journal-title":"J Ment Health"},{"key":"2024092415373458000_R25","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1111\/inm.12063","article-title":"Clinical responsibility, accountability, and risk aversion in mental health nursing: a descriptive, qualitative study","volume":"23","author":"Manuel","year":"2014","journal-title":"Int J Ment Health Nurs"},{"key":"2024092415373458000_R26","doi-asserted-by":"publisher","first-page":"e497","DOI":"10.1097\/PTS.0000000000000396","article-title":"The consequences of whistle-blowing: an integrative review","volume":"17","author":"Lim","year":"2021","journal-title":"J Patient Saf"},{"key":"2024092415373458000_R27","doi-asserted-by":"publisher","first-page":"71","DOI":"10.5465\/amj.2010.0176","article-title":"Psychological antecedents of promotive and prohibitive voice: a two-wave examination","volume":"55","author":"Liang","year":"2012","journal-title":"Acad Manage J"},{"key":"2024092415373458000_R28","doi-asserted-by":"publisher","first-page":"869","DOI":"10.5465\/amj.2007.26279183","article-title":"Leadership behavior and employee voice: is the door really open?","volume":"50","author":"Detert","year":"2007","journal-title":"Acad Manage J"},{"key":"2024092415373458000_R29","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCOUTCOMES.116.003422","article-title":"Development and psychometric properties of a scale to measure hospital organizational culture for cardiovascular care","volume":"10","author":"Bradley","year":"2017","journal-title":"Circ Cardiovasc Qual Outcomes"},{"key":"2024092415373458000_R30","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1108\/TLO-05-2016-0029","article-title":"Team psychological safety and team learning: a cultural perspective","volume":"23","author":"Cauwelier","year":"2016","journal-title":"Learn Organ"},{"key":"2024092415373458000_R31","doi-asserted-by":"publisher","first-page":"1903","DOI":"10.1080\/03075079.2021.1983534","article-title":"Diversity approaches matter in international classrooms: how a multicultural approach buffers against cultural misunderstandings and encourages inclusion and psychological safety","volume":"47","author":"De Leersnyder","year":"2022","journal-title":"Stud Higher Educ"},{"key":"2024092415373458000_R32","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.jpsychires.2019.12.015","article-title":"Global prevalence of burnout symptoms among nurses: a systematic review and meta-analysis","volume":"123","author":"Woo","year":"2020","journal-title":"J Psychiatr Res"}],"container-title":["International Journal for Quality in Health Care"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/intqhc\/advance-article-pdf\/doi\/10.1093\/intqhc\/mzae086\/58985293\/mzae086.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/intqhc\/advance-article-pdf\/doi\/10.1093\/intqhc\/mzae086\/59312799\/mzae086.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/intqhc\/advance-article-pdf\/doi\/10.1093\/intqhc\/mzae086\/59312799\/mzae086.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,24]],"date-time":"2024-09-24T15:37:51Z","timestamp":1727192271000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/intqhc\/article\/doi\/10.1093\/intqhc\/mzae086\/7746667"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7]]},"references-count":32,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,24]]}},"URL":"https:\/\/doi.org\/10.1093\/intqhc\/mzae086","relation":{},"ISSN":["1353-4505","1464-3677"],"issn-type":[{"value":"1353-4505","type":"print"},{"value":"1464-3677","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,7]]},"published":{"date-parts":[[2024,7]]},"article-number":"mzae086"}}