{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T21:39:44Z","timestamp":1776461984182,"version":"3.51.2"},"reference-count":60,"publisher":"BMJ","issue":"11","license":[{"start":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T00:00:00Z","timestamp":1604620800000},"content-version":"unspecified","delay-in-days":5,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Coordination for the Improvement of Higher Education Personnel, Brazil","award":["13271\/13-0"],"award-info":[{"award-number":["13271\/13-0"]}]},{"name":"Brazilian National Council of Technological and Scientific Development","award":["229753\/2013-2"],"award-info":[{"award-number":["229753\/2013-2"]}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Open"],"published-print":{"date-parts":[[2020,11]]},"abstract":"<jats:sec><jats:title>Objectives<\/jats:title><jats:p>This study describes the development and implementation of a model to assess students\u2019 communication skills highlighting the use of reflective writing. We aimed to evaluate the usefulness of the students\u2019 reflections in the assessment of communication skills.<\/jats:p><\/jats:sec><jats:sec><jats:title>Design<\/jats:title><jats:p>Third-year and fourth-year medical students enrolled in an elective course on clinical communication skills development were assessed using different assessment methods.<\/jats:p><\/jats:sec><jats:sec><jats:title>Setting and participants<\/jats:title><jats:p>The communication skills course was offered at four universities (three in Brazil and one in Portugal) and included 69 students.<\/jats:p><\/jats:sec><jats:sec><jats:title>Outcome measures<\/jats:title><jats:p>The students were assessed by a Multiple-Choice Questionnaire (MCQ), an objective structured clinical examination (OSCE) and reflective writing narratives. The Cronbach\u2019s alpha, dimensionality and the person\u2019s correlation were applied to evaluate the reliability of the assessment methods and their correlations. Reflective witting was assessed by applying the Reflection Evaluation for Enhanced Competencies Tool Rubric (Reflect Score (RS)) to measure reflections\u2019 depth, and the Thematic Score (TS) to map and grade reflections\u2019 themes.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The Cronbach alpha for the MCQ, OSCE global score, TS and RS were, respectively, 0.697, 0.633, 0.784 and 0.850. The interobserver correlation for the TS and RS were, respectively, 0.907 and 0.816. The assessment of reflection using the TS was significantly correlated with the MCQ (r=0.412; p=0.019), OSCE (0.439; p=0.012) and RS (0.410; p=0.020). The RS did not correlate with the MCQ and OSCE.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>Assessing reflection through mapping the themes and analysing the depth of reflective writing expands the assessment of communication skills. While the assessment of reflective themes is related to the cognitive and behavioural domains of learning, the reflective depth seems to be a specific competence, not correlated with other assessment methods\u2014possibly a metacognitive domain.<\/jats:p><\/jats:sec>","DOI":"10.1136\/bmjopen-2020-038898","type":"journal-article","created":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T21:27:55Z","timestamp":1604698075000},"page":"e038898","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":12,"title":["Added value of assessing medical students\u2019 reflective writings in communication skills training: a longitudinal study in four academic centres"],"prefix":"10.1136","volume":"10","author":[{"given":"Camila","family":"Ament Giuliani Franco","sequence":"first","affiliation":[]},{"given":"Renato Soleiman","family":"Franco","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8746-1680","authenticated-orcid":false,"given":"Dario","family":"Cecilio-Fernandes","sequence":"additional","affiliation":[]},{"given":"Milton","family":"Severo","sequence":"additional","affiliation":[]},{"given":"Maria Am\u00e9lia","family":"Ferreira","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7008-4092","authenticated-orcid":false,"given":"Marco Antonio","family":"de Carvalho-Filho","sequence":"additional","affiliation":[]}],"member":"239","published-online":{"date-parts":[[2020,11,6]]},"reference":[{"key":"2020110613201970000_10.11.e038898.1","first-page":"796","article-title":"The impact of patient-centered care on outcomes","volume":"49","author":"Stewart","year":"2000","journal-title":"J Fam Pract"},{"key":"2020110613201970000_10.11.e038898.2","doi-asserted-by":"publisher","DOI":"10.1007\/s11606-014-2765-5"},{"key":"2020110613201970000_10.11.e038898.3","doi-asserted-by":"publisher","DOI":"10.1080\/01421590903050374"},{"key":"2020110613201970000_10.11.e038898.4","doi-asserted-by":"crossref","DOI":"10.1016\/j.pec.2020.06.028","article-title":"The associations between reflective ability and communication skills among medical students","author":"Karnieli-Miller","year":"2020","journal-title":"Patient Educ Couns"},{"key":"2020110613201970000_10.11.e038898.5","doi-asserted-by":"crossref","first-page":"730","DOI":"10.1097\/ACM.0000000000001090","article-title":"Tools to assess behavioral and social science competencies in medical education: a systematic review","volume":"91","author":"Carney","year":"2016","journal-title":"Acad Med"},{"key":"2020110613201970000_10.11.e038898.6","doi-asserted-by":"crossref","first-page":"1287","DOI":"10.1016\/j.pec.2015.08.011","article-title":"Communication training: skills and beyond","volume":"98","author":"Deveugele","year":"2015","journal-title":"Patient Educ Couns"},{"key":"2020110613201970000_10.11.e038898.7","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1016\/j.pec.2015.11.024","article-title":"A teaching skills assessment tool inspired by the Calgary-Cambridge model and the patient-centered approach","volume":"99","author":"Sommer","year":"2016","journal-title":"Patient Educ Couns"},{"key":"2020110613201970000_10.11.e038898.8","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2929.2004.01917.x"},{"key":"2020110613201970000_10.11.e038898.9","doi-asserted-by":"crossref","first-page":"1176","DOI":"10.1111\/medu.12583","article-title":"What is reflection? A conceptual analysis of major definitions and a proposal of a five-component model","volume":"48","author":"Nguyen","year":"2014","journal-title":"Med Educ"},{"key":"2020110613201970000_10.11.e038898.10","unstructured":"Sch\u00f6n DA . The reflective practitioner: how professionals think in action, 1985."},{"key":"2020110613201970000_10.11.e038898.11","doi-asserted-by":"crossref","first-page":"2166","DOI":"10.1016\/j.pec.2020.06.021","article-title":"Reflective practice in the teaching of communication skills","volume":"103","author":"Karnieli-Miller","year":"2020","journal-title":"Patient Educ Couns"},{"key":"2020110613201970000_10.11.e038898.12","doi-asserted-by":"publisher","DOI":"10.1007\/s10912-006-9020-y"},{"key":"2020110613201970000_10.11.e038898.13","first-page":"7","article-title":"What makes a good reflective paper?","volume":"45","author":"Walling","year":"2013","journal-title":"Fam Med"},{"key":"2020110613201970000_10.11.e038898.14","doi-asserted-by":"crossref","DOI":"10.3352\/jeehp.2014.11.20","article-title":"Reflective writing and its impact on empathy in medical education: systematic review","volume":"11","author":"Chen","year":"2014","journal-title":"J Educ Eval Health Prof"},{"key":"2020110613201970000_10.11.e038898.15","doi-asserted-by":"crossref","first-page":"602","DOI":"10.3109\/0142159X.2014.899686","article-title":"Scaffolding reflective learning in clinical practice: a comparison of two types of reflective activities","volume":"36","author":"Embo","year":"2014","journal-title":"Med Teach"},{"key":"2020110613201970000_10.11.e038898.16","doi-asserted-by":"crossref","first-page":"572","DOI":"10.3109\/09593985.2014.928919","article-title":"Arts, literature and reflective writing as educational strategies to promote narrative reasoning capabilities among physiotherapy students","volume":"30","author":"Caeiro","year":"2014","journal-title":"Physiother Theory Pract"},{"key":"2020110613201970000_10.11.e038898.17","doi-asserted-by":"crossref","DOI":"10.7812\/TPP\/17-013","article-title":"The art of healing through narrative medicine in clinical practice: a reflection","volume":"21","author":"Muneeb","year":"2017","journal-title":"Perm J"},{"key":"2020110613201970000_10.11.e038898.18","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1007\/s40037-016-0293-1","article-title":"Using reflection to influence practice: student perceptions of daily reflection in clinical education","volume":"5","author":"Larsen","year":"2016","journal-title":"Perspect Med Educ"},{"key":"2020110613201970000_10.11.e038898.19","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1111\/medu.13582","article-title":"See, reflect, learn more: qualitative analysis of breaking bad news reflective narratives","volume":"52","author":"Karnieli-Miller","year":"2018","journal-title":"Med Educ"},{"key":"2020110613201970000_10.11.e038898.20","doi-asserted-by":"crossref","first-page":"1645","DOI":"10.1016\/j.pec.2018.04.004","article-title":"What you \u201csee\u201d is how you communicate: Medical students\u2019 meaning making of a patient\u2019s vignette","volume":"101","author":"Karnieli-Miller","year":"2018","journal-title":"Patient Educ Couns"},{"key":"2020110613201970000_10.11.e038898.21","first-page":"775","article-title":"Finding the words: medical students\u2019 reflections on communication challenges in clinic","volume":"48","author":"Braverman","year":"2016","journal-title":"Fam Med"},{"key":"2020110613201970000_10.11.e038898.22","doi-asserted-by":"publisher","DOI":"10.1097\/ACM.0b013e31823b55fa"},{"key":"2020110613201970000_10.11.e038898.23","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1080\/0142159X.2018.1505037","article-title":"A best evidence in medical education systematic review to determine the most effective teaching methods that develop reflection in medical students: BEME guide No. 51","volume":"41","author":"Uygur","year":"2019","journal-title":"Med Teach"},{"key":"2020110613201970000_10.11.e038898.24","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.amsu.2018.10.005","article-title":"Developing a novel framework for non-technical skills learning strategies for undergraduates: a systematic review","volume":"36","author":"Nicolaides","year":"2018","journal-title":"Ann Med Surg"},{"key":"2020110613201970000_10.11.e038898.25","doi-asserted-by":"crossref","DOI":"10.1186\/1472-6920-14-10","article-title":"National survey of clinical communication assessment in medical education in the United Kingdom (UK)","volume":"14","author":"Laidlaw","year":"2014","journal-title":"BMC Med Educ"},{"key":"2020110613201970000_10.11.e038898.26","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2929.2006.02673.x"},{"key":"2020110613201970000_10.11.e038898.27","doi-asserted-by":"crossref","unstructured":"Locklear W , Meires J , Quallich SA . Letters to the editor. Urol Nurs 2018;38:307. doi:10.7257\/1053-816X.2018.38.6.307","DOI":"10.7257\/1053-816X.2018.38.6.307"},{"key":"2020110613201970000_10.11.e038898.28","first-page":"745","article-title":"Evaluation of a communication skills training course for medical students using peer role-play","volume":"67","author":"Ayuob","year":"2017","journal-title":"J Pak Med Assoc"},{"key":"2020110613201970000_10.11.e038898.29","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1097\/ACM.0000000000001420","article-title":"From communication skills to Skillful communication: a longitudinal integrated curriculum for critical care medicine fellows","volume":"92","author":"Roze des Ordons","year":"2017","journal-title":"Acad Med"},{"key":"2020110613201970000_10.11.e038898.30","doi-asserted-by":"crossref","first-page":"212","DOI":"10.5116\/ijme.5454.c665","article-title":"Psychometric properties of the Calgary Cambridge guides to assess communication skills of undergraduate medical students","volume":"5","author":"Simmenroth-Nayda","year":"2014","journal-title":"Int J Med Educ"},{"key":"2020110613201970000_10.11.e038898.31","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2923.1996.tb00724.x"},{"key":"2020110613201970000_10.11.e038898.32","unstructured":"Stewart M , Brown JB , Weston WW , et al . Patient-centered medicine: transforming the clinical method. 3rd edn. London: Radcliffe Publishing, 2014."},{"key":"2020110613201970000_10.11.e038898.33","unstructured":"Gibbs G . Learning by doing: A guide to teaching and learning methods. London: FEU, 1988."},{"key":"2020110613201970000_10.11.e038898.34","doi-asserted-by":"crossref","first-page":"809","DOI":"10.20344\/amp.8393","article-title":"A brief course on clinical communication skills: a multi- centered study","volume":"29","author":"Franco","year":"2016","journal-title":"Acta Med Port"},{"key":"2020110613201970000_10.11.e038898.35","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1080\/02602930802062659","article-title":"Validity and reliability of portfolio assessment in pre\u2010service teacher education","volume":"34","author":"Meeus","year":"2009","journal-title":"Assess Eval High Educ"},{"key":"2020110613201970000_10.11.e038898.36","doi-asserted-by":"publisher","DOI":"10.1097\/00001888-200104000-00021"},{"key":"2020110613201970000_10.11.e038898.37","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1590\/1981-52712015v39n3e02832014","article-title":"OSCE para Compet\u00eancias de Comunica\u00e7\u00e3o Cl\u00ednica E Profissionalismo: Relato de Experi\u00eancia E Meta-Avalia\u00e7\u00e3o","volume":"39","author":"Franco","year":"2015","journal-title":"Rev Bras Educ Med"},{"key":"2020110613201970000_10.11.e038898.38","doi-asserted-by":"publisher","DOI":"10.1016\/j.ambp.2007.04.006"},{"key":"2020110613201970000_10.11.e038898.39","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1080\/07294360.2013.841650","article-title":"Multimodal approaches to reflective teaching and assessment in higher education","volume":"33","author":"Barton","year":"2014","journal-title":"High Educ Res Develop"},{"key":"2020110613201970000_10.11.e038898.40","doi-asserted-by":"publisher","DOI":"10.1016\/j.pec.2008.12.001"},{"key":"2020110613201970000_10.11.e038898.41","doi-asserted-by":"publisher","DOI":"10.1097\/00001888-199009000-00045"},{"key":"2020110613201970000_10.11.e038898.42","doi-asserted-by":"crossref","first-page":"e1197","DOI":"10.3109\/0142159X.2013.788789","article-title":"Frameworks for learner assessment in medicine: AMEE guide No. 78","volume":"35","author":"Pangaro","year":"2013","journal-title":"Med Teach"},{"key":"2020110613201970000_10.11.e038898.43","doi-asserted-by":"crossref","first-page":"e1561","DOI":"10.3109\/0142159X.2013.828153","article-title":"Adult learning theories: implications for learning and teaching in medical education: AMEE guide No. 83","volume":"35","author":"Taylor","year":"2013","journal-title":"Med Teach"},{"key":"2020110613201970000_10.11.e038898.44","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1111\/medu.12771","article-title":"Considerations in the use of reflective writing for student assessment: issues of reliability and validity","volume":"49","author":"Moniz","year":"2015","journal-title":"Med Educ"},{"key":"2020110613201970000_10.11.e038898.45","doi-asserted-by":"crossref","first-page":"410","DOI":"10.1080\/10401334.2015.1077124","article-title":"Teaching medical students to reflect more deeply","volume":"27","author":"Hayton","year":"2015","journal-title":"Teach Learn Med"},{"key":"2020110613201970000_10.11.e038898.46","doi-asserted-by":"publisher","DOI":"10.1097\/ACM.0b013e31824d22e9"},{"key":"2020110613201970000_10.11.e038898.47","unstructured":"Manca A , Gormley GJ , Johnston JL , et al . Honoring medicine\u2019s social contract. Acad Med 2019;1."},{"key":"2020110613201970000_10.11.e038898.48","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2923.2012.04217.x"},{"key":"2020110613201970000_10.11.e038898.49","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1111\/j.1365-2923.1994.tb02690.x","article-title":"Portfolio learning: a proposal for undergraduate cancer teaching","volume":"28","author":"Finlay","year":"1994","journal-title":"Med Educ"},{"key":"2020110613201970000_10.11.e038898.50","doi-asserted-by":"publisher","DOI":"10.1097\/ACM.0000000000000913"},{"key":"2020110613201970000_10.11.e038898.51","doi-asserted-by":"crossref","first-page":"1253","DOI":"10.1111\/medu.13101","article-title":"A fresh look at Miller\u2019s pyramid: assessment at the \u2018Is\u2019 and \u2018Do\u2019 levels","volume":"50","author":"Al-Eraky","year":"2016","journal-title":"Med Educ"},{"key":"2020110613201970000_10.11.e038898.52","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1007\/s40037-015-0233-5","article-title":"Competence-based education misses the essence of the medical profession","volume":"4","author":"Lombarts","year":"2015","journal-title":"Perspect Med Educ"},{"key":"2020110613201970000_10.11.e038898.53","doi-asserted-by":"crossref","first-page":"440","DOI":"10.4236\/ce.2015.64044","article-title":"Best practice for teaching and learning strategies to facilitate student reflection in Pre-Registration health professional education: an integrative review","volume":"06","author":"McLeod","year":"2015","journal-title":"Creat Educ"},{"key":"2020110613201970000_10.11.e038898.54","doi-asserted-by":"publisher","DOI":"10.1136\/archdischild-2013-303948"},{"key":"2020110613201970000_10.11.e038898.55","doi-asserted-by":"publisher","DOI":"10.1007\/s10459-007-9090-2"},{"key":"2020110613201970000_10.11.e038898.56","doi-asserted-by":"crossref","first-page":"704","DOI":"10.1111\/medu.13058","article-title":"The challenge of changing to an assessment for learning culture","volume":"50","author":"Harrison","year":"2016","journal-title":"Med Educ"},{"key":"2020110613201970000_10.11.e038898.57","doi-asserted-by":"publisher","DOI":"10.1111\/medu.12680"},{"key":"2020110613201970000_10.11.e038898.58","doi-asserted-by":"crossref","first-page":"742","DOI":"10.1097\/00001888-200307000-00018","article-title":"Promoting reflection on professionalism: a comparison trial of educational interventions for medical students","volume":"78","author":"Baernstein","year":"2003","journal-title":"Acad Med"},{"key":"2020110613201970000_10.11.e038898.59","doi-asserted-by":"crossref","first-page":"877","DOI":"10.1080\/01421590802263490","article-title":"Reflective learning and the net generation","volume":"30","author":"Sandars","year":"2008","journal-title":"Med Teach"},{"key":"2020110613201970000_10.11.e038898.60","doi-asserted-by":"publisher","DOI":"10.5688\/ajpe8118"}],"container-title":["BMJ Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjopen-2020-038898","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,27]],"date-time":"2022-11-27T04:32:14Z","timestamp":1669523534000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmjopen.bmj.com\/lookup\/doi\/10.1136\/bmjopen-2020-038898"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11]]},"references-count":60,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2020,11,6]]},"published-print":{"date-parts":[[2020,11]]}},"alternative-id":["10.1136\/bmjopen-2020-038898"],"URL":"https:\/\/doi.org\/10.1136\/bmjopen-2020-038898","relation":{},"ISSN":["2044-6055","2044-6055"],"issn-type":[{"value":"2044-6055","type":"print"},{"value":"2044-6055","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11]]}}}