{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T11:43:43Z","timestamp":1782819823290,"version":"3.54.5"},"reference-count":50,"publisher":"Ubiquity Press, Ltd.","issue":"1","license":[{"start":{"date-parts":[[2023,12,22]],"date-time":"2023-12-22T00:00:00Z","timestamp":1703203200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Research Foundation Flanders (FWO, Strategic Basic Research (SBO))","award":["S003219N"],"award-info":[{"award-number":["S003219N"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Perspect Med Educ"],"abstract":"<jats:p>Introduction: Competency-based education requires high-quality feedback to guide students\u2019 acquisition of competencies. Sound assessment and feedback systems, such as ePortfolios, are needed to facilitate seeking and giving feedback during clinical placements. However, it is unclear whether the written feedback comments in ePortfolios are of high quality and aligned with the current competency focus. Therefore, this study investigates the quality of written feedback comments in ePortfolios of healthcare students, as well as how these feedback comments align with the CanMEDS roles.\nMethods: A qualitative textual analysis was conducted. 2,349 written feedback comments retrieved from the ePortfolios of 149 healthcare students (specialist medicine, general practice, occupational therapy, speech therapy and midwifery) were analysed retrospectively using deductive content analysis. Two structured categorisation matrices, one based on four literature-derived feedback quality criteria (performance, judgment, elaboration and improvement) and another one on the seven CanMEDS roles (Medical Expert, Communicator, Collaborator, Leader, Health Advocate, Scholar and Professional), guided the analysis.\nResults: The minority of the feedback comments (n = 352; 14.9%) could be considered of high quality because they met all four quality criteria. Most feedback comments were of moderate quality and met only two to three quality criteria. Regarding the CanMEDS roles, the Medical Expert role was most frequently represented in the feedback comments, as opposed to the roles Leader and Health Advocate.\nDiscussion: The results highlighted that providing high-quality feedback is challenging. To respond to these challenges, it is recommended to set up individual and continuous feedback training.<\/jats:p>","DOI":"10.5334\/pme.1050","type":"journal-article","created":{"date-parts":[[2023,12,22]],"date-time":"2023-12-22T07:55:39Z","timestamp":1703231739000},"source":"Crossref","is-referenced-by-count":6,"title":["A Qualitative Textual Analysis of Feedback Comments in ePortfolios: Quality and Alignment with the CanMEDS Roles"],"prefix":"10.5334","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4435-1462","authenticated-orcid":false,"given":"Sofie","family":"Van Ostaeyen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2915-6987","authenticated-orcid":false,"given":"Mieke","family":"Embo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7946-2096","authenticated-orcid":false,"given":"Tijs","family":"Rotsaert","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6090-5552","authenticated-orcid":false,"given":"Orph\u00e9e","family":"De Clercq","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3615-7347","authenticated-orcid":false,"given":"Tammy","family":"Schellens","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9544-4197","authenticated-orcid":false,"given":"Martin","family":"Valcke","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"3285","published-online":{"date-parts":[[2023,12,22]]},"reference":[{"issue":"6","key":"key20231222075520_B1","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1080\/0142159X.2017.1315067","article-title":"A call to action: the controversy of and rationale for competency-based medical education","volume":"39","year":"2017","journal-title":"Med Teach"},{"issue":"260","key":"key20231222075520_B2","first-page":"1","article-title":"An online Delphi study to investigate the completeness of the CanMEDS roles and the relevance, formulation, and measurability of their key competencies within eight healthcare disciplines in Flanders","volume":"22","year":"2022","journal-title":"BMC Med Educ"},{"key":"key20231222075520_B3","first-page":"17","volume-title":"Educational design a CanMEDS guide for the health professions","year":"2011"},{"issue":"1","key":"key20231222075520_B4","first-page":"53","article-title":"A clinical education and practice placements in the allied health professions: an international perspective","volume":"37","year":"2008","journal-title":"J Allied Health"},{"issue":"3","key":"key20231222075520_B5","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1097\/ACM.0000000000000129","article-title":"How clerkship students learn from real patients in practice settings","volume":"89","year":"2014","journal-title":"Acad Med"},{"issue":"8","key":"key20231222075520_B6","doi-asserted-by":"crossref","first-page":"676","DOI":"10.3109\/0142159X.2010.500704","article-title":"The role of assessment in competency-based medical education","volume":"32","year":"2010","journal-title":"Med Teach"},{"key":"key20231222075520_B7","first-page":"1","article-title":"The role of ePortfolios in supporting learning in eight healthcare disciplines: a scoping review","volume":"63","year":"2022","journal-title":"Nurse Educ Pract"},{"issue":"2","key":"key20231222075520_B8","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1111\/j.1365-2923.2007.02973.x","article-title":"What is feedback in clinical education?","volume":"42","year":"2008","journal-title":"Med Educ"},{"issue":"7","key":"key20231222075520_B9","first-page":"81","article-title":"Numbers encapsulate, words elaborate: toward the best use of comments for assessment and feedback on entrustment ratings","volume":"96","year":"2021","journal-title":"Acad Med"},{"issue":"1","key":"key20231222075520_B10","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1007\/s13187-018-1456-z","article-title":"Exploring how the new entrustable professional activity assessment tools affect the quality of feedback given to medical oncology residents","volume":"35","year":"2020","journal-title":"J Cancer Educ"},{"issue":"3","key":"key20231222075520_B11","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1177\/0163278718820415","article-title":"Examining the effects of narrative commentary on evaluators\u2019 summative assessments of resident performance","volume":"43","year":"2020","journal-title":"Eval Heal Prof"},{"issue":"7","key":"key20231222075520_B12","doi-asserted-by":"crossref","first-page":"738","DOI":"10.3109\/0142159X.2015.1075649","article-title":"A feedback system in residency to evaluate CanMEDS roles and provide high-quality feedback: exploring its application","volume":"38","year":"2016","journal-title":"Med Teach"},{"issue":"1","key":"key20231222075520_B13","first-page":"1","article-title":"\u2018I did not check if the teacher gave feedback\u2019: a qualitative analysis of Taiwanese postgraduate year 1 trainees\u2019 talk around e-portfolio feedback-seeking behaviours","volume":"9","year":"2019","journal-title":"BMJ Open"},{"issue":"10","key":"key20231222075520_B14","first-page":"106","article-title":"The quality of written comments on professional behaviors in a developmental multisource feedback program","volume":"85","year":"2010","journal-title":"Acad Med"},{"issue":"3","key":"key20231222075520_B15","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1111\/medu.12637","article-title":"Reading between the lines: faculty interpretations of narrative evaluation comments","volume":"49","year":"2015","journal-title":"Med Educ"},{"issue":"11","key":"key20231222075520_B16","doi-asserted-by":"crossref","first-page":"1617","DOI":"10.1097\/ACM.0000000000001669","article-title":"The hidden value of narrative comments for assessment: a quantitative reliability analysis of qualitative data","volume":"92","year":"2017","journal-title":"Acad Med"},{"issue":"1","key":"key20231222075520_B17","doi-asserted-by":"crossref","first-page":"81","DOI":"10.3102\/003465430298487","article-title":"The power of feedback","volume":"77","year":"2007","journal-title":"Rev. Educ. Res"},{"issue":"6","key":"key20231222075520_B18","doi-asserted-by":"crossref","first-page":"560","DOI":"10.1111\/medu.12678","article-title":"A contemporary approach to validity arguments: A practical guide to Kane\u2019s framework","volume":"49","year":"2015","journal-title":"Med Educ"},{"issue":"409","key":"key20231222075520_B19","first-page":"1","article-title":"Shaping the right conditions in programmatic assessment: how quality of narrative information affects the quality of high-stakes decision-making","volume":"22","year":"2022","journal-title":"BMC Med Educ"},{"key":"key20231222075520_B20","doi-asserted-by":"crossref","unstructured":"Ginsburg S. Hidden in plain sight: the untapped potential of written assessment comments. PhD thesis. Maastricht University, Maastricht. 2016. DOI: 10.26481\/dis.20160901sg","DOI":"10.26481\/dis.20160901sg"},{"issue":"133","key":"key20231222075520_B21","first-page":"1","article-title":"Profiling postgraduate workplace-based assessment implementation in Ireland: a retrospective cohort study","volume":"5","year":"2016","journal-title":"Springerplus"},{"issue":"278","key":"key20231222075520_B22","first-page":"1","article-title":"Feedback using an ePortfolio for medicine long cases: quality not quantity","volume":"16","year":"2016","journal-title":"BMC Med Educ"},{"issue":"4","key":"key20231222075520_B23","first-page":"303","article-title":"Evaluation of feedback given to trainees in medical specialties","volume":"17","year":"2017","journal-title":"Clin Med J R Coll Physicians London"},{"issue":"4","key":"key20231222075520_B24","doi-asserted-by":"crossref","first-page":"372","DOI":"10.1111\/j.1365-2923.2011.04184.x","article-title":"Working with words: exploring textual analysis in medical education research","volume":"46","year":"2012","journal-title":"Med Educ"},{"issue":"5","key":"key20231222075520_B25","doi-asserted-by":"crossref","first-page":"26","DOI":"10.5430\/ijhe.v6n5p26","article-title":"Understanding and Applying Research Paradigms in Educational Contexts","volume":"6","year":"2017","journal-title":"Int. J. High. Educ"},{"issue":"1","key":"key20231222075520_B26","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1097\/00002800-198800210-00025","article-title":"Content analysis: process and application","volume":"2","year":"1988","journal-title":"Clin nurse Spec"},{"issue":"1","key":"key20231222075520_B27","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1111\/j.1365-2648.2007.04569.x","article-title":"The qualitative content analysis process","volume":"62","year":"2008","journal-title":"J Adv Nurs"},{"key":"key20231222075520_B28","first-page":"5","article-title":"The INCEpTION Platform: machine-assisted and knowledge-oriented interactive annotation","year":"2018"},{"key":"key20231222075520_B29","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales","volume":"20","year":"1960","journal-title":"Educ Psychol Meas"},{"issue":"6","key":"key20231222075520_B30","doi-asserted-by":"crossref","first-page":"929","DOI":"10.1007\/s11606-019-04946-3","article-title":"Milestone implementation\u2019s impact on narrative comments and perception of feedback for internal medicine residents: a mixed methods study","volume":"34","year":"2019","journal-title":"J Gen Intern Med"},{"issue":"1","key":"key20231222075520_B31","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1108\/03090591311293310","article-title":"Quality of feedback following performance assessments: does assessor expertise matter?","volume":"37","year":"2013","journal-title":"Eur J Train Dev"},{"issue":"6","key":"key20231222075520_B32","doi-asserted-by":"crossref","first-page":"2070","DOI":"10.1016\/j.jsurg.2021.06.012","article-title":"Comparing the quality of narrative comments by rotation setting","volume":"78","year":"2021","journal-title":"J Surg Educ"},{"issue":"1","key":"key20231222075520_B33","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1097\/CEH.0000000000000322","article-title":"Experiences of Faculty Members Giving Corrective Feedback to Medical Trainees in a Clinical Setting","volume":"41","year":"2021","journal-title":"J Contin Educ Health Prof"},{"key":"key20231222075520_B34","first-page":"1","article-title":"Sharing perspectives on feedback: a combined resident-faculty workshop","volume":"22","year":"2022","journal-title":"BMC Med Educ"},{"issue":"6","key":"key20231222075520_B35","doi-asserted-by":"crossref","first-page":"628","DOI":"10.4103\/1735-9066.197671","article-title":"Feedback in the nonshifting context of the midwifery clinical education in Indonesia: A mixed methods study","volume":"21","year":"2016","journal-title":"Iran J Nurs Midwifery Res"},{"issue":"4","key":"key20231222075520_B36","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1097\/ACM.0000000000001927","article-title":"The feedback tango: An integrative review and analysis of the content of the teacher-learner feedback exchange","volume":"93","year":"2018","journal-title":"Acad Med"},{"issue":"3","key":"key20231222075520_B37","doi-asserted-by":"crossref","first-page":"1069","DOI":"10.1002\/nop2.717","article-title":"Challenges in the assessment of nursing students in clinical placements: Exploring perceptions among nurse mentors","volume":"8","year":"2021","journal-title":"Nurs Open"},{"key":"key20231222075520_B38","doi-asserted-by":"crossref","first-page":"1061","DOI":"10.2147\/AMEP.S367129","article-title":"Perceptions of the CanMEDS competencies of faculty and students in different curriculum systems of a medical school in China","volume":"13","year":"2022","journal-title":"Adv Med Educ Pract"},{"issue":"8","key":"key20231222075520_B39","doi-asserted-by":"crossref","first-page":"883","DOI":"10.1080\/0142159X.2017.1309377","article-title":"Shedding light into the black box: A prospective longitudinal study identifying the CanMEDS roles of final year medical students\u2019 on-ward activities","volume":"39","year":"2017","journal-title":"Med Teach"},{"issue":"4","key":"key20231222075520_B40","first-page":"39","article-title":"Not wanted on the voyage: highlighting intrinsic CanMEDS gaps in Competence by Design curricula","volume":"12","year":"2021","journal-title":"Can. Med. Educ. J"},{"issue":"100","key":"key20231222075520_B41","first-page":"1","article-title":"Designing faculty development to support the evaluation of resident competency in the intrinsic CanMEDS roles: practical outcomes of an assessment of program director needs","volume":"15","year":"2015","journal-title":"BMC Med Educ"},{"issue":"9","key":"key20231222075520_B42","doi-asserted-by":"crossref","first-page":"942","DOI":"10.1111\/medu.13335","article-title":"Integrated and implicit: how residents learn CanMEDS roles by participating in practice","volume":"51","year":"2017","journal-title":"Med Educ"},{"key":"key20231222075520_B43","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1007\/s10459-015-9634-9","article-title":"What supervisors say in their feedback: construction of CanMEDS roles in workplace settings","volume":"21","year":"2016","journal-title":"Adv in Health Sci Educ"},{"issue":"11","key":"key20231222075520_B44","doi-asserted-by":"crossref","first-page":"e725","DOI":"10.3109\/0142159X.2012.689444","article-title":"Quality evaluation reports: can a faculty development program make a difference?","volume":"34","year":"2012","journal-title":"Med Teach"},{"issue":"97","key":"key20231222075520_B45","first-page":"1","article-title":"Quality of written narrative feedback and reflection in a modified mini-clinical evaluation exercise: an observational study","volume":"12","year":"2012","journal-title":"BMC Med Educ"},{"issue":"4","key":"key20231222075520_B46","doi-asserted-by":"crossref","first-page":"921","DOI":"10.1007\/s40670-022-01583-0","article-title":"Training to support ePortfolio users during clinical placements: a scoping review","volume":"32","year":"2022","journal-title":"Med Sci Educ"},{"issue":"10","key":"key20231222075520_B47","doi-asserted-by":"crossref","first-page":"1457","DOI":"10.1097\/ACM.0000000000004153","article-title":"Using natural language processing to automatically assess feedback quality: findings from 3 surgical residencies","volume":"96","year":"2021","journal-title":"Acad Med"},{"issue":"58","key":"key20231222075520_B48","first-page":"1","article-title":"Using natural language processing to determine factors associated with high-quality feedback","volume":"1","year":"2022","journal-title":"Glob Surg Educ \u2013 J Assoc Surg Educ"},{"issue":"3","key":"key20231222075520_B49","first-page":"1","article-title":"The ePortfolio in UK cardiology training: time for a new digital platform?","volume":"28","year":"2021","journal-title":"Br J Cardiol"},{"issue":"12","key":"key20231222075520_B50","doi-asserted-by":"crossref","first-page":"1223","DOI":"10.1111\/medu.14911","article-title":"Predictors of faculty narrative evaluation quality in medical school clerkships","volume":"56","year":"2022","journal-title":"Med Educ"}],"container-title":["Perspectives on Medical Education"],"original-title":[],"link":[{"URL":"https:\/\/account.pmejournal.org\/index.php\/up-j-pme\/article\/download\/1050\/1616","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/account.pmejournal.org\/index.php\/up-j-pme\/article\/download\/1050\/1617","content-type":"text\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/account.pmejournal.org\/index.php\/up-j-pme\/article\/download\/1050\/1618","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/account.pmejournal.org\/index.php\/up-j-pme\/article\/download\/1050\/1619","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/account.pmejournal.org\/index.php\/up-j-pme\/article\/download\/1050\/1620","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/account.pmejournal.org\/index.php\/up-j-pme\/article\/download\/1050\/1616","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T10:44:07Z","timestamp":1782816247000},"score":1,"resource":{"primary":{"URL":"https:\/\/account.pmejournal.org\/index.php\/up-j-pme\/article\/view\/1050"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,22]]},"references-count":50,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1,4]]}},"URL":"https:\/\/doi.org\/10.5334\/pme.1050","relation":{},"ISSN":["2212-277X"],"issn-type":[{"value":"2212-277X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,22]]}}}